diff --git a/CHANGES b/CHANGES index 318ab3d38d..b240ad7772 100644 --- a/CHANGES +++ b/CHANGES @@ -1,4 +1,331 @@ +2.1-56 | 2012-10-03 16:04:52 -0700 + + * Add general FAQ entry about upgrading Bro. (Jon Siwek) + +2.1-53 | 2012-10-03 16:00:40 -0700 + + * Add new Tunnel::delay_teredo_confirmation option that indicates + that the Teredo analyzer should wait until it sees both sides of a + connection using a valid Teredo encapsulation before issuing a + protocol_confirmation. Default is on. Addresses #890. (Jon Siwek) + +2.1-50 | 2012-10-02 12:06:08 -0700 + + * Fix a typing issue that prevented the ElasticSearch timeout to + work. (Matthias Vallentin) + + * Use second granularity for ElasticSearch timeouts. (Matthias + Vallentin) + + * Fix compile issues with older versions of libcurl, which don't + offer *_MS timeout constants. (Matthias Vallentin) + +2.1-47 | 2012-10-02 11:59:29 -0700 + + * Fix for the input framework: BroStrings were constructed without a + final \0, which makes them unusable by basically all internal + functions (like to_count). (Bernhard Amann) + + * Remove deprecated script functionality (see NEWS for details). + (Daniel Thayer) + +2.1-39 | 2012-09-29 14:09:16 -0700 + + * Reliability adjustments to istate tests with network + communication. (Jon Siwek) + +2.1-37 | 2012-09-25 14:21:37 -0700 + + * Reenable some tests that previously would cause Bro to exit with + an error. (Daniel Thayer) + + * Fix parsing of large integers on 32-bit systems. (Daniel Thayer) + + * Serialize language.when unit test with the "comm" group. (Jon + Siwek) + +2.1-32 | 2012-09-24 16:24:34 -0700 + + * Fix race condition in language/when.bro test. (Daniel Thayer) + +2.1-26 | 2012-09-23 08:46:03 -0700 + + * Add an item to FAQ page about broctl options. (Daniel Thayer) + + * Add more language tests. We now have tests of all built-in Bro + data types (including different representations of constant + values, and max./min. values), keywords, and operators (including + special properties of certain operators, such as short-circuit + evaluation and associativity). (Daniel Thayer) + + * Fix construction of ip6_ah (Authentication Header) record values. + + Authentication Headers with a Payload Len field set to zero would + cause a crash due to invalid memory allocation because the + previous code assumed Payload Len would always be great enough to + contain all mandatory fields of the header. (Jon Siwek) + + * Update compile/dependency docs for OS X. (Jon Siwek) + + * Adjusting Mac binary packaging script. Setting CMAKE_PREFIX_PATH + helps link against standard system libs instead of ones that come + from other package manager (e.g. MacPorts). (Jon Siwek) + + * Adjusting some unit tests that do cluster communication. (Jon Siwek) + + * Small change to non-blocking DNS initialization. (Jon Siwek) + + * Reorder a few statements in scan.l to make 1.5msecs etc work. + Adresses #872. (Bernhard Amann) + +2.1-6 | 2012-09-06 23:23:14 -0700 + + * Fixed a bug where "a -= b" (both operands are intervals) was not + allowed in Bro scripts (although "a = a - b" is allowed). (Daniel + Thayer) + + * Fixed a bug where the "!=" operator with subnet operands was + treated the same as the "==" operator. (Daniel Thayer) + + * Add sleeps to configuration_update test for better reliability. + (Jon Siwek) + + * Fix a segfault when iterating over a set when using malformed + index. (Daniel Thayer) + +2.1 | 2012-08-28 16:46:42 -0700 + + * Make bif.identify_magic robust against FreeBSD's libmagic config. + (Robin Sommer) + + * Remove automatic use of gperftools on non-Linux systems. + --enable-perftools must now explicity be supplied to ./configure + on non-Linux systems to link against the tcmalloc library. + + * Fix uninitialized value for 'is_partial' in TCP analyzer. (Jon + Siwek) + + * Parse 64-bit consts in Bro scripts correctly. (Bernhard Amann) + + * Output 64-bit counts correctly on 32-bit machines (Bernhard Amann) + + * Input framework fixes, including: (Bernhard Amann) + + - One of the change events got the wrong parameters. + + - Escape commas in sets and vectors that were unescaped before + tokenization. + + - Handling of zero-length-strings as last element in a set was + broken (sets ending with a ,). + + - Hashing of lines just containing zero-length-strings was broken. + + - Make set_separators different from , work for input framework. + + - Input framework was not handling counts and ints out of + 32-bit-range correctly. + + - Errors in single lines do not kill processing, but simply ignore + the line, log it, and continue. + + * Update documentation for builtin types. (Daniel Thayer) + + - Add missing description of interval "msec" unit. + + - Improved description of pattern by clarifying the issue of + operand order and difference between exact and embedded + matching. + + * Documentation fixes for signature 'eval' conditions. (Jon Siwek) + + * Remove orphaned 1.5 unit tests. (Jon Siwek) + + * Add type checking for signature 'eval' condition functions. (Jon + Siwek) + + * Adding an identifier to the SMTP blocklist notices for duplicate + suppression. (Seth Hall) + +2.1-beta-45 | 2012-08-22 16:11:10 -0700 + + * Add an option to the input framework that allows the user to chose + to not die upon encountering files/functions. (Bernhard Amann) + +2.1-beta-41 | 2012-08-22 16:05:21 -0700 + + * Add test serialization to "leak" unit tests that use + communication. (Jon Siwek) + + * Change to metrics/basic-cluster unit test for reliability. (Jon + Siwek) + + * Fixed ack tracking which could overflow quickly in some + situations. (Seth Hall) + + * Minor tweak to coverage.bare-mode-errors unit test to work with a + symlinked 'scripts' dir. (Jon Siwek) + +2.1-beta-35 | 2012-08-22 08:44:52 -0700 + + * Add testcase for input framework reading sets (rather than + tables). (Bernhard Amann) + +2.1-beta-31 | 2012-08-21 15:46:05 -0700 + + * Tweak to rotate-custom.bro unit test. (Jon Siwek) + + * Ignore small mem leak every rotation interval for dataseries logs. + (Jon Siwek) + +2.1-beta-28 | 2012-08-21 08:32:42 -0700 + + * Linking ES docs into logging document. (Robin Sommer) + +2.1-beta-27 | 2012-08-20 20:06:20 -0700 + + * Add the Stream record to Log:active_streams to make more dynamic + logging possible. (Seth Hall) + + * Fix portability of printing to files returned by + open("/dev/stderr"). (Jon Siwek) + + * Fix mime type diff canonifier to also skip mime_desc columns. (Jon + Siwek) + + * Unit test tweaks/fixes. (Jon Siwek) + + - Some baselines for tests in "leaks" group were outdated. + + - Changed a few of the cluster/communication tests to terminate + more explicitly instead of relying on btest-bg-wait to kill + processes. This makes the tests finish faster in the success case + and makes the reason for failing clearer in the that case. + + * Fix memory leak of serialized IDs when compiled with + --enable-debug. (Jon Siwek) + +2.1-beta-21 | 2012-08-16 11:48:56 -0700 + + * Installing a handler for running out of memory in "new". Bro will + now print an error message in that case rather than abort with an + uncaught exception. (Robin Sommer) + +2.1-beta-20 | 2012-08-16 11:43:31 -0700 + + * Fixed potential problems with ElasticSearch output plugin. (Seth + Hall) + +2.1-beta-13 | 2012-08-10 12:28:04 -0700 + + * Reporter warnings and error now print to stderr by default. New + options Reporter::warnings_to_stderr and + Reporter::errors_to_stderr to disable. (Seth Hall) + +2.1-beta-9 | 2012-08-10 12:24:29 -0700 + + * Add more BIF tests. (Daniel Thayer) + +2.1-beta-6 | 2012-08-10 12:22:52 -0700 + + * Fix bug in input framework with an edge case. (Bernhard Amann) + + * Fix small bug in input framework test script. (Bernhard Amann) + +2.1-beta-3 | 2012-08-03 10:46:49 -0700 + + * Merge branch 'master' of ssh://git.bro-ids.org/bro (Robin Sommer) + + * Fix configure script to exit with non-zero status on error (Jon + Siwek) + + * Improve ASCII output performance. (Robin Sommer) + +2.1-beta | 2012-07-30 11:59:53 -0700 + + * Improve log filter compatibility with remote logging. Addresses + #842. (Jon Siwek) + +2.0-907 | 2012-07-30 09:13:36 -0700 + + * Add missing breaks to switch cases in + ElasticSearch::HTTPReceive(). (Jon Siwek) + +2.0-905 | 2012-07-28 16:24:34 -0700 + + * Fix log manager hanging on waiting for pending file rotations, + plus writer API tweak for failed rotations. Addresses #860. (Jon + Siwek and Robin Sommer) + + * Tweaking logs-to-elasticsearch.bro so that it doesn't do anything + if ES server is unset. (Robin Sommer) + +2.0-902 | 2012-07-27 12:42:13 -0700 + + * New variable in logging framework Log::active_streams to indicate + Log:ID enums which are currently active. (Seth Hall) + + * Reworked how the logs-to-elasticsearch scripts works to stop + abusing the logging framework. (Seth Hall) + + * Fix input test for recent default change on fastpath. (Robin + Sommer) + +2.0-898 | 2012-07-27 12:22:03 -0700 + + * Small (potential performance) improvement for logging framework. (Seth Hall) + + * Script-level rotation postprocessor fix. This fixes a problem with + writers that don't have a postprocessor. (Seth Hall) + + * Update input framework documentation to reflect want_record + change. (Bernhard Amann) + + * Fix crash when encountering an InterpreterException in a predicate + in logging or input Framework. (Bernhard Amann) + + * Input framework: Make want_record=T the default for events + (Bernhard Amann) + + * Changing the start/end markers in logs to open/close now + reflecting wall clock. (Robin Sommer) + +2.0-891 | 2012-07-26 17:15:10 -0700 + + * Reader/writer API: preventing plugins from receiving further + messages after a failure. (Robin Sommer) + + * New test for input framework that fails to find a file. (Robin + Sommer) + + * Improving error handling for threads. (Robin Sommer) + + * Tweaking the custom-rotate test to produce stable output. (Robin + Sommer) + +2.0-884 | 2012-07-26 14:33:21 -0700 + + * Add comprehensive error handling for close() calls. (Jon Siwek) + + * Add more test cases for input framework. (Bernhard Amann) + + * Input framework: make error output for non-matching event types + much more verbose. (Bernhard Amann) + +2.0-877 | 2012-07-25 17:20:34 -0700 + + * Fix double close() in FilerSerializer class. (Jon Siwek) + + * Fix build warnings. (Daniel Thayer) + + * Fixes to ElasticSearch plugin to make libcurl handle http + responses correctly. (Seth Hall) + + * Fixing FreeBSD compiler error. (Robin Sommer) + + * Silencing compiler warnings. (Robin Sommer) + 2.0-871 | 2012-07-25 13:08:00 -0700 * Fix complaint from valgrind about uninitialized memory usage. (Jon diff --git a/CMakeLists.txt b/CMakeLists.txt index f667c0cfe0..2c8a726a1a 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -88,24 +88,30 @@ if (LIBGEOIP_FOUND) list(APPEND OPTLIBS ${LibGeoIP_LIBRARY}) endif () -set(USE_PERFTOOLS false) +set(HAVE_PERFTOOLS false) set(USE_PERFTOOLS_DEBUG false) +set(USE_PERFTOOLS_TCMALLOC false) if (NOT DISABLE_PERFTOOLS) find_package(GooglePerftools) endif () if (GOOGLEPERFTOOLS_FOUND) - include_directories(BEFORE ${GooglePerftools_INCLUDE_DIR}) - set(USE_PERFTOOLS true) + set(HAVE_PERFTOOLS true) + # Non-Linux systems may not be well-supported by gperftools, so + # require explicit request from user to enable it in that case. + if (${CMAKE_SYSTEM_NAME} MATCHES "Linux" OR ENABLE_PERFTOOLS) + set(USE_PERFTOOLS_TCMALLOC true) - if (ENABLE_PERFTOOLS_DEBUG) - # Enable heap debugging with perftools. - set(USE_PERFTOOLS_DEBUG true) - list(APPEND OPTLIBS ${GooglePerftools_LIBRARIES_DEBUG}) - else () - # Link in tcmalloc for better performance. - list(APPEND OPTLIBS ${GooglePerftools_LIBRARIES}) + if (ENABLE_PERFTOOLS_DEBUG) + # Enable heap debugging with perftools. + set(USE_PERFTOOLS_DEBUG true) + include_directories(BEFORE ${GooglePerftools_INCLUDE_DIR}) + list(APPEND OPTLIBS ${GooglePerftools_LIBRARIES_DEBUG}) + else () + # Link in tcmalloc for better performance. + list(APPEND OPTLIBS ${GooglePerftools_LIBRARIES}) + endif () endif () endif () @@ -224,7 +230,8 @@ message( "\nAux. Tools: ${INSTALL_AUX_TOOLS}" "\n" "\nGeoIP: ${USE_GEOIP}" - "\nGoogle perftools: ${USE_PERFTOOLS}" + "\ngperftools found: ${HAVE_PERFTOOLS}" + "\n tcmalloc: ${USE_PERFTOOLS_TCMALLOC}" "\n debugging: ${USE_PERFTOOLS_DEBUG}" "\ncURL: ${USE_CURL}" "\n" diff --git a/NEWS b/NEWS index 00aeb62132..3b14f2c7ac 100644 --- a/NEWS +++ b/NEWS @@ -7,8 +7,33 @@ release. For a complete list of changes, see the ``CHANGES`` file (note that submodules, such as BroControl and Broccoli, come with their own CHANGES.) -Bro 2.1 Beta ------------- +Bro 2.2 +------- + +New Functionality +~~~~~~~~~~~~~~~~~ + +- TODO: Update. + +Changed Functionality +~~~~~~~~~~~~~~~~~~~~~ + +- We removed the following, already deprecated, functionality: + + * Scripting language: + - &disable_print_hook attribute. + + * BiF functions: + - parse_dotted_addr(), dump_config(), + make_connection_persistent(), generate_idmef(), + split_complete() + +- Removed a now unused argument from "do_split" helper function. + +- "this" is no longer a reserved keyword. + +Bro 2.1 +------- New Functionality ~~~~~~~~~~~~~~~~~ @@ -82,7 +107,8 @@ New Functionality * ElasticSearch: a distributed RESTful, storage engine and search engine built on top of Apache Lucene. It scales very well, both - for distributed indexing and distributed searching. + for distributed indexing and distributed searching. See + doc/logging-elasticsearch.rst for more information. Note that at this point, we consider Bro's support for these two formats as prototypes for collecting experience with alternative @@ -101,9 +127,14 @@ the full set. * Bro now requires CMake >= 2.6.3. - * Bro now links in tcmalloc (part of Google perftools) if found at - configure time. Doing so can significantly improve memory and - CPU use. + * On Linux, Bro now links in tcmalloc (part of Google perftools) + if found at configure time. Doing so can significantly improve + memory and CPU use. + + On the other platforms, the new configure option + --enable-perftools can be used to enable linking to tcmalloc. + (Note that perftools's support for non-Linux platforms may be + less reliable). - The configure switch --enable-brov6 is gone. @@ -152,14 +183,15 @@ the full set. understands. - ASCII logs now record the time when they were opened/closed at the - beginning and end of the file, respectively. The options - LogAscii::header_prefix and LogAscii::include_header have been - renamed to LogAscii::meta_prefix and LogAscii::include_meta, + beginning and end of the file, respectively (wall clock). The + options LogAscii::header_prefix and LogAscii::include_header have + been renamed to LogAscii::meta_prefix and LogAscii::include_meta, respectively. - The ASCII writers "header_*" options have been renamed to "meta_*" (because there's now also a footer). + Bro 2.0 ------- diff --git a/VERSION b/VERSION index b7a1951099..72d5fb372d 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -2.0-871 +2.1-56 diff --git a/aux/binpac b/aux/binpac index 4f01ea4081..a93ef13735 160000 --- a/aux/binpac +++ b/aux/binpac @@ -1 +1 @@ -Subproject commit 4f01ea40817ad232a96535c64fce7dc16d4e2fff +Subproject commit a93ef1373512c661ffcd0d0a61bd19b96667e0d5 diff --git a/aux/bro-aux b/aux/bro-aux index c691c01e9c..6748ec3a96 160000 --- a/aux/bro-aux +++ b/aux/bro-aux @@ -1 +1 @@ -Subproject commit c691c01e9cefae5a79bcd4b0f84ca387c8c587a7 +Subproject commit 6748ec3a96d582a977cd9114ef19c76fe75c57ff diff --git a/aux/broccoli b/aux/broccoli index 8234b8903c..ebfa4de45a 160000 --- a/aux/broccoli +++ b/aux/broccoli @@ -1 +1 @@ -Subproject commit 8234b8903cbc775f341bdb6a1c0159981d88d27b +Subproject commit ebfa4de45a839e58aec200e7e4bad33eaab4f1ed diff --git a/aux/broctl b/aux/broctl index 231358f166..b0e3c0d846 160000 --- a/aux/broctl +++ b/aux/broctl @@ -1 +1 @@ -Subproject commit 231358f166f61cc32201a8ac3671ea0c0f5c324e +Subproject commit b0e3c0d84643878c135dcb8a9774ed78147dd648 diff --git a/aux/btest b/aux/btest index 44441a6c91..44a43e6245 160000 --- a/aux/btest +++ b/aux/btest @@ -1 +1 @@ -Subproject commit 44441a6c912c7c9f8d4771e042306ec5f44e461d +Subproject commit 44a43e62452302277f88e8fac08d1f979dc53f98 diff --git a/cmake b/cmake index 2a72c5e08e..125f9a5fa8 160000 --- a/cmake +++ b/cmake @@ -1 +1 @@ -Subproject commit 2a72c5e08e018cf632033af3920432d5f684e130 +Subproject commit 125f9a5fa851381d0350efa41a4d14f27be263a2 diff --git a/configure b/configure index bfe54123f0..8e4aaa8425 100755 --- a/configure +++ b/configure @@ -1,7 +1,7 @@ #!/bin/sh # Convenience wrapper for easily viewing/setting options that # the project's CMake scripts will recognize - +set -e command="$0 $*" # check for `cmake` command @@ -29,6 +29,8 @@ Usage: $0 [OPTION]... [VAR=VALUE]... Optional Features: --enable-debug compile in debugging mode --enable-mobile-ipv6 analyze mobile IPv6 features defined by RFC 6275 + --enable-perftools force use of Google perftools on non-Linux systems + (automatically on when perftools is present on Linux) --enable-perftools-debug use Google's perftools for debugging --disable-broccoli don't build or install the Broccoli library --disable-broctl don't install Broctl @@ -98,6 +100,7 @@ append_cache_entry PY_MOD_INSTALL_DIR PATH $prefix/lib/broctl append_cache_entry BRO_SCRIPT_INSTALL_PATH STRING $prefix/share/bro append_cache_entry BRO_ETC_INSTALL_DIR PATH $prefix/etc append_cache_entry ENABLE_DEBUG BOOL false +append_cache_entry ENABLE_PERFTOOLS BOOL false append_cache_entry ENABLE_PERFTOOLS_DEBUG BOOL false append_cache_entry BinPAC_SKIP_INSTALL BOOL true append_cache_entry BUILD_SHARED_LIBS BOOL true @@ -146,7 +149,11 @@ while [ $# -ne 0 ]; do --enable-mobile-ipv6) append_cache_entry ENABLE_MOBILE_IPV6 BOOL true ;; + --enable-perftools) + append_cache_entry ENABLE_PERFTOOLS BOOL true + ;; --enable-perftools-debug) + append_cache_entry ENABLE_PERFTOOLS BOOL true append_cache_entry ENABLE_PERFTOOLS_DEBUG BOOL true ;; --disable-broccoli) diff --git a/doc/ext/bro_lexer/bro.py b/doc/ext/bro_lexer/bro.py index 8cb4475f3b..ae2566a8de 100644 --- a/doc/ext/bro_lexer/bro.py +++ b/doc/ext/bro_lexer/bro.py @@ -29,7 +29,7 @@ class BroLexer(RegexLexer): r'|vector)\b', Keyword.Type), (r'(T|F)\b', Keyword.Constant), (r'(&)((?:add|delete|expire)_func|attr|(create|read|write)_expire' - r'|default|disable_print_hook|raw_output|encrypt|group|log' + r'|default|raw_output|encrypt|group|log' r'|mergeable|optional|persistent|priority|redef' r'|rotate_(?:interval|size)|synchronized)\b', bygroups(Punctuation, Keyword)), diff --git a/doc/ext/bro_lexer/bro.pyc b/doc/ext/bro_lexer/bro.pyc index 6471e1528d..c7b4fde790 100644 Binary files a/doc/ext/bro_lexer/bro.pyc and b/doc/ext/bro_lexer/bro.pyc differ diff --git a/doc/faq.rst b/doc/faq.rst index 8545cc57ee..76f81cc618 100644 --- a/doc/faq.rst +++ b/doc/faq.rst @@ -12,6 +12,43 @@ Frequently Asked Questions Installation and Configuration ============================== +How do I upgrade to a new version of Bro? +----------------------------------------- + +There's two suggested approaches, either install Bro using the same +installation prefix directory as before, or pick a new prefix and copy +local customizations over. + +Re-Use Previous Install Prefix +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +If you choose to configure and install Bro with the same prefix +directory as before, local customization and configuration to files in +``$prefix/share/bro/site`` and ``$prefix/etc`` won't be overwritten +(``$prefix`` indicating the root of where Bro was installed). Also, logs +generated at run-time won't be touched by the upgrade. (But making +a backup of local changes before proceeding is still recommended.) + +After upgrading, remember to check ``$prefix/share/bro/site`` and +``$prefix/etc`` for ``.example`` files, which indicate the +distribution's version of the file differs from the local one, which may +include local changes. Review the differences, and make adjustments +as necessary (for differences that aren't the result of a local change, +use the new version's). + +Pick a New Install prefix +^^^^^^^^^^^^^^^^^^^^^^^^^ + +If you want to install the newer version in a different prefix +directory than before, you can just copy local customization and +configuration files from ``$prefix/share/bro/site`` and ``$prefix/etc`` +to the new location (``$prefix`` indicating the root of where Bro was +originally installed). Make sure to review the files for difference +before copying and make adjustments as necessary (for differences that +aren't the result of a local change, use the new version's). Of +particular note, the copied version of ``$prefix/etc/broctl.cfg`` is +likely to need changes to the ``SpoolDir`` and ``LogDir`` settings. + How can I tune my operating system for best capture performance? ---------------------------------------------------------------- @@ -46,7 +83,7 @@ directions: http://securityonion.blogspot.com/2011/10/when-is-full-packet-capture-not-full.html What does an error message like ``internal error: NB-DNS error`` mean? ---------------------------------------------------------------------------------------------------------------------------------- +---------------------------------------------------------------------- That often means that DNS is not set up correctly on the system running Bro. Try verifying from the command line that DNS lookups @@ -65,6 +102,15 @@ Generally, please note that we do not regularly test OpenBSD builds. We appreciate any patches that improve Bro's support for this platform. +How do BroControl options affect Bro script variables? +------------------------------------------------------ + +Some (but not all) BroControl options override a corresponding Bro script variable. +For example, setting the BroControl option "LogRotationInterval" will override +the value of the Bro script variable "Log::default_rotation_interval". +See the :doc:`BroControl Documentation ` to find out +which BroControl options override Bro script variables, and for more discussion +on site-specific customization. Usage ===== diff --git a/doc/logging.rst b/doc/logging.rst index cc6cb1e54d..7fb4205b9a 100644 --- a/doc/logging.rst +++ b/doc/logging.rst @@ -383,3 +383,4 @@ Bro supports the following output formats other than ASCII: :maxdepth: 1 logging-dataseries + logging-elasticsearch diff --git a/doc/quickstart.rst b/doc/quickstart.rst index cc18956836..3780eb982a 100644 --- a/doc/quickstart.rst +++ b/doc/quickstart.rst @@ -1,5 +1,6 @@ .. _CMake: http://www.cmake.org .. _SWIG: http://www.swig.org +.. _Xcode: https://developer.apple.com/xcode/ .. _MacPorts: http://www.macports.org .. _Fink: http://www.finkproject.org .. _Homebrew: http://mxcl.github.com/homebrew @@ -85,17 +86,20 @@ The following dependencies are required to build Bro: * Mac OS X - Snow Leopard (10.6) comes with all required dependencies except for CMake_. + Compiling source code on Macs requires first downloading Xcode_, + then going through its "Preferences..." -> "Downloads" menus to + install the "Command Line Tools" component. - Lion (10.7) comes with all required dependencies except for CMake_ and SWIG_. + Lion (10.7) and Mountain Lion (10.8) come with all required + dependencies except for CMake_, SWIG_, and ``libmagic``. - Distributions of these dependencies can be obtained from the project websites - linked above, but they're also likely available from your preferred Mac OS X - package management system (e.g. MacPorts_, Fink_, or Homebrew_). + Distributions of these dependencies can be obtained from the project + websites linked above, but they're also likely available from your + preferred Mac OS X package management system (e.g. MacPorts_, Fink_, + or Homebrew_). - Note that the MacPorts ``swig`` package may not include any specific - language support so you may need to also install ``swig-ruby`` and - ``swig-python``. + Specifically for MacPorts, the ``swig``, ``swig-ruby``, ``swig-python`` + and ``file`` packages provide the required dependencies. Optional Dependencies ~~~~~~~~~~~~~~~~~~~~~ diff --git a/doc/scripts/builtins.rst b/doc/scripts/builtins.rst index 32908f71fd..d274de6b7b 100644 --- a/doc/scripts/builtins.rst +++ b/doc/scripts/builtins.rst @@ -55,8 +55,8 @@ The Bro scripting language supports the following built-in types. A temporal type representing a relative time. An ``interval`` constant can be written as a numeric constant followed by a time - unit where the time unit is one of ``usec``, ``sec``, ``min``, - ``hr``, or ``day`` which respectively represent microseconds, + unit where the time unit is one of ``usec``, ``msec``, ``sec``, ``min``, + ``hr``, or ``day`` which respectively represent microseconds, milliseconds, seconds, minutes, hours, and days. Whitespace between the numeric constant and time unit is optional. Appending the letter "s" to the time unit in order to pluralize it is also optional (to no semantic @@ -95,14 +95,14 @@ The Bro scripting language supports the following built-in types. and embedded. In exact matching the ``==`` equality relational operator is used - with one :bro:type:`string` operand and one :bro:type:`pattern` - operand to check whether the full string exactly matches the - pattern. In this case, the ``^`` beginning-of-line and ``$`` - end-of-line anchors are redundant since pattern is implicitly - anchored to the beginning and end of the line to facilitate an exact - match. For example:: + with one :bro:type:`pattern` operand and one :bro:type:`string` + operand (order of operands does not matter) to check whether the full + string exactly matches the pattern. In exact matching, the ``^`` + beginning-of-line and ``$`` end-of-line anchors are redundant since + the pattern is implicitly anchored to the beginning and end of the + line to facilitate an exact match. For example:: - "foo" == /foo|bar/ + /foo|bar/ == "foo" yields true, while:: @@ -110,9 +110,9 @@ The Bro scripting language supports the following built-in types. yields false. The ``!=`` operator would yield the negation of ``==``. - In embedded matching the ``in`` operator is again used with one - :bro:type:`string` operand and one :bro:type:`pattern` operand - (which must be on the left-hand side), but tests whether the pattern + In embedded matching the ``in`` operator is used with one + :bro:type:`pattern` operand (which must be on the left-hand side) and + one :bro:type:`string` operand, but tests whether the pattern appears anywhere within the given string. For example:: /foo|bar/ in "foobar" @@ -600,10 +600,6 @@ scripting language supports the following built-in attributes. .. TODO: needs to be documented. -.. bro:attr:: &disable_print_hook - - Deprecated. Will be removed. - .. bro:attr:: &raw_output Opens a file in raw mode, i.e., non-ASCII characters are not diff --git a/doc/signatures.rst b/doc/signatures.rst index f65215eceb..36099ba40f 100644 --- a/doc/signatures.rst +++ b/doc/signatures.rst @@ -229,20 +229,10 @@ matched. The following context conditions are defined: confirming the match. If false is returned, no signature match is going to be triggered. The function has to be of type ``function cond(state: signature_state, data: string): bool``. Here, - ``content`` may contain the most recent content chunk available at + ``data`` may contain the most recent content chunk available at the time the signature was matched. If no such chunk is available, - ``content`` will be the empty string. ``signature_state`` is - defined as follows: - - .. code:: bro - - type signature_state: record { - id: string; # ID of the signature - conn: connection; # Current connection - is_orig: bool; # True if current endpoint is originator - payload_size: count; # Payload size of the first packet - }; - + ``data`` will be the empty string. See :bro:type:`signature_state` + for its definition. ``payload-size `` Compares the integer to the size of the payload of a packet. For diff --git a/pkg/make-mac-packages b/pkg/make-mac-packages index 829a64ca25..2930f8f393 100755 --- a/pkg/make-mac-packages +++ b/pkg/make-mac-packages @@ -3,7 +3,13 @@ # This script creates binary packages for Mac OS X. # They can be found in ../build/ after running. -./check-cmake || { exit 1; } +cmake -P /dev/stdin << "EOF" +if ( ${CMAKE_VERSION} VERSION_LESS 2.8.9 ) + message(FATAL_ERROR "CMake >= 2.8.9 required to build package") +endif () +EOF + +[ $? -ne 0 ] && exit 1; type sw_vers > /dev/null 2>&1 || { echo "Unable to get Mac OS X version" >&2; @@ -34,26 +40,26 @@ prefix=/opt/bro cd .. # Minimum Bro -CMAKE_OSX_ARCHITECTURES=${arch} ./configure --prefix=${prefix} \ +CMAKE_PREFIX_PATH=/usr CMAKE_OSX_ARCHITECTURES=${arch} ./configure --prefix=${prefix} \ --disable-broccoli --disable-broctl --pkg-name-prefix=Bro-minimal \ --binary-package ( cd build && make package ) # Full Bro package -CMAKE_OSX_ARCHITECTURES=${arch} ./configure --prefix=${prefix} \ +CMAKE_PREFIX_PATH=/usr CMAKE_OSX_ARCHITECTURES=${arch} ./configure --prefix=${prefix} \ --pkg-name-prefix=Bro --binary-package ( cd build && make package ) # Broccoli cd aux/broccoli -CMAKE_OSX_ARCHITECTURES=${arch} ./configure --prefix=${prefix} \ +CMAKE_PREFIX_PATH=/usr CMAKE_OSX_ARCHITECTURES=${arch} ./configure --prefix=${prefix} \ --binary-package ( cd build && make package && mv *.dmg ../../../build/ ) cd ../.. # Broctl cd aux/broctl -CMAKE_OSX_ARCHITECTURES=${arch} ./configure --prefix=${prefix} \ +CMAKE_PREFIX_PATH=/usr CMAKE_OSX_ARCHITECTURES=${arch} ./configure --prefix=${prefix} \ --binary-package ( cd build && make package && mv *.dmg ../../../build/ ) cd ../.. diff --git a/scripts/base/frameworks/input/main.bro b/scripts/base/frameworks/input/main.bro index c31f92dba5..b5f44669c2 100644 --- a/scripts/base/frameworks/input/main.bro +++ b/scripts/base/frameworks/input/main.bro @@ -8,8 +8,16 @@ export { ## The default input reader used. Defaults to `READER_ASCII`. const default_reader = READER_ASCII &redef; + ## The default reader mode used. Defaults to `MANUAL`. const default_mode = MANUAL &redef; + ## Flag that controls if the input framework accepts records + ## that contain types that are not supported (at the moment + ## file and function). If true, the input framework will + ## warn in these cases, but continue. If false, it will + ## abort. Defaults to false (abort) + const accept_unsupported_types = F &redef; + ## TableFilter description type used for the `table` method. type TableDescription: record { ## Common definitions for tables and events @@ -82,11 +90,11 @@ export { ## Record describing the fields to be retrieved from the source input. fields: any; - ## If want_record if false (default), the event receives each value in fields as a seperate argument. - ## If it is set to true, the event receives all fields in a signle record value. - want_record: bool &default=F; + ## If want_record if false, the event receives each value in fields as a separate argument. + ## If it is set to true (default), the event receives all fields in a single record value. + want_record: bool &default=T; - ## The event that is rised each time a new line is received from the reader. + ## The event that is raised each time a new line is received from the reader. ## The event will receive an Input::Event enum as the first element, and the fields as the following arguments. ev: any; diff --git a/scripts/base/frameworks/logging/main.bro b/scripts/base/frameworks/logging/main.bro index cc0d341605..bed76a1ae5 100644 --- a/scripts/base/frameworks/logging/main.bro +++ b/scripts/base/frameworks/logging/main.bro @@ -96,6 +96,12 @@ export { ## file name. Generally, filenames are expected to given ## without any extensions; writers will add appropiate ## extensions automatically. + ## + ## If this path is found to conflict with another filter's + ## for the same writer type, it is automatically corrected + ## by appending "-N", where N is the smallest integer greater + ## or equal to 2 that allows the corrected path name to not + ## conflict with another filter's. path: string &optional; ## A function returning the output path for recording entries @@ -115,7 +121,10 @@ export { ## rec: An instance of the streams's ``columns`` type with its ## fields set to the values to be logged. ## - ## Returns: The path to be used for the filter. + ## Returns: The path to be used for the filter, which will be subject + ## to the same automatic correction rules as the *path* + ## field of :bro:type:`Log::Filter` in the case of conflicts + ## with other filters trying to use the same writer/path pair. path_func: function(id: ID, path: string, rec: any): string &optional; ## Subset of column names to record. If not given, all @@ -318,6 +327,11 @@ export { ## Log::default_rotation_postprocessor_cmd ## Log::default_rotation_postprocessors global run_rotation_postprocessor_cmd: function(info: RotationInfo, npath: string) : bool; + + ## The streams which are currently active and not disabled. + ## This table is not meant to be modified by users! Only use it for + ## examining which streams are active. + global active_streams: table[ID] of Stream = table(); } # We keep a script-level copy of all filters so that we can manipulate them. @@ -332,22 +346,23 @@ function __default_rotation_postprocessor(info: RotationInfo) : bool { if ( info$writer in default_rotation_postprocessors ) return default_rotation_postprocessors[info$writer](info); - - return F; + else + # Return T by default so that postprocessor-less writers don't shutdown. + return T; } function default_path_func(id: ID, path: string, rec: any) : string { + # The suggested path value is a previous result of this function + # or a filter path explicitly set by the user, so continue using it. + if ( path != "" ) + return path; + local id_str = fmt("%s", id); local parts = split1(id_str, /::/); if ( |parts| == 2 ) { - # The suggested path value is a previous result of this function - # or a filter path explicitly set by the user, so continue using it. - if ( path != "" ) - return path; - # Example: Notice::LOG -> "notice" if ( parts[2] == "LOG" ) { @@ -402,11 +417,15 @@ function create_stream(id: ID, stream: Stream) : bool if ( ! __create_stream(id, stream) ) return F; + active_streams[id] = stream; + return add_default_filter(id); } function disable_stream(id: ID) : bool { + delete active_streams[id]; + return __disable_stream(id); } diff --git a/scripts/base/frameworks/logging/writers/elasticsearch.bro b/scripts/base/frameworks/logging/writers/elasticsearch.bro index a6a485226a..1901759730 100644 --- a/scripts/base/frameworks/logging/writers/elasticsearch.bro +++ b/scripts/base/frameworks/logging/writers/elasticsearch.bro @@ -23,11 +23,13 @@ export { const index_prefix = "bro" &redef; ## The ES type prefix comes before the name of the related log. - ## e.g. prefix = "bro_" would create types of bro_dns, bro_software, etc. + ## e.g. prefix = "bro\_" would create types of bro_dns, bro_software, etc. const type_prefix = "" &redef; - ## The time before an ElasticSearch transfer will timeout. - ## This is not working! + ## The time before an ElasticSearch transfer will timeout. Note that + ## the fractional part of the timeout will be ignored. In particular, time + ## specifications less than a second result in a timeout value of 0, which + ## means "no timeout." const transfer_timeout = 2secs; ## The batch size is the number of messages that will be queued up before diff --git a/scripts/base/frameworks/reporter/main.bro b/scripts/base/frameworks/reporter/main.bro index 3c19005364..edc5b1779a 100644 --- a/scripts/base/frameworks/reporter/main.bro +++ b/scripts/base/frameworks/reporter/main.bro @@ -1,5 +1,5 @@ -##! This framework is intended to create an output and filtering path for -##! internal messages/warnings/errors. It should typically be loaded to +##! This framework is intended to create an output and filtering path for +##! internal messages/warnings/errors. It should typically be loaded to ##! avoid Bro spewing internal messages to standard error and instead log ##! them to a file in a standard way. Note that this framework deals with ##! the handling of internally-generated reporter messages, for the @@ -13,11 +13,11 @@ export { redef enum Log::ID += { LOG }; ## An indicator of reporter message severity. - type Level: enum { + type Level: enum { ## Informational, not needing specific attention. - INFO, + INFO, ## Warning of a potential problem. - WARNING, + WARNING, ## A non-fatal error that should be addressed, but doesn't ## terminate program execution. ERROR @@ -36,24 +36,55 @@ export { ## Not all reporter messages will have locations in them though. location: string &log &optional; }; + + ## Tunable for sending reporter warning messages to STDERR. The option to + ## turn it off is presented here in case Bro is being run by some + ## external harness and shouldn't output anything to the console. + const warnings_to_stderr = T &redef; + + ## Tunable for sending reporter error messages to STDERR. The option to + ## turn it off is presented here in case Bro is being run by some + ## external harness and shouldn't output anything to the console. + const errors_to_stderr = T &redef; } +global stderr: file; + event bro_init() &priority=5 { Log::create_stream(Reporter::LOG, [$columns=Info]); + + if ( errors_to_stderr || warnings_to_stderr ) + stderr = open("/dev/stderr"); } -event reporter_info(t: time, msg: string, location: string) +event reporter_info(t: time, msg: string, location: string) &priority=-5 { Log::write(Reporter::LOG, [$ts=t, $level=INFO, $message=msg, $location=location]); } - -event reporter_warning(t: time, msg: string, location: string) + +event reporter_warning(t: time, msg: string, location: string) &priority=-5 { + if ( warnings_to_stderr ) + { + if ( t > double_to_time(0.0) ) + print stderr, fmt("WARNING: %.6f %s (%s)", t, msg, location); + else + print stderr, fmt("WARNING: %s (%s)", msg, location); + } + Log::write(Reporter::LOG, [$ts=t, $level=WARNING, $message=msg, $location=location]); } -event reporter_error(t: time, msg: string, location: string) +event reporter_error(t: time, msg: string, location: string) &priority=-5 { + if ( errors_to_stderr ) + { + if ( t > double_to_time(0.0) ) + print stderr, fmt("ERROR: %.6f %s (%s)", t, msg, location); + else + print stderr, fmt("ERROR: %s (%s)", msg, location); + } + Log::write(Reporter::LOG, [$ts=t, $level=ERROR, $message=msg, $location=location]); } diff --git a/scripts/base/init-bare.bro b/scripts/base/init-bare.bro index ec75c76beb..70026394e9 100644 --- a/scripts/base/init-bare.bro +++ b/scripts/base/init-bare.bro @@ -1135,10 +1135,10 @@ type ip6_ah: record { rsv: count; ## Security Parameter Index. spi: count; - ## Sequence number. - seq: count; - ## Authentication data. - data: string; + ## Sequence number, unset in the case that *len* field is zero. + seq: count &optional; + ## Authentication data, unset in the case that *len* field is zero. + data: string &optional; }; ## Values extracted from an IPv6 ESP extension header. @@ -2784,6 +2784,14 @@ export { ## to have a valid Teredo encapsulation. const yielding_teredo_decapsulation = T &redef; + ## With this set, the Teredo analyzer waits until it sees both sides + ## of a connection using a valid Teredo encapsulation before issuing + ## a :bro:see:`protocol_confirmation`. If it's false, the first + ## occurence of a packet with valid Teredo encapsulation causes a + ## confirmation. Both cases are still subject to effects of + ## :bro:see:`Tunnel::yielding_teredo_decapsulation`. + const delay_teredo_confirmation = T &redef; + ## How often to cleanup internal state for inactive IP tunnels. const ip_tunnel_timeout = 24hrs &redef; } # end export diff --git a/scripts/policy/protocols/smtp/blocklists.bro b/scripts/policy/protocols/smtp/blocklists.bro index a3e75318bb..b1fb0e498d 100644 --- a/scripts/policy/protocols/smtp/blocklists.bro +++ b/scripts/policy/protocols/smtp/blocklists.bro @@ -1,3 +1,4 @@ +##! Watch for various SPAM blocklist URLs in SMTP error messages. @load base/protocols/smtp @@ -5,9 +6,11 @@ module SMTP; export { redef enum Notice::Type += { - ## Indicates that the server sent a reply mentioning an SMTP block list. + ## An SMTP server sent a reply mentioning an SMTP block list. Blocklist_Error_Message, - ## Indicates the client's address is seen in the block list error message. + ## The originator's address is seen in the block list error message. + ## This is useful to detect local hosts sending SPAM with a high + ## positive rate. Blocklist_Blocked_Host, }; @@ -52,7 +55,8 @@ event smtp_reply(c: connection, is_orig: bool, code: count, cmd: string, message = fmt("%s is on an SMTP block list", c$id$orig_h); } - NOTICE([$note=note, $conn=c, $msg=message, $sub=msg]); + NOTICE([$note=note, $conn=c, $msg=message, $sub=msg, + $identifier=cat(c$id$orig_h)]); } } } diff --git a/scripts/policy/tuning/logs-to-elasticsearch.bro b/scripts/policy/tuning/logs-to-elasticsearch.bro index c3cc9d5002..2a4b70362a 100644 --- a/scripts/policy/tuning/logs-to-elasticsearch.bro +++ b/scripts/policy/tuning/logs-to-elasticsearch.bro @@ -4,42 +4,33 @@ module LogElasticSearch; export { ## An elasticsearch specific rotation interval. - const rotation_interval = 24hr &redef; + const rotation_interval = 3hr &redef; - ## Optionally ignore any :bro:enum:`Log::ID` from being sent to + ## Optionally ignore any :bro:type:`Log::ID` from being sent to ## ElasticSearch with this script. - const excluded_log_ids: set[string] = set("Communication::LOG") &redef; + const excluded_log_ids: set[Log::ID] &redef; - ## If you want to explicitly only send certain :bro:enum:`Log::ID` + ## If you want to explicitly only send certain :bro:type:`Log::ID` ## streams, add them to this set. If the set remains empty, all will - ## be sent. The :bro:id:`excluded_log_ids` option will remain in + ## be sent. The :bro:id:`LogElasticSearch::excluded_log_ids` option will remain in ## effect as well. - const send_logs: set[string] = set() &redef; + const send_logs: set[Log::ID] &redef; } -module Log; - event bro_init() &priority=-5 { - local my_filters: table[ID, string] of Filter = table(); - - for ( [id, name] in filters ) + if ( server_host == "" ) + return; + + for ( stream_id in Log::active_streams ) { - local filter = filters[id, name]; - if ( fmt("%s", id) in LogElasticSearch::excluded_log_ids || - (|LogElasticSearch::send_logs| > 0 && fmt("%s", id) !in LogElasticSearch::send_logs) ) + if ( stream_id in excluded_log_ids || + (|send_logs| > 0 && stream_id !in send_logs) ) next; - filter$name = cat(name, "-es"); - filter$writer = Log::WRITER_ELASTICSEARCH; - filter$interv = LogElasticSearch::rotation_interval; - my_filters[id, name] = filter; + local filter: Log::Filter = [$name = "default-es", + $writer = Log::WRITER_ELASTICSEARCH, + $interv = LogElasticSearch::rotation_interval]; + Log::add_filter(stream_id, filter); } - - # This had to be done separately to avoid an ever growing filters list - # where the for loop would never end. - for ( [id, name] in my_filters ) - { - Log::add_filter(id, filter); - } - } \ No newline at end of file + } diff --git a/scripts/test-all-policy.bro b/scripts/test-all-policy.bro index c4acece25d..a7c43b14b3 100644 --- a/scripts/test-all-policy.bro +++ b/scripts/test-all-policy.bro @@ -60,5 +60,5 @@ @load tuning/defaults/__load__.bro @load tuning/defaults/packet-fragments.bro @load tuning/defaults/warnings.bro -# @load tuning/logs-to-elasticsearch.bro +@load tuning/logs-to-elasticsearch.bro @load tuning/track-all-assets.bro diff --git a/src/Attr.cc b/src/Attr.cc index 2e4e090c0b..bdf247b4f5 100644 --- a/src/Attr.cc +++ b/src/Attr.cc @@ -15,7 +15,7 @@ const char* attr_name(attr_tag t) "&add_func", "&delete_func", "&expire_func", "&read_expire", "&write_expire", "&create_expire", "&persistent", "&synchronized", "&postprocessor", - "&encrypt", "&match", "&disable_print_hook", + "&encrypt", "&match", "&raw_output", "&mergeable", "&priority", "&group", "&log", "&error_handler", "&type_column", "(&tracked)", @@ -385,11 +385,6 @@ void Attributes::CheckAttr(Attr* a) // FIXME: Check here for global ID? break; - case ATTR_DISABLE_PRINT_HOOK: - if ( type->Tag() != TYPE_FILE ) - Error("&disable_print_hook only applicable to files"); - break; - case ATTR_RAW_OUTPUT: if ( type->Tag() != TYPE_FILE ) Error("&raw_output only applicable to files"); diff --git a/src/Attr.h b/src/Attr.h index e6b09cf96b..c9a0dedb33 100644 --- a/src/Attr.h +++ b/src/Attr.h @@ -28,7 +28,6 @@ typedef enum { ATTR_POSTPROCESSOR, ATTR_ENCRYPT, ATTR_MATCH, - ATTR_DISABLE_PRINT_HOOK, ATTR_RAW_OUTPUT, ATTR_MERGEABLE, ATTR_PRIORITY, diff --git a/src/ChunkedIO.cc b/src/ChunkedIO.cc index f5bcb4b7c1..2c766c7eb1 100644 --- a/src/ChunkedIO.cc +++ b/src/ChunkedIO.cc @@ -76,7 +76,7 @@ void ChunkedIO::DumpDebugData(const char* basefnname, bool want_reads) ChunkedIOFd io(fd, "dump-file"); io.Write(*i); io.Flush(); - close(fd); + safe_close(fd); } l->clear(); @@ -127,7 +127,7 @@ ChunkedIOFd::~ChunkedIOFd() delete [] read_buffer; delete [] write_buffer; - close(fd); + safe_close(fd); if ( partial ) { @@ -686,7 +686,7 @@ ChunkedIOSSL::~ChunkedIOSSL() ssl = 0; } - close(socket); + safe_close(socket); } diff --git a/src/Expr.cc b/src/Expr.cc index b62f119bae..e6936267d8 100644 --- a/src/Expr.cc +++ b/src/Expr.cc @@ -872,10 +872,12 @@ Val* BinaryExpr::SubNetFold(Val* v1, Val* v2) const const IPPrefix& n1 = v1->AsSubNet(); const IPPrefix& n2 = v2->AsSubNet(); - if ( n1 == n2 ) - return new Val(1, TYPE_BOOL); - else - return new Val(0, TYPE_BOOL); + bool result = ( n1 == n2 ) ? true : false; + + if ( tag == EXPR_NE ) + result = ! result; + + return new Val(result, TYPE_BOOL); } void BinaryExpr::SwapOps() @@ -1515,6 +1517,8 @@ RemoveFromExpr::RemoveFromExpr(Expr* arg_op1, Expr* arg_op2) if ( BothArithmetic(bt1, bt2) ) PromoteType(max_type(bt1, bt2), is_vector(op1) || is_vector(op2)); + else if ( BothInterval(bt1, bt2) ) + SetType(base_type(bt1)); else ExprError("requires two arithmetic operands"); } diff --git a/src/File.cc b/src/File.cc index 20e845c09f..880fd254ef 100644 --- a/src/File.cc +++ b/src/File.cc @@ -138,11 +138,22 @@ BroFile::BroFile(FILE* arg_f, const char* arg_name, const char* arg_access) BroFile::BroFile(const char* arg_name, const char* arg_access, BroType* arg_t) { Init(); - + f = 0; name = copy_string(arg_name); access = copy_string(arg_access); t = arg_t ? arg_t : base_type(TYPE_STRING); - if ( ! Open() ) + + if ( streq(name, "/dev/stdin") ) + f = stdin; + else if ( streq(name, "/dev/stdout") ) + f = stdout; + else if ( streq(name, "/dev/stderr") ) + f = stderr; + + if ( f ) + is_open = 1; + + else if ( ! Open() ) { reporter->Error("cannot open %s: %s", name, strerror(errno)); is_open = 0; @@ -342,8 +353,8 @@ int BroFile::Close() FinishEncrypt(); - // Do not close stdout/stderr. - if ( f == stdout || f == stderr ) + // Do not close stdin/stdout/stderr. + if ( f == stdin || f == stdout || f == stderr ) return 0; if ( is_in_cache ) @@ -503,12 +514,9 @@ void BroFile::SetAttrs(Attributes* arg_attrs) InitEncrypt(log_encryption_key->AsString()->CheckString()); } - if ( attrs->FindAttr(ATTR_DISABLE_PRINT_HOOK) ) - DisablePrintHook(); - if ( attrs->FindAttr(ATTR_RAW_OUTPUT) ) EnableRawOutput(); - + InstallRotateTimer(); } @@ -523,6 +531,10 @@ RecordVal* BroFile::Rotate() if ( ! is_open ) return 0; + // Do not rotate stdin/stdout/stderr. + if ( f == stdin || f == stdout || f == stderr ) + return 0; + if ( okay_to_manage && ! is_in_cache ) BringIntoCache(); diff --git a/src/File.h b/src/File.h index 37f844867b..8e3d0ca6e7 100644 --- a/src/File.h +++ b/src/File.h @@ -57,7 +57,7 @@ public: RecordVal* Rotate(); // Set &rotate_interval, &rotate_size, &postprocessor, - // &disable_print_hook, and &raw_output attributes. + // and &raw_output attributes. void SetAttrs(Attributes* attrs); // Returns the current size of the file, after fresh stat'ing. diff --git a/src/FlowSrc.cc b/src/FlowSrc.cc index fe6998ea79..59ce3fd6a4 100644 --- a/src/FlowSrc.cc +++ b/src/FlowSrc.cc @@ -58,7 +58,7 @@ void FlowSrc::Process() void FlowSrc::Close() { - close(selectable_fd); + safe_close(selectable_fd); } diff --git a/src/IP.cc b/src/IP.cc index 45afd593a9..16424e26f2 100644 --- a/src/IP.cc +++ b/src/IP.cc @@ -148,9 +148,15 @@ RecordVal* IPv6_Hdr::BuildRecordVal(VectorVal* chain) const rv->Assign(1, new Val(((ip6_ext*)data)->ip6e_len, TYPE_COUNT)); rv->Assign(2, new Val(ntohs(((uint16*)data)[1]), TYPE_COUNT)); rv->Assign(3, new Val(ntohl(((uint32*)data)[1]), TYPE_COUNT)); - rv->Assign(4, new Val(ntohl(((uint32*)data)[2]), TYPE_COUNT)); - uint16 off = 3 * sizeof(uint32); - rv->Assign(5, new StringVal(new BroString(data + off, Length() - off, 1))); + + if ( Length() >= 12 ) + { + // Sequence Number and ICV fields can only be extracted if + // Payload Len was non-zero for this header. + rv->Assign(4, new Val(ntohl(((uint32*)data)[2]), TYPE_COUNT)); + uint16 off = 3 * sizeof(uint32); + rv->Assign(5, new StringVal(new BroString(data + off, Length() - off, 1))); + } } break; diff --git a/src/RemoteSerializer.cc b/src/RemoteSerializer.cc index 7ed8b9318e..564ad2be68 100644 --- a/src/RemoteSerializer.cc +++ b/src/RemoteSerializer.cc @@ -647,7 +647,7 @@ void RemoteSerializer::Fork() exit(1); // FIXME: Better way to handle this? } - close(pipe[1]); + safe_close(pipe[1]); return; } @@ -664,12 +664,12 @@ void RemoteSerializer::Fork() } child.SetParentIO(io); - close(pipe[0]); + safe_close(pipe[0]); // Close file descriptors. - close(0); - close(1); - close(2); + safe_close(0); + safe_close(1); + safe_close(2); // Be nice. setpriority(PRIO_PROCESS, 0, 5); @@ -2716,7 +2716,8 @@ bool RemoteSerializer::ProcessLogCreateWriter() id_val = new EnumVal(id, BifType::Enum::Log::ID); writer_val = new EnumVal(writer, BifType::Enum::Log::Writer); - if ( ! log_mgr->CreateWriter(id_val, writer_val, info, num_fields, fields, true, false) ) + if ( ! log_mgr->CreateWriter(id_val, writer_val, info, num_fields, fields, + true, false, true) ) goto error; Unref(id_val); @@ -2896,11 +2897,6 @@ void RemoteSerializer::GotID(ID* id, Val* val) (desc && *desc) ? desc : "not set"), current_peer); -#ifdef USE_PERFTOOLS_DEBUG - // May still be cached, but we don't care. - heap_checker->IgnoreObject(id); -#endif - Unref(id); return; } @@ -4001,7 +3997,7 @@ bool SocketComm::Connect(Peer* peer) if ( connect(sockfd, res->ai_addr, res->ai_addrlen) < 0 ) { Error(fmt("connect failed: %s", strerror(errno)), peer); - close(sockfd); + safe_close(sockfd); sockfd = -1; continue; } @@ -4174,16 +4170,18 @@ bool SocketComm::Listen() { Error(fmt("can't bind to %s:%s, %s", l_addr_str.c_str(), port_str, strerror(errno))); - close(fd); if ( errno == EADDRINUSE ) { // Abandon completely this attempt to set up listening sockets, // try again later. + safe_close(fd); CloseListenFDs(); listen_next_try = time(0) + bind_retry_interval; return false; } + + safe_close(fd); continue; } @@ -4191,7 +4189,7 @@ bool SocketComm::Listen() { Error(fmt("can't listen on %s:%s, %s", l_addr_str.c_str(), port_str, strerror(errno))); - close(fd); + safe_close(fd); continue; } @@ -4227,7 +4225,7 @@ bool SocketComm::AcceptConnection(int fd) { Error(fmt("accept fail, unknown address family %d", client.ss.ss_family)); - close(clientfd); + safe_close(clientfd); return false; } @@ -4298,7 +4296,7 @@ const char* SocketComm::MakeLogString(const char* msg, Peer* peer) void SocketComm::CloseListenFDs() { for ( size_t i = 0; i < listen_fds.size(); ++i ) - close(listen_fds[i]); + safe_close(listen_fds[i]); listen_fds.clear(); } diff --git a/src/RuleCondition.cc b/src/RuleCondition.cc index 8852747cc4..410f6a1b3e 100644 --- a/src/RuleCondition.cc +++ b/src/RuleCondition.cc @@ -126,6 +126,23 @@ RuleConditionEval::RuleConditionEval(const char* func) rules_error("unknown identifier", func); return; } + + if ( id->Type()->Tag() == TYPE_FUNC ) + { + // Validate argument quantity and type. + FuncType* f = id->Type()->AsFuncType(); + + if ( f->YieldType()->Tag() != TYPE_BOOL ) + rules_error("eval function type must yield a 'bool'", func); + + TypeList tl; + tl.Append(internal_type("signature_state")->Ref()); + tl.Append(base_type(TYPE_STRING)); + + if ( ! f->CheckArgs(tl.Types()) ) + rules_error("eval function parameters must be a 'signature_state' " + "and a 'string' type", func); + } } bool RuleConditionEval::DoMatch(Rule* rule, RuleEndpointState* state, diff --git a/src/Serializer.cc b/src/Serializer.cc index 06bbf73f48..fc6d00d06c 100644 --- a/src/Serializer.cc +++ b/src/Serializer.cc @@ -742,10 +742,11 @@ FileSerializer::~FileSerializer() io->Flush(); delete [] file; - delete io; - if ( fd >= 0 ) - close(fd); + if ( io ) + delete io; // destructor will call close() on fd + else if ( fd >= 0 ) + safe_close(fd); } bool FileSerializer::Open(const char* file, bool pure) @@ -808,8 +809,8 @@ void FileSerializer::CloseFile() if ( io ) io->Flush(); - if ( fd >= 0 ) - close(fd); + if ( fd >= 0 && ! io ) // destructor of io calls close() on fd + safe_close(fd); fd = -1; delete [] file; diff --git a/src/Stats.cc b/src/Stats.cc index c3035231e9..8d48c47a25 100644 --- a/src/Stats.cc +++ b/src/Stats.cc @@ -12,10 +12,10 @@ int killed_by_inactivity = 0; -uint32 tot_ack_events = 0; -uint32 tot_ack_bytes = 0; -uint32 tot_gap_events = 0; -uint32 tot_gap_bytes = 0; +uint64 tot_ack_events = 0; +uint64 tot_ack_bytes = 0; +uint64 tot_gap_events = 0; +uint64 tot_gap_bytes = 0; class ProfileTimer : public Timer { diff --git a/src/Stats.h b/src/Stats.h index eeebfe2213..a11d66828a 100644 --- a/src/Stats.h +++ b/src/Stats.h @@ -116,10 +116,10 @@ extern SampleLogger* sample_logger; extern int killed_by_inactivity; // Content gap statistics. -extern uint32 tot_ack_events; -extern uint32 tot_ack_bytes; -extern uint32 tot_gap_events; -extern uint32 tot_gap_bytes; +extern uint64 tot_ack_events; +extern uint64 tot_ack_bytes; +extern uint64 tot_gap_events; +extern uint64 tot_gap_bytes; // A TCPStateStats object tracks the distribution of TCP states for diff --git a/src/Stmt.cc b/src/Stmt.cc index 582323bf91..7d754d8e72 100644 --- a/src/Stmt.cc +++ b/src/Stmt.cc @@ -943,7 +943,10 @@ ForStmt::ForStmt(id_list* arg_loop_vars, Expr* loop_expr) { const type_list* indices = e->Type()->AsTableType()->IndexTypes(); if ( indices->length() != loop_vars->length() ) + { e->Error("wrong index size"); + return; + } for ( int i = 0; i < indices->length(); i++ ) { diff --git a/src/TCP.cc b/src/TCP.cc index 57e4449bf8..555adf1b57 100644 --- a/src/TCP.cc +++ b/src/TCP.cc @@ -46,6 +46,7 @@ TCP_Analyzer::TCP_Analyzer(Connection* conn) finished = 0; reassembling = 0; first_packet_seen = 0; + is_partial = 0; orig = new TCP_Endpoint(this, 1); resp = new TCP_Endpoint(this, 0); diff --git a/src/TCP_Reassembler.cc b/src/TCP_Reassembler.cc index fb67dba7ee..eb2709373c 100644 --- a/src/TCP_Reassembler.cc +++ b/src/TCP_Reassembler.cc @@ -20,10 +20,10 @@ const bool DEBUG_tcp_connection_close = false; const bool DEBUG_tcp_match_undelivered = false; static double last_gap_report = 0.0; -static uint32 last_ack_events = 0; -static uint32 last_ack_bytes = 0; -static uint32 last_gap_events = 0; -static uint32 last_gap_bytes = 0; +static uint64 last_ack_events = 0; +static uint64 last_ack_bytes = 0; +static uint64 last_gap_events = 0; +static uint64 last_gap_bytes = 0; TCP_Reassembler::TCP_Reassembler(Analyzer* arg_dst_analyzer, TCP_Analyzer* arg_tcp_analyzer, @@ -513,10 +513,10 @@ void TCP_Reassembler::AckReceived(int seq) if ( gap_report && gap_report_freq > 0.0 && dt >= gap_report_freq ) { - int devents = tot_ack_events - last_ack_events; - int dbytes = tot_ack_bytes - last_ack_bytes; - int dgaps = tot_gap_events - last_gap_events; - int dgap_bytes = tot_gap_bytes - last_gap_bytes; + uint64 devents = tot_ack_events - last_ack_events; + uint64 dbytes = tot_ack_bytes - last_ack_bytes; + uint64 dgaps = tot_gap_events - last_gap_events; + uint64 dgap_bytes = tot_gap_bytes - last_gap_bytes; RecordVal* r = new RecordVal(gap_info); r->Assign(0, new Val(devents, TYPE_COUNT)); diff --git a/src/Teredo.cc b/src/Teredo.cc index 54676c3255..7794d1cb3b 100644 --- a/src/Teredo.cc +++ b/src/Teredo.cc @@ -138,6 +138,11 @@ void Teredo_Analyzer::DeliverPacket(int len, const u_char* data, bool orig, { Analyzer::DeliverPacket(len, data, orig, seq, ip, caplen); + if ( orig ) + valid_orig = false; + else + valid_resp = false; + TeredoEncapsulation te(this); if ( ! te.Parse(data, len) ) @@ -150,7 +155,7 @@ void Teredo_Analyzer::DeliverPacket(int len, const u_char* data, bool orig, if ( e && e->Depth() >= BifConst::Tunnel::max_depth ) { - Weird("tunnel_depth"); + Weird("tunnel_depth", true); return; } @@ -162,7 +167,7 @@ void Teredo_Analyzer::DeliverPacket(int len, const u_char* data, bool orig, if ( inner->NextProto() == IPPROTO_NONE && inner->PayloadLen() == 0 ) // Teredo bubbles having data after IPv6 header isn't strictly a // violation, but a little weird. - Weird("Teredo_bubble_with_payload"); + Weird("Teredo_bubble_with_payload", true); else { delete inner; @@ -173,6 +178,11 @@ void Teredo_Analyzer::DeliverPacket(int len, const u_char* data, bool orig, if ( rslt == 0 || rslt > 0 ) { + if ( orig ) + valid_orig = true; + else + valid_resp = true; + if ( BifConst::Tunnel::yielding_teredo_decapsulation && ! ProtocolConfirmed() ) { @@ -193,7 +203,7 @@ void Teredo_Analyzer::DeliverPacket(int len, const u_char* data, bool orig, } if ( ! sibling_has_confirmed ) - ProtocolConfirmation(); + Confirm(); else { delete inner; @@ -201,10 +211,8 @@ void Teredo_Analyzer::DeliverPacket(int len, const u_char* data, bool orig, } } else - { - // Aggressively decapsulate anything with valid Teredo encapsulation - ProtocolConfirmation(); - } + // Aggressively decapsulate anything with valid Teredo encapsulation. + Confirm(); } else diff --git a/src/Teredo.h b/src/Teredo.h index 84ff8ddf38..e720d3f37c 100644 --- a/src/Teredo.h +++ b/src/Teredo.h @@ -6,7 +6,8 @@ class Teredo_Analyzer : public Analyzer { public: - Teredo_Analyzer(Connection* conn) : Analyzer(AnalyzerTag::Teredo, conn) + Teredo_Analyzer(Connection* conn) : Analyzer(AnalyzerTag::Teredo, conn), + valid_orig(false), valid_resp(false) {} virtual ~Teredo_Analyzer() @@ -26,18 +27,34 @@ public: /** * Emits a weird only if the analyzer has previously been able to - * decapsulate a Teredo packet since otherwise the weirds could happen - * frequently enough to be less than helpful. + * decapsulate a Teredo packet in both directions or if *force* param is + * set, since otherwise the weirds could happen frequently enough to be less + * than helpful. The *force* param is meant for cases where just one side + * has a valid encapsulation and so the weird would be informative. */ - void Weird(const char* name) const + void Weird(const char* name, bool force = false) const { - if ( ProtocolConfirmed() ) + if ( ProtocolConfirmed() || force ) reporter->Weird(Conn(), name); } + /** + * If the delayed confirmation option is set, then a valid encapsulation + * seen from both end points is required before confirming. + */ + void Confirm() + { + if ( ! BifConst::Tunnel::delay_teredo_confirmation || + ( valid_orig && valid_resp ) ) + ProtocolConfirmation(); + } + protected: friend class AnalyzerTimer; void ExpireTimer(double t); + + bool valid_orig; + bool valid_resp; }; class TeredoEncapsulation { diff --git a/src/Val.cc b/src/Val.cc index 8a8c2b18c0..79fa8a0c69 100644 --- a/src/Val.cc +++ b/src/Val.cc @@ -64,7 +64,7 @@ Val::~Val() Unref(type); #ifdef DEBUG - Unref(bound_id); + delete [] bound_id; #endif } diff --git a/src/Val.h b/src/Val.h index 2ca18e6131..c3ec5b04fb 100644 --- a/src/Val.h +++ b/src/Val.h @@ -347,13 +347,15 @@ public: #ifdef DEBUG // For debugging, we keep a reference to the global ID to which a // value has been bound *last*. - ID* GetID() const { return bound_id; } + ID* GetID() const + { + return bound_id ? global_scope()->Lookup(bound_id) : 0; + } + void SetID(ID* id) { - if ( bound_id ) - ::Unref(bound_id); - bound_id = id; - ::Ref(bound_id); + delete [] bound_id; + bound_id = id ? copy_string(id->Name()) : 0; } #endif @@ -401,8 +403,8 @@ protected: RecordVal* attribs; #ifdef DEBUG - // For debugging, we keep the ID to which a Val is bound. - ID* bound_id; + // For debugging, we keep the name of the ID to which a Val is bound. + const char* bound_id; #endif }; diff --git a/src/bro.bif b/src/bro.bif index 2a37429ad6..8ddde6ef86 100644 --- a/src/bro.bif +++ b/src/bro.bif @@ -3787,7 +3787,7 @@ static GeoIP* open_geoip_db(GeoIPDBTypes type) geoip = GeoIP_open_type(type, GEOIP_MEMORY_CACHE); if ( ! geoip ) - reporter->Warning("Failed to open GeoIP database: %s", + reporter->Info("Failed to open GeoIP database: %s", GeoIPDBFileName[type]); return geoip; } @@ -3827,7 +3827,7 @@ function lookup_location%(a: addr%) : geo_location if ( ! geoip ) builtin_error("Can't initialize GeoIP City/Country database"); else - reporter->Warning("Fell back to GeoIP Country database"); + reporter->Info("Fell back to GeoIP Country database"); } else have_city_db = true; @@ -4858,7 +4858,7 @@ function file_size%(f: string%) : double %} ## Disables sending :bro:id:`print_hook` events to remote peers for a given -## file. This function is equivalent to :bro:attr:`&disable_print_hook`. In a +## file. In a ## distributed setup, communicating Bro instances generate the event ## :bro:id:`print_hook` for each print statement and send it to the remote ## side. When disabled for a particular file, these events will not be @@ -4874,7 +4874,7 @@ function disable_print_hook%(f: file%): any %} ## Prevents escaping of non-ASCII characters when writing to a file. -## This function is equivalent to :bro:attr:`&disable_print_hook`. +## This function is equivalent to :bro:attr:`&raw_output`. ## ## f: The file to disable raw output for. ## @@ -5683,12 +5683,6 @@ function match_signatures%(c: connection, pattern_type: int, s: string, # # =========================================================================== -## Deprecated. Will be removed. -function parse_dotted_addr%(s: string%): addr - %{ - IPAddr a(s->CheckString()); - return new AddrVal(a); - %} %%{ @@ -5788,75 +5782,3 @@ function anonymize_addr%(a: addr, cl: IPAddrAnonymizationClass%): addr } %} -## Deprecated. Will be removed. -function dump_config%(%) : bool - %{ - return new Val(persistence_serializer->WriteConfig(true), TYPE_BOOL); - %} - -## Deprecated. Will be removed. -function make_connection_persistent%(c: connection%) : any - %{ - c->MakePersistent(); - return 0; - %} - -%%{ -// Experimental code to add support for IDMEF XML output based on -// notices. For now, we're implementing it as a builtin you can call on an -// notices record. - -#ifdef USE_IDMEF -extern "C" { -#include -} -#endif - -#include - -char* port_to_string(PortVal* port) - { - char buf[256]; // to hold sprintf results on port numbers - snprintf(buf, sizeof(buf), "%u", port->Port()); - return copy_string(buf); - } - -%%} - -## Deprecated. Will be removed. -function generate_idmef%(src_ip: addr, src_port: port, - dst_ip: addr, dst_port: port%) : bool - %{ -#ifdef USE_IDMEF - xmlNodePtr message = - newIDMEF_Message(newAttribute("version","1.0"), - newAlert(newCreateTime(NULL), - newSource( - newNode(newAddress( - newAttribute("category","ipv4-addr"), - newSimpleElement("address", - copy_string(src_ip->AsAddr().AsString().c_str())), - NULL), NULL), - newService( - newSimpleElement("port", - port_to_string(src_port)), - NULL), NULL), - newTarget( - newNode(newAddress( - newAttribute("category","ipv4-addr"), - newSimpleElement("address", - copy_string(dst_ip->AsAddr().AsString().c_str())), - NULL), NULL), - newService( - newSimpleElement("port", - port_to_string(dst_port)), - NULL), NULL), NULL), NULL); - - // if ( validateCurrentDoc() ) - printCurrentMessage(stderr); - return new Val(1, TYPE_BOOL); -#else - builtin_error("Bro was not configured for IDMEF support"); - return new Val(0, TYPE_BOOL); -#endif - %} diff --git a/src/const.bif b/src/const.bif index 499dc63314..7373403c11 100644 --- a/src/const.bif +++ b/src/const.bif @@ -16,6 +16,7 @@ const Tunnel::enable_ip: bool; const Tunnel::enable_ayiya: bool; const Tunnel::enable_teredo: bool; const Tunnel::yielding_teredo_decapsulation: bool; +const Tunnel::delay_teredo_confirmation: bool; const Tunnel::ip_tunnel_timeout: interval; const Threading::heartbeat_interval: interval; diff --git a/src/input.bif b/src/input.bif index f494ef3b2f..199b665fa6 100644 --- a/src/input.bif +++ b/src/input.bif @@ -34,6 +34,10 @@ function Input::__force_update%(id: string%) : bool return new Val(res, TYPE_BOOL); %} +# Options for the input framework + +const accept_unsupported_types: bool; + # Options for Ascii Reader module InputAscii; diff --git a/src/input/Manager.cc b/src/input/Manager.cc index 90d7eae2f4..83e9dc9bc5 100644 --- a/src/input/Manager.cc +++ b/src/input/Manager.cc @@ -388,6 +388,8 @@ bool Manager::CreateEventStream(RecordVal* fval) FuncType* etype = event->FType()->AsFuncType(); + bool allow_file_func = false; + if ( ! etype->IsEvent() ) { reporter->Error("stream event is a function, not an event"); @@ -441,12 +443,20 @@ bool Manager::CreateEventStream(RecordVal* fval) return false; } - if ( !same_type((*args)[2], fields ) ) + if ( ! same_type((*args)[2], fields ) ) { - reporter->Error("Incompatible type for event"); + ODesc desc1; + ODesc desc2; + (*args)[2]->Describe(&desc1); + fields->Describe(&desc2); + reporter->Error("Incompatible type '%s':%s for event, which needs type '%s':%s\n", + type_name((*args)[2]->Tag()), desc1.Description(), + type_name(fields->Tag()), desc2.Description()); return false; } + allow_file_func = BifConst::Input::accept_unsupported_types; + } else @@ -455,7 +465,7 @@ bool Manager::CreateEventStream(RecordVal* fval) vector fieldsV; // vector, because UnrollRecordType needs it - bool status = !UnrollRecordType(&fieldsV, fields, ""); + bool status = (! UnrollRecordType(&fieldsV, fields, "", allow_file_func)); if ( status ) { @@ -603,12 +613,12 @@ bool Manager::CreateTableStream(RecordVal* fval) vector fieldsV; // vector, because we don't know the length beforehands - bool status = !UnrollRecordType(&fieldsV, idx, ""); + bool status = (! UnrollRecordType(&fieldsV, idx, "", false)); int idxfields = fieldsV.size(); if ( val ) // if we are not a set - status = status || !UnrollRecordType(&fieldsV, val, ""); + status = status || ! UnrollRecordType(&fieldsV, val, "", BifConst::Input::accept_unsupported_types); int valfields = fieldsV.size() - idxfields; @@ -766,15 +776,29 @@ bool Manager::RemoveStreamContinuation(ReaderFrontend* reader) return true; } -bool Manager::UnrollRecordType(vector *fields, - const RecordType *rec, const string& nameprepend) +bool Manager::UnrollRecordType(vector *fields, const RecordType *rec, + const string& nameprepend, bool allow_file_func) { - for ( int i = 0; i < rec->NumFields(); i++ ) { if ( ! IsCompatibleType(rec->FieldType(i)) ) - { + { + // If the field is a file or a function type + // and it is optional, we accept it nevertheless. + // This allows importing logfiles containing this + // stuff that we actually cannot read :) + if ( allow_file_func ) + { + if ( ( rec->FieldType(i)->Tag() == TYPE_FILE || + rec->FieldType(i)->Tag() == TYPE_FUNC ) && + rec->FieldDecl(i)->FindAttr(ATTR_OPTIONAL) ) + { + reporter->Info("Encountered incompatible type \"%s\" in table definition for ReaderFrontend. Ignoring field.", type_name(rec->FieldType(i)->Tag())); + continue; + } + } + reporter->Error("Incompatible type \"%s\" in table definition for ReaderFrontend", type_name(rec->FieldType(i)->Tag())); return false; } @@ -783,7 +807,7 @@ bool Manager::UnrollRecordType(vector *fields, { string prep = nameprepend + rec->FieldName(i) + "."; - if ( !UnrollRecordType(fields, rec->FieldType(i)->AsRecordType(), prep) ) + if ( !UnrollRecordType(fields, rec->FieldType(i)->AsRecordType(), prep, allow_file_func) ) { return false; } @@ -1038,9 +1062,7 @@ int Manager::SendEntryTable(Stream* i, const Value* const *vals) if ( ! updated ) { - // throw away. Hence - we quit. And remove the entry from the current dictionary... - // (but why should it be in there? assert this). - assert ( stream->currDict->RemoveEntry(idxhash) == 0 ); + // just quit and delete everything we created. delete idxhash; delete h; return stream->num_val_fields + stream->num_idx_fields; @@ -1206,7 +1228,7 @@ void Manager::EndCurrentSend(ReaderFrontend* reader) Ref(predidx); Ref(val); Ref(ev); - SendEvent(stream->event, 3, ev, predidx, val); + SendEvent(stream->event, 4, stream->description->Ref(), ev, predidx, val); } if ( predidx ) // if we have a stream or an event... @@ -1540,7 +1562,7 @@ bool Manager::Delete(ReaderFrontend* reader, Value* *vals) bool Manager::CallPred(Func* pred_func, const int numvals, ...) { - bool result; + bool result = false; val_list vl(numvals); va_list lP; @@ -1551,10 +1573,13 @@ bool Manager::CallPred(Func* pred_func, const int numvals, ...) va_end(lP); Val* v = pred_func->Call(&vl); - result = v->AsBool(); - Unref(v); + if ( v ) + { + result = v->AsBool(); + Unref(v); + } - return(result); + return result; } bool Manager::SendEvent(const string& name, const int num_vals, Value* *vals) @@ -1668,6 +1693,18 @@ RecordVal* Manager::ValueToRecordVal(const Value* const *vals, Val* fieldVal = 0; if ( request_type->FieldType(i)->Tag() == TYPE_RECORD ) fieldVal = ValueToRecordVal(vals, request_type->FieldType(i)->AsRecordType(), position); + else if ( request_type->FieldType(i)->Tag() == TYPE_FILE || + request_type->FieldType(i)->Tag() == TYPE_FUNC ) + { + // If those two unsupported types are encountered here, they have + // been let through by the type checking. + // That means that they are optional & the user agreed to ignore + // them and has been warned by reporter. + // Hence -> assign null to the field, done. + + // Better check that it really is optional. Uou never know. + assert(request_type->FieldDecl(i)->FindAttr(ATTR_OPTIONAL)); + } else { fieldVal = ValueToVal(vals[*position], request_type->FieldType(i)); @@ -1711,7 +1748,7 @@ int Manager::GetValueLength(const Value* val) { case TYPE_STRING: case TYPE_ENUM: { - length += val->val.string_val.length; + length += val->val.string_val.length + 1; break; } @@ -1811,7 +1848,10 @@ int Manager::CopyValue(char *data, const int startpos, const Value* val) case TYPE_ENUM: { memcpy(data+startpos, val->val.string_val.data, val->val.string_val.length); - return val->val.string_val.length; + // Add a \0 to the end. To be able to hash zero-length + // strings and differentiate from !present. + memset(data + startpos + val->val.string_val.length, 0, 1); + return val->val.string_val.length + 1; } case TYPE_ADDR: @@ -1902,13 +1942,15 @@ HashKey* Manager::HashValues(const int num_elements, const Value* const *vals) const Value* val = vals[i]; if ( val->present ) length += GetValueLength(val); + + // And in any case add 1 for the end-of-field-identifier. + length++; } - if ( length == 0 ) - { - reporter->Error("Input reader sent line where all elements are null values. Ignoring line"); + assert ( length >= num_elements ); + + if ( length == num_elements ) return NULL; - } int position = 0; char *data = (char*) malloc(length); @@ -1920,6 +1962,12 @@ HashKey* Manager::HashValues(const int num_elements, const Value* const *vals) const Value* val = vals[i]; if ( val->present ) position += CopyValue(data, position, val); + + memset(data + position, 1, 1); // Add end-of-field-marker. Does not really matter which value it is, + // it just has to be... something. + + position++; + } HashKey *key = new HashKey(data, length); @@ -1959,7 +2007,7 @@ Val* Manager::ValueToVal(const Value* val, BroType* request_type) case TYPE_STRING: { - BroString *s = new BroString((const u_char*)val->val.string_val.data, val->val.string_val.length, 0); + BroString *s = new BroString((const u_char*)val->val.string_val.data, val->val.string_val.length, 1); return new StringVal(s); } diff --git a/src/input/Manager.h b/src/input/Manager.h index 1590042183..cc81df38b7 100644 --- a/src/input/Manager.h +++ b/src/input/Manager.h @@ -158,7 +158,7 @@ private: // Check if a record is made up of compatible types and return a list // of all fields that are in the record in order. Recursively unrolls // records - bool UnrollRecordType(vector *fields, const RecordType *rec, const string& nameprepend); + bool UnrollRecordType(vector *fields, const RecordType *rec, const string& nameprepend, bool allow_file_func); // Send events void SendEvent(EventHandlerPtr ev, const int numvals, ...); diff --git a/src/input/ReaderBackend.cc b/src/input/ReaderBackend.cc index 88a78c3cd7..81060be7d5 100644 --- a/src/input/ReaderBackend.cc +++ b/src/input/ReaderBackend.cc @@ -191,6 +191,9 @@ void ReaderBackend::SendEntry(Value* *vals) bool ReaderBackend::Init(const int arg_num_fields, const threading::Field* const* arg_fields) { + if ( Failed() ) + return true; + num_fields = arg_num_fields; fields = arg_fields; @@ -210,7 +213,9 @@ bool ReaderBackend::Init(const int arg_num_fields, bool ReaderBackend::OnFinish(double network_time) { - DoClose(); + if ( ! Failed() ) + DoClose(); + disabled = true; // frontend disables itself when it gets the Close-message. SendOut(new ReaderClosedMessage(frontend)); @@ -231,6 +236,9 @@ bool ReaderBackend::Update() if ( disabled ) return false; + if ( Failed() ) + return true; + bool success = DoUpdate(); if ( ! success ) DisableFrontend(); @@ -248,6 +256,9 @@ void ReaderBackend::DisableFrontend() bool ReaderBackend::OnHeartbeat(double network_time, double current_time) { + if ( Failed() ) + return true; + return DoHeartbeat(network_time, current_time); } diff --git a/src/input/readers/Ascii.cc b/src/input/readers/Ascii.cc index fd936b07b6..e9cba27205 100644 --- a/src/input/readers/Ascii.cc +++ b/src/input/readers/Ascii.cc @@ -11,6 +11,7 @@ #include #include #include +#include using namespace input::reader; using threading::Value; @@ -209,6 +210,42 @@ bool Ascii::GetLine(string& str) return false; } +bool Ascii::CheckNumberError(const string& s, const char * end) + { + // Do this check first, before executing s.c_str() or similar. + // otherwise the value to which *end is pointing at the moment might + // be gone ... + bool endnotnull = (*end != '\0'); + + if ( s.length() == 0 ) + { + Error("Got empty string for number field"); + return true; + } + + if ( end == s.c_str() ) { + Error(Fmt("String '%s' contained no parseable number", s.c_str())); + return true; + } + + if ( endnotnull ) + Warning(Fmt("Number '%s' contained non-numeric trailing characters. Ignored trailing characters '%s'", s.c_str(), end)); + + if ( errno == EINVAL ) + { + Error(Fmt("String '%s' could not be converted to a number", s.c_str())); + return true; + } + + else if ( errno == ERANGE ) + { + Error(Fmt("Number '%s' out of supported range.", s.c_str())); + return true; + } + + return false; + } + Value* Ascii::EntryToVal(string s, FieldMapping field) { @@ -216,10 +253,13 @@ Value* Ascii::EntryToVal(string s, FieldMapping field) return new Value(field.type, false); Value* val = new Value(field.type, true); + char* end = 0; + errno = 0; switch ( field.type ) { case TYPE_ENUM: case TYPE_STRING: + s = get_unescaped_string(s); val->val.string_val.length = s.size(); val->val.string_val.data = copy_string(s.c_str()); break; @@ -238,27 +278,37 @@ Value* Ascii::EntryToVal(string s, FieldMapping field) break; case TYPE_INT: - val->val.int_val = atoi(s.c_str()); + val->val.int_val = strtoll(s.c_str(), &end, 10); + if ( CheckNumberError(s, end) ) + return 0; break; case TYPE_DOUBLE: case TYPE_TIME: case TYPE_INTERVAL: - val->val.double_val = atof(s.c_str()); + val->val.double_val = strtod(s.c_str(), &end); + if ( CheckNumberError(s, end) ) + return 0; break; case TYPE_COUNT: case TYPE_COUNTER: - val->val.uint_val = atoi(s.c_str()); + val->val.uint_val = strtoull(s.c_str(), &end, 10); + if ( CheckNumberError(s, end) ) + return 0; break; case TYPE_PORT: - val->val.port_val.port = atoi(s.c_str()); + val->val.port_val.port = strtoull(s.c_str(), &end, 10); + if ( CheckNumberError(s, end) ) + return 0; + val->val.port_val.proto = TRANSPORT_UNKNOWN; break; case TYPE_SUBNET: { + s = get_unescaped_string(s); size_t pos = s.find("/"); if ( pos == s.npos ) { @@ -266,7 +316,11 @@ Value* Ascii::EntryToVal(string s, FieldMapping field) return 0; } - int width = atoi(s.substr(pos+1).c_str()); + uint8_t width = (uint8_t) strtol(s.substr(pos+1).c_str(), &end, 10); + + if ( CheckNumberError(s, end) ) + return 0; + string addr = s.substr(0, pos); val->val.subnet_val.prefix = StringToAddr(addr); @@ -275,6 +329,7 @@ Value* Ascii::EntryToVal(string s, FieldMapping field) } case TYPE_ADDR: + s = get_unescaped_string(s); val->val.addr_val = StringToAddr(s); break; @@ -288,7 +343,10 @@ Value* Ascii::EntryToVal(string s, FieldMapping field) // how many entries do we have... unsigned int length = 1; for ( unsigned int i = 0; i < s.size(); i++ ) - if ( s[i] == ',' ) length++; + { + if ( s[i] == set_separator[0] ) + length++; + } unsigned int pos = 0; @@ -342,9 +400,24 @@ Value* Ascii::EntryToVal(string s, FieldMapping field) pos++; } + // Test if the string ends with a set_separator... or if the + // complete string is empty. In either of these cases we have + // to push an empty val on top of it. + if ( s.empty() || *s.rbegin() == set_separator[0] ) + { + lvals[pos] = EntryToVal("", field.subType()); + if ( lvals[pos] == 0 ) + { + Error("Error while trying to add empty set element"); + return 0; + } + + pos++; + } + if ( pos != length ) { - Error("Internal error while parsing set: did not find all elements"); + Error(Fmt("Internal error while parsing set: did not find all elements: %s", s.c_str())); return 0; } @@ -428,6 +501,7 @@ bool Ascii::DoUpdate() while ( GetLine(line ) ) { // split on tabs + bool error = false; istringstream splitstream(line); map stringfields; @@ -438,8 +512,6 @@ bool Ascii::DoUpdate() if ( ! getline(splitstream, s, separator[0]) ) break; - s = get_unescaped_string(s); - stringfields[pos] = s; pos++; } @@ -474,8 +546,9 @@ bool Ascii::DoUpdate() Value* val = EntryToVal(stringfields[(*fit).position], *fit); if ( val == 0 ) { - Error("Could not convert String value to Val"); - return false; + Error(Fmt("Could not convert line '%s' to Val. Ignoring line.", line.c_str())); + error = true; + break; } if ( (*fit).secondary_position != -1 ) @@ -492,6 +565,19 @@ bool Ascii::DoUpdate() fpos++; } + if ( error ) + { + // Encountered non-fatal error, ignoring line. But + // first, delete all successfully read fields and the + // array structure. + + for ( int i = 0; i < fpos; i++ ) + delete fields[fpos]; + + delete [] fields; + continue; + } + //printf("fpos: %d, second.num_fields: %d\n", fpos, (*it).second.num_fields); assert ( fpos == NumFields() ); diff --git a/src/input/readers/Ascii.h b/src/input/readers/Ascii.h index e1506cbe82..6e693fc74b 100644 --- a/src/input/readers/Ascii.h +++ b/src/input/readers/Ascii.h @@ -48,6 +48,7 @@ private: bool ReadHeader(bool useCached); bool GetLine(string& str); threading::Value* EntryToVal(string s, FieldMapping type); + bool CheckNumberError(const string& s, const char * end); ifstream* file; time_t mtime; diff --git a/src/logging/Manager.cc b/src/logging/Manager.cc index 026eb6702a..bea19018ee 100644 --- a/src/logging/Manager.cc +++ b/src/logging/Manager.cc @@ -95,6 +95,7 @@ struct Manager::WriterInfo { Func* postprocessor; WriterFrontend* writer; WriterBackend::WriterInfo* info; + bool from_remote; string instantiating_filter; }; @@ -249,6 +250,29 @@ Manager::WriterInfo* Manager::FindWriter(WriterFrontend* writer) return 0; } +bool Manager::CompareFields(const Filter* filter, const WriterFrontend* writer) + { + if ( filter->num_fields != writer->NumFields() ) + return false; + + for ( int i = 0; i < filter->num_fields; ++ i) + if ( filter->fields[i]->type != writer->Fields()[i]->type ) + return false; + + return true; + } + +bool Manager::CheckFilterWriterConflict(const WriterInfo* winfo, const Filter* filter) + { + if ( winfo->from_remote ) + // If the writer was instantiated as a result of remote logging, then + // a filter and writer are only compatible if field types match + return ! CompareFields(filter, winfo->writer); + else + // If the writer was instantiated locally, it is bound to one filter + return winfo->instantiating_filter != filter->name; + } + void Manager::RemoveDisabledWriters(Stream* stream) { list disabled; @@ -695,16 +719,13 @@ bool Manager::Write(EnumVal* id, RecordVal* columns) int result = 1; - try + Val* v = filter->pred->Call(&vl); + if ( v ) { - Val* v = filter->pred->Call(&vl); result = v->AsBool(); Unref(v); } - catch ( InterpreterException& e ) - { /* Already reported. */ } - if ( ! result ) continue; } @@ -735,15 +756,10 @@ bool Manager::Write(EnumVal* id, RecordVal* columns) Val* v = 0; - try - { - v = filter->path_func->Call(&vl); - } + v = filter->path_func->Call(&vl); - catch ( InterpreterException& e ) - { + if ( ! v ) return false; - } if ( ! v->Type()->Tag() == TYPE_STRING ) { @@ -767,22 +783,43 @@ bool Manager::Write(EnumVal* id, RecordVal* columns) #endif } + Stream::WriterPathPair wpp(filter->writer->AsEnum(), path); + // See if we already have a writer for this path. - Stream::WriterMap::iterator w = - stream->writers.find(Stream::WriterPathPair(filter->writer->AsEnum(), path)); + Stream::WriterMap::iterator w = stream->writers.find(wpp); + + if ( w != stream->writers.end() && + CheckFilterWriterConflict(w->second, filter) ) + { + // Auto-correct path due to conflict over the writer/path pairs. + string instantiator = w->second->instantiating_filter; + string new_path; + unsigned int i = 2; + + do { + char num[32]; + snprintf(num, sizeof(num), "-%u", i++); + new_path = path + num; + wpp.second = new_path; + w = stream->writers.find(wpp); + } while ( w != stream->writers.end() && + CheckFilterWriterConflict(w->second, filter) ); + + Unref(filter->path_val); + filter->path_val = new StringVal(new_path.c_str()); + + reporter->Warning("Write using filter '%s' on path '%s' changed to" + " use new path '%s' to avoid conflict with filter '%s'", + filter->name.c_str(), path.c_str(), new_path.c_str(), + instantiator.c_str()); + + path = filter->path = filter->path_val->AsString()->CheckString(); + } WriterFrontend* writer = 0; if ( w != stream->writers.end() ) { - if ( w->second->instantiating_filter != filter->name ) - { - reporter->Warning("Skipping write to filter '%s' on path '%s'" - " because filter '%s' has already instantiated the same" - " writer type for that path", filter->name.c_str(), - filter->path.c_str(), w->second->instantiating_filter.c_str()); - continue; - } // We know this writer already. writer = w->second->writer; } @@ -819,8 +856,8 @@ bool Manager::Write(EnumVal* id, RecordVal* columns) // CreateWriter() will set the other fields in info. writer = CreateWriter(stream->id, filter->writer, - info, filter->num_fields, - arg_fields, filter->local, filter->remote, filter->name); + info, filter->num_fields, arg_fields, filter->local, + filter->remote, false, filter->name); if ( ! writer ) { @@ -1019,7 +1056,7 @@ threading::Value** Manager::RecordToFilterVals(Stream* stream, Filter* filter, } WriterFrontend* Manager::CreateWriter(EnumVal* id, EnumVal* writer, WriterBackend::WriterInfo* info, - int num_fields, const threading::Field* const* fields, bool local, bool remote, + int num_fields, const threading::Field* const* fields, bool local, bool remote, bool from_remote, const string& instantiating_filter) { Stream* stream = FindStream(id); @@ -1044,6 +1081,7 @@ WriterFrontend* Manager::CreateWriter(EnumVal* id, EnumVal* writer, WriterBacken winfo->interval = 0; winfo->postprocessor = 0; winfo->info = info; + winfo->from_remote = from_remote; winfo->instantiating_filter = instantiating_filter; // Search for a corresponding filter for the writer/path pair and use its @@ -1210,12 +1248,16 @@ bool Manager::Flush(EnumVal* id) void Manager::Terminate() { // Make sure we process all the pending rotations. - while ( rotations_pending ) + + while ( rotations_pending > 0 ) { thread_mgr->ForceProcessing(); // A blatant layering violation ... usleep(1000); } + if ( rotations_pending < 0 ) + reporter->InternalError("Negative pending log rotations: %d", rotations_pending); + for ( vector::iterator s = streams.begin(); s != streams.end(); ++s ) { if ( ! *s ) @@ -1329,13 +1371,18 @@ void Manager::Rotate(WriterInfo* winfo) } bool Manager::FinishedRotation(WriterFrontend* writer, const char* new_name, const char* old_name, - double open, double close, bool terminating) + double open, double close, bool success, bool terminating) { + assert(writer); + --rotations_pending; - if ( ! writer ) - // Writer didn't produce local output. + if ( ! success ) + { + DBG_LOG(DBG_LOGGING, "Non-successful rotating writer '%s', file '%s' at %.6f,", + writer->Name(), filename, network_time); return true; + } DBG_LOG(DBG_LOGGING, "Finished rotating %s at %.6f, new name %s", writer->Name(), network_time, new_name); @@ -1369,16 +1416,12 @@ bool Manager::FinishedRotation(WriterFrontend* writer, const char* new_name, con int result = 0; - try + Val* v = func->Call(&vl); + if ( v ) { - Val* v = func->Call(&vl); result = v->AsBool(); Unref(v); } - catch ( InterpreterException& e ) - { /* Already reported. */ } - return result; } - diff --git a/src/logging/Manager.h b/src/logging/Manager.h index d2041592c1..90ad944bc6 100644 --- a/src/logging/Manager.h +++ b/src/logging/Manager.h @@ -153,6 +153,7 @@ public: protected: friend class WriterFrontend; friend class RotationFinishedMessage; + friend class RotationFailedMessage; friend class ::RemoteSerializer; friend class ::RotationTimer; @@ -165,7 +166,7 @@ protected: // Takes ownership of fields and info. WriterFrontend* CreateWriter(EnumVal* id, EnumVal* writer, WriterBackend::WriterInfo* info, int num_fields, const threading::Field* const* fields, - bool local, bool remote, const string& instantiating_filter=""); + bool local, bool remote, bool from_remote, const string& instantiating_filter=""); // Takes ownership of values.. bool Write(EnumVal* id, EnumVal* writer, string path, @@ -176,7 +177,7 @@ protected: // Signals that a file has been rotated. bool FinishedRotation(WriterFrontend* writer, const char* new_name, const char* old_name, - double open, double close, bool terminating); + double open, double close, bool success, bool terminating); // Deletes the values as passed into Write(). void DeleteVals(int num_fields, threading::Value** vals); @@ -199,6 +200,8 @@ private: void Rotate(WriterInfo* info); Filter* FindFilter(EnumVal* id, StringVal* filter); WriterInfo* FindWriter(WriterFrontend* writer); + bool CompareFields(const Filter* filter, const WriterFrontend* writer); + bool CheckFilterWriterConflict(const WriterInfo* winfo, const Filter* filter); vector streams; // Indexed by stream enum. int rotations_pending; // Number of rotations not yet finished. diff --git a/src/logging/WriterBackend.cc b/src/logging/WriterBackend.cc index 2933062eff..47fdec27ef 100644 --- a/src/logging/WriterBackend.cc +++ b/src/logging/WriterBackend.cc @@ -19,10 +19,10 @@ class RotationFinishedMessage : public threading::OutputMessage { public: RotationFinishedMessage(WriterFrontend* writer, const char* new_name, const char* old_name, - double open, double close, bool terminating) + double open, double close, bool success, bool terminating) : threading::OutputMessage("RotationFinished", writer), new_name(copy_string(new_name)), old_name(copy_string(old_name)), open(open), - close(close), terminating(terminating) { } + close(close), success(success), terminating(terminating) { } virtual ~RotationFinishedMessage() { @@ -32,7 +32,7 @@ public: virtual bool Process() { - return log_mgr->FinishedRotation(Object(), new_name, old_name, open, close, terminating); + return log_mgr->FinishedRotation(Object(), new_name, old_name, open, close, success, terminating); } private: @@ -40,6 +40,7 @@ private: const char* old_name; double open; double close; + bool success; bool terminating; }; @@ -126,6 +127,7 @@ WriterBackend::WriterBackend(WriterFrontend* arg_frontend) : MsgThread() buffering = true; frontend = arg_frontend; info = new WriterInfo(frontend->Info()); + rotation_counter = 0; SetName(frontend->Name()); } @@ -160,7 +162,15 @@ void WriterBackend::DeleteVals(int num_writes, Value*** vals) bool WriterBackend::FinishedRotation(const char* new_name, const char* old_name, double open, double close, bool terminating) { - SendOut(new RotationFinishedMessage(frontend, new_name, old_name, open, close, terminating)); + --rotation_counter; + SendOut(new RotationFinishedMessage(frontend, new_name, old_name, open, close, true, terminating)); + return true; + } + +bool WriterBackend::FinishedRotation() + { + --rotation_counter; + SendOut(new RotationFinishedMessage(frontend, 0, 0, 0, 0, false, false)); return true; } @@ -174,6 +184,9 @@ bool WriterBackend::Init(int arg_num_fields, const Field* const* arg_fields) num_fields = arg_num_fields; fields = arg_fields; + if ( Failed() ) + return true; + if ( ! DoInit(*info, arg_num_fields, arg_fields) ) { DisableFrontend(); @@ -222,12 +235,15 @@ bool WriterBackend::Write(int arg_num_fields, int num_writes, Value*** vals) bool success = true; - for ( int j = 0; j < num_writes; j++ ) + if ( ! Failed() ) { - success = DoWrite(num_fields, fields, vals[j]); + for ( int j = 0; j < num_writes; j++ ) + { + success = DoWrite(num_fields, fields, vals[j]); - if ( ! success ) - break; + if ( ! success ) + break; + } } DeleteVals(num_writes, vals); @@ -244,6 +260,9 @@ bool WriterBackend::SetBuf(bool enabled) // No change. return true; + if ( Failed() ) + return true; + buffering = enabled; if ( ! DoSetBuf(enabled) ) @@ -258,17 +277,32 @@ bool WriterBackend::SetBuf(bool enabled) bool WriterBackend::Rotate(const char* rotated_path, double open, double close, bool terminating) { + if ( Failed() ) + return true; + + rotation_counter = 1; + if ( ! DoRotate(rotated_path, open, close, terminating) ) { DisableFrontend(); return false; } + // Insurance against broken writers. + if ( rotation_counter > 0 ) + InternalError(Fmt("writer %s did not call FinishedRotation() in DoRotation()", Name())); + + if ( rotation_counter < 0 ) + InternalError(Fmt("writer %s called FinishedRotation() more than once in DoRotation()", Name())); + return true; } bool WriterBackend::Flush(double network_time) { + if ( Failed() ) + return true; + if ( ! DoFlush(network_time) ) { DisableFrontend(); @@ -280,11 +314,17 @@ bool WriterBackend::Flush(double network_time) bool WriterBackend::OnFinish(double network_time) { + if ( Failed() ) + return true; + return DoFinish(network_time); } bool WriterBackend::OnHeartbeat(double network_time, double current_time) { + if ( Failed() ) + return true; + SendOut(new FlushWriteBufferMessage(frontend)); return DoHeartbeat(network_time, current_time); } diff --git a/src/logging/WriterBackend.h b/src/logging/WriterBackend.h index d5f2be225e..89185619c4 100644 --- a/src/logging/WriterBackend.h +++ b/src/logging/WriterBackend.h @@ -182,6 +182,8 @@ public: /** * Disables the frontend that has instantiated this backend. Once * disabled,the frontend will not send any further message over. + * + * TODO: Do we still need this method (and the corresponding message)? */ void DisableFrontend(); @@ -208,11 +210,15 @@ public: bool IsBuf() { return buffering; } /** - * Signals that a file has been rotated. This must be called by a - * writer's implementation of DoRotate() once rotation has finished. + * Signals that a file has been successfully rotated and any + * potential post-processor can now run. * * Most of the parameters should be passed through from DoRotate(). * + * Note: Exactly one of the two FinishedRotation() methods must be + * called by a writer's implementation of DoRotate() once rotation + * has finished. + * * @param new_name The filename of the rotated file. * * @param old_name The filename of the original file. @@ -227,6 +233,29 @@ public: bool FinishedRotation(const char* new_name, const char* old_name, double open, double close, bool terminating); + /** + * Signals that a file rotation request has been processed, but no + * further post-processing needs to be performed (either because + * there was an error, or there was nothing to rotate to begin with + * with this writer). + * + * Note: Exactly one of the two FinishedRotation() methods must be + * called by a writer's implementation of DoRotate() once rotation + * has finished. + * + * @param new_name The filename of the rotated file. + * + * @param old_name The filename of the original file. + * + * @param open: The timestamp when the original file was opened. + * + * @param close: The timestamp when the origina file was closed. + * + * @param terminating: True if the original rotation request occured + * due to the main Bro process shutting down. + */ + bool FinishedRotation(); + /** Helper method to render an IP address as a string. * * @param addr The address. @@ -323,8 +352,8 @@ protected: * Writer-specific method implementing log rotation. Most directly * this only applies to writers writing into files, which should then * close the current file and open a new one. However, a writer may - * also trigger other apppropiate actions if semantics are similar. * - * Once rotation has finished, the implementation must call + * also trigger other apppropiate actions if semantics are similar. + * Once rotation has finished, the implementation *must* call * FinishedRotation() to signal the log manager that potential * postprocessors can now run. * @@ -386,6 +415,8 @@ private: int num_fields; // Number of log fields. const threading::Field* const* fields; // Log fields. bool buffering; // True if buffering is enabled. + + int rotation_counter; // Tracks FinishedRotation() calls. }; diff --git a/src/logging/WriterFrontend.cc b/src/logging/WriterFrontend.cc index 7c8f6861cf..a97f48c1ed 100644 --- a/src/logging/WriterFrontend.cc +++ b/src/logging/WriterFrontend.cc @@ -248,9 +248,8 @@ void WriterFrontend::Rotate(const char* rotated_path, double open, double close, if ( backend ) backend->SendIn(new RotateMessage(backend, this, rotated_path, open, close, terminating)); else - // Still signal log manager that we're done, but signal that - // nothing happened by setting the writer to zeri. - log_mgr->FinishedRotation(0, "", rotated_path, open, close, terminating); + // Still signal log manager that we're done. + log_mgr->FinishedRotation(this, 0, 0, 0, 0, false, terminating); } void WriterFrontend::DeleteVals(Value** vals) diff --git a/src/logging/writers/Ascii.cc b/src/logging/writers/Ascii.cc index 4d2f59ea72..11b322f5a3 100644 --- a/src/logging/writers/Ascii.cc +++ b/src/logging/writers/Ascii.cc @@ -81,18 +81,15 @@ void Ascii::CloseFile(double t) return; if ( include_meta ) - { - string ts = t ? Timestamp(t) : string(""); - WriteHeaderField("end", ts); - } + WriteHeaderField("close", Timestamp(0)); - close(fd); + safe_close(fd); fd = 0; } bool Ascii::DoInit(const WriterInfo& info, int num_fields, const Field* const * fields) { - assert(! fd); + assert(! fd); string path = info.path; @@ -124,8 +121,6 @@ bool Ascii::DoInit(const WriterInfo& info, int num_fields, const Field* const * if ( ! safe_write(fd, str.c_str(), str.length()) ) goto write_error; - string ts = Timestamp(info.network_time); - if ( ! (WriteHeaderField("set_separator", get_escaped_string( string(set_separator, set_separator_len), false)) && WriteHeaderField("empty_field", get_escaped_string( @@ -133,7 +128,7 @@ bool Ascii::DoInit(const WriterInfo& info, int num_fields, const Field* const * WriteHeaderField("unset_field", get_escaped_string( string(unset_field, unset_field_len), false)) && WriteHeaderField("path", get_escaped_string(path, false)) && - WriteHeaderField("start", ts)) ) + WriteHeaderField("open", Timestamp(0))) ) goto write_error; for ( int i = 0; i < num_fields; ++i ) @@ -364,7 +359,7 @@ bool Ascii::DoWrite(int num_fields, const Field* const * fields, if ( ! safe_write(fd, bytes, len) ) goto write_error; - if ( IsBuf() ) + if ( ! IsBuf() ) fsync(fd); return true; @@ -378,7 +373,10 @@ bool Ascii::DoRotate(const char* rotated_path, double open, double close, bool t { // Don't rotate special files or if there's not one currently open. if ( ! fd || IsSpecial(Info().path) ) + { + FinishedRotation(); return true; + } CloseFile(close); @@ -419,6 +417,16 @@ string Ascii::Timestamp(double t) { time_t teatime = time_t(t); + if ( ! teatime ) + { + // Use wall clock. + struct timeval tv; + if ( gettimeofday(&tv, 0) < 0 ) + Error("gettimeofday failed"); + else + teatime = tv.tv_sec; + } + struct tm tmbuf; struct tm* tm = localtime_r(&teatime, &tmbuf); diff --git a/src/logging/writers/Ascii.h b/src/logging/writers/Ascii.h index cb82860cb7..cf0190aa80 100644 --- a/src/logging/writers/Ascii.h +++ b/src/logging/writers/Ascii.h @@ -35,7 +35,7 @@ private: bool DoWriteOne(ODesc* desc, threading::Value* val, const threading::Field* field); bool WriteHeaderField(const string& key, const string& value); void CloseFile(double t); - string Timestamp(double t); + string Timestamp(double t); // Uses current time if t is zero. int fd; string fname; diff --git a/src/logging/writers/DataSeries.cc b/src/logging/writers/DataSeries.cc index 7d3053e341..bc5a82ec54 100644 --- a/src/logging/writers/DataSeries.cc +++ b/src/logging/writers/DataSeries.cc @@ -243,8 +243,25 @@ bool DataSeries::OpenLog(string path) log_file->writeExtentLibrary(log_types); for( size_t i = 0; i < schema_list.size(); ++i ) - extents.insert(std::make_pair(schema_list[i].field_name, - GeneralField::create(log_series, schema_list[i].field_name))); + { + string fn = schema_list[i].field_name; + GeneralField* gf = 0; +#ifdef USE_PERFTOOLS_DEBUG + { + // GeneralField isn't cleaning up some results of xml parsing, reported + // here: https://github.com/dataseries/DataSeries/issues/1 + // Ignore for now to make leak tests pass. There's confidence that + // we do clean up the GeneralField* since the ExtentSeries dtor for + // member log_series would trigger an assert if dynamically allocated + // fields aren't deleted beforehand. + HeapLeakChecker::Disabler disabler; +#endif + gf = GeneralField::create(log_series, fn); +#ifdef USE_PERFTOOLS_DEBUG + } +#endif + extents.insert(std::make_pair(fn, gf)); + } if ( ds_extent_size < ROW_MIN ) { diff --git a/src/logging/writers/ElasticSearch.cc b/src/logging/writers/ElasticSearch.cc index 2da79ed7b9..ae825ac997 100644 --- a/src/logging/writers/ElasticSearch.cc +++ b/src/logging/writers/ElasticSearch.cc @@ -48,7 +48,7 @@ ElasticSearch::ElasticSearch(WriterFrontend* frontend) : WriterBackend(frontend) last_send = current_time(); failing = false; - transfer_timeout = BifConst::LogElasticSearch::transfer_timeout * 1000; + transfer_timeout = static_cast(BifConst::LogElasticSearch::transfer_timeout); curl_handle = HTTPSetup(); } @@ -322,9 +322,7 @@ bool ElasticSearch::DoRotate(const char* rotated_path, double open, double close } if ( ! FinishedRotation(current_index.c_str(), prev_index.c_str(), open, close, terminating) ) - { Error(Fmt("error rotating %s to %s", prev_index.c_str(), current_index.c_str())); - } return true; } @@ -359,10 +357,10 @@ CURL* ElasticSearch::HTTPSetup() return handle; } -bool ElasticSearch::HTTPReceive(void* ptr, int size, int nmemb, void* userdata) +size_t ElasticSearch::HTTPReceive(void* ptr, int size, int nmemb, void* userdata) { //TODO: Do some verification on the result? - return true; + return size; } bool ElasticSearch::HTTPSend(CURL *handle) @@ -373,7 +371,11 @@ bool ElasticSearch::HTTPSend(CURL *handle) // The best (only?) way to disable that is to just use HTTP 1.0 curl_easy_setopt(handle, CURLOPT_HTTP_VERSION, CURL_HTTP_VERSION_1_0); - //curl_easy_setopt(handle, CURLOPT_TIMEOUT_MS, transfer_timeout); + // Some timeout options. These will need more attention later. + curl_easy_setopt(handle, CURLOPT_NOSIGNAL, 1); + curl_easy_setopt(handle, CURLOPT_CONNECTTIMEOUT, transfer_timeout); + curl_easy_setopt(handle, CURLOPT_TIMEOUT, transfer_timeout); + curl_easy_setopt(handle, CURLOPT_DNS_CACHE_TIMEOUT, 60*60); CURLcode return_code = curl_easy_perform(handle); @@ -386,12 +388,16 @@ bool ElasticSearch::HTTPSend(CURL *handle) { if ( ! failing ) Error(Fmt("ElasticSearch server may not be accessible.")); + + break; } case CURLE_OPERATION_TIMEDOUT: { if ( ! failing ) Warning(Fmt("HTTP operation with elasticsearch server timed out at %" PRIu64 " msecs.", transfer_timeout)); + + break; } case CURLE_OK: @@ -403,10 +409,13 @@ bool ElasticSearch::HTTPSend(CURL *handle) return true; else if ( ! failing ) Error(Fmt("Received a non-successful status code back from ElasticSearch server, check the elasticsearch server log.")); + + break; } default: { + break; } } // The "successful" return happens above diff --git a/src/logging/writers/ElasticSearch.h b/src/logging/writers/ElasticSearch.h index 0d863f2f19..fef0a00ffd 100644 --- a/src/logging/writers/ElasticSearch.h +++ b/src/logging/writers/ElasticSearch.h @@ -45,7 +45,7 @@ private: bool UpdateIndex(double now, double rinterval, double rbase); CURL* HTTPSetup(); - bool HTTPReceive(void* ptr, int size, int nmemb, void* userdata); + size_t HTTPReceive(void* ptr, int size, int nmemb, void* userdata); bool HTTPSend(CURL *handle); // Buffers, etc. @@ -68,7 +68,7 @@ private: string path; string index_prefix; - uint64 transfer_timeout; + long transfer_timeout; bool failing; uint64 batch_size; diff --git a/src/main.cc b/src/main.cc index 407f67c9af..5999186240 100644 --- a/src/main.cc +++ b/src/main.cc @@ -337,6 +337,8 @@ void terminate_bro() delete log_mgr; delete thread_mgr; delete reporter; + + reporter = 0; } void termination_signal() @@ -380,6 +382,8 @@ static void bro_new_handler() int main(int argc, char** argv) { + std::set_new_handler(bro_new_handler); + brofiler.ReadStats(); bro_argc = argc; diff --git a/src/modp_numtoa.c b/src/modp_numtoa.c index 6deb8a70ed..2024f7c55b 100644 --- a/src/modp_numtoa.c +++ b/src/modp_numtoa.c @@ -56,7 +56,7 @@ void modp_uitoa10(uint32_t value, char* str) void modp_litoa10(int64_t value, char* str) { char* wstr=str; - unsigned long uvalue = (value < 0) ? -value : value; + uint64_t uvalue = (value < 0) ? -value : value; // Conversion. Number is reversed. do *wstr++ = (char)(48 + (uvalue % 10)); while(uvalue /= 10); diff --git a/src/nb_dns.c b/src/nb_dns.c index d3b3c5c4de..3051be9bc2 100644 --- a/src/nb_dns.c +++ b/src/nb_dns.c @@ -124,7 +124,7 @@ nb_dns_init(char *errstr) nd->s = -1; /* XXX should be able to init static hostent struct some other way */ - (void)gethostbyname("localhost."); + (void)gethostbyname("localhost"); if ((_res.options & RES_INIT) == 0 && res_init() == -1) { snprintf(errstr, NB_DNS_ERRSIZE, "res_init() failed"); diff --git a/src/parse.y b/src/parse.y index 75e09dc60f..c1f6ddd96e 100644 --- a/src/parse.y +++ b/src/parse.y @@ -2,7 +2,7 @@ // See the file "COPYING" in the main distribution directory for copyright. %} -%expect 90 +%expect 87 %token TOK_ADD TOK_ADD_TO TOK_ADDR TOK_ANY %token TOK_ATENDIF TOK_ATELSE TOK_ATIF TOK_ATIFDEF TOK_ATIFNDEF @@ -14,7 +14,7 @@ %token TOK_NEXT TOK_OF TOK_PATTERN TOK_PATTERN_TEXT %token TOK_PORT TOK_PRINT TOK_RECORD TOK_REDEF %token TOK_REMOVE_FROM TOK_RETURN TOK_SCHEDULE TOK_SET -%token TOK_STRING TOK_SUBNET TOK_SWITCH TOK_TABLE TOK_THIS +%token TOK_STRING TOK_SUBNET TOK_SWITCH TOK_TABLE %token TOK_TIME TOK_TIMEOUT TOK_TIMER TOK_TYPE TOK_UNION TOK_VECTOR TOK_WHEN %token TOK_ATTR_ADD_FUNC TOK_ATTR_ATTR TOK_ATTR_ENCRYPT TOK_ATTR_DEFAULT @@ -22,7 +22,7 @@ %token TOK_ATTR_ROTATE_SIZE TOK_ATTR_DEL_FUNC TOK_ATTR_EXPIRE_FUNC %token TOK_ATTR_EXPIRE_CREATE TOK_ATTR_EXPIRE_READ TOK_ATTR_EXPIRE_WRITE %token TOK_ATTR_PERSISTENT TOK_ATTR_SYNCHRONIZED -%token TOK_ATTR_DISABLE_PRINT_HOOK TOK_ATTR_RAW_OUTPUT TOK_ATTR_MERGEABLE +%token TOK_ATTR_RAW_OUTPUT TOK_ATTR_MERGEABLE %token TOK_ATTR_PRIORITY TOK_ATTR_GROUP TOK_ATTR_LOG TOK_ATTR_ERROR_HANDLER %token TOK_ATTR_TYPE_COLUMN @@ -118,7 +118,6 @@ extern const char* g_curr_debug_error; #define YYLTYPE yyltype -Expr* bro_this = 0; int in_init = 0; int in_record = 0; bool resolving_global_ID = false; @@ -584,12 +583,6 @@ expr: $$ = new ConstExpr(new PatternVal($1)); } - | TOK_THIS - { - set_location(@1); - $$ = bro_this->Ref(); - } - | '|' expr '|' { set_location(@1, @3); @@ -1297,8 +1290,6 @@ attr: { $$ = new Attr(ATTR_ENCRYPT); } | TOK_ATTR_ENCRYPT '=' expr { $$ = new Attr(ATTR_ENCRYPT, $3); } - | TOK_ATTR_DISABLE_PRINT_HOOK - { $$ = new Attr(ATTR_DISABLE_PRINT_HOOK); } | TOK_ATTR_RAW_OUTPUT { $$ = new Attr(ATTR_RAW_OUTPUT); } | TOK_ATTR_MERGEABLE diff --git a/src/scan.l b/src/scan.l index 645ce659cd..6c87766781 100644 --- a/src/scan.l +++ b/src/scan.l @@ -306,7 +306,6 @@ string return TOK_STRING; subnet return TOK_SUBNET; switch return TOK_SWITCH; table return TOK_TABLE; -this return TOK_THIS; time return TOK_TIME; timeout return TOK_TIMEOUT; timer return TOK_TIMER; @@ -320,7 +319,6 @@ when return TOK_WHEN; &create_expire return TOK_ATTR_EXPIRE_CREATE; &default return TOK_ATTR_DEFAULT; &delete_func return TOK_ATTR_DEL_FUNC; -&disable_print_hook return TOK_ATTR_DISABLE_PRINT_HOOK; &raw_output return TOK_ATTR_RAW_OUTPUT; &encrypt return TOK_ATTR_ENCRYPT; &error_handler return TOK_ATTR_ERROR_HANDLER; @@ -437,9 +435,7 @@ F RET_CONST(new Val(false, TYPE_BOOL)) } {D} { - // TODO: check if we can use strtoull instead of atol, - // and similarly for {HEX}. - RET_CONST(new Val(static_cast(atol(yytext)), + RET_CONST(new Val(static_cast(strtoull(yytext, (char**) NULL, 10)), TYPE_COUNT)) } {FLOAT} RET_CONST(new Val(atof(yytext), TYPE_DOUBLE)) @@ -481,12 +477,6 @@ F RET_CONST(new Val(false, TYPE_BOOL)) RET_CONST(new PortVal(p, TRANSPORT_UNKNOWN)) } -({D}"."){3}{D} RET_CONST(new AddrVal(yytext)) - -"0x"{HEX}+ RET_CONST(new Val(static_cast(strtol(yytext, 0, 16)), TYPE_COUNT)) - -{H}("."{H})+ RET_CONST(dns_mgr->LookupHost(yytext)) - {FLOAT}{OWS}day(s?) RET_CONST(new IntervalVal(atof(yytext),Days)) {FLOAT}{OWS}hr(s?) RET_CONST(new IntervalVal(atof(yytext),Hours)) {FLOAT}{OWS}min(s?) RET_CONST(new IntervalVal(atof(yytext),Minutes)) @@ -494,6 +484,12 @@ F RET_CONST(new Val(false, TYPE_BOOL)) {FLOAT}{OWS}msec(s?) RET_CONST(new IntervalVal(atof(yytext),Milliseconds)) {FLOAT}{OWS}usec(s?) RET_CONST(new IntervalVal(atof(yytext),Microseconds)) +({D}"."){3}{D} RET_CONST(new AddrVal(yytext)) + +"0x"{HEX}+ RET_CONST(new Val(static_cast(strtoull(yytext, 0, 16)), TYPE_COUNT)) + +{H}("."{H})+ RET_CONST(dns_mgr->LookupHost(yytext)) + \"([^\\\n\"]|{ESCSEQ})*\" { const char* text = yytext; int len = strlen(text) + 1; diff --git a/src/strings.bif b/src/strings.bif index 22e29950ee..dc5e064dc6 100644 --- a/src/strings.bif +++ b/src/strings.bif @@ -311,15 +311,9 @@ static int match_prefix(int s_len, const char* s, int t_len, const char* t) return 1; } -Val* do_split(StringVal* str_val, RE_Matcher* re, TableVal* other_sep, - int incl_sep, int max_num_sep) +Val* do_split(StringVal* str_val, RE_Matcher* re, int incl_sep, int max_num_sep) { TableVal* a = new TableVal(string_array); - ListVal* other_strings = 0; - - if ( other_sep && other_sep->Size() > 0 ) - other_strings = other_sep->ConvertToPureList(); - const u_char* s = str_val->Bytes(); int n = str_val->Len(); const u_char* end_of_s = s + n; @@ -373,9 +367,6 @@ Val* do_split(StringVal* str_val, RE_Matcher* re, TableVal* other_sep, reporter->InternalError("RegMatch in split goes beyond the string"); } - if ( other_strings ) - delete other_strings; - return a; } @@ -483,7 +474,7 @@ Val* do_sub(StringVal* str_val, RE_Matcher* re, StringVal* repl, int do_all) ## function split%(str: string, re: pattern%): string_array %{ - return do_split(str, re, 0, 0, 0); + return do_split(str, re, 0, 0); %} ## Splits a string *once* into a two-element array of strings according to a @@ -503,7 +494,7 @@ function split%(str: string, re: pattern%): string_array ## .. bro:see:: split split_all split_n str_split function split1%(str: string, re: pattern%): string_array %{ - return do_split(str, re, 0, 0, 1); + return do_split(str, re, 0, 1); %} ## Splits a string into an array of strings according to a pattern. This @@ -523,7 +514,7 @@ function split1%(str: string, re: pattern%): string_array ## .. bro:see:: split split1 split_n str_split function split_all%(str: string, re: pattern%): string_array %{ - return do_split(str, re, 0, 1, 0); + return do_split(str, re, 1, 0); %} ## Splits a string a given number of times into an array of strings according @@ -549,16 +540,7 @@ function split_all%(str: string, re: pattern%): string_array function split_n%(str: string, re: pattern, incl_sep: bool, max_num_sep: count%): string_array %{ - return do_split(str, re, 0, incl_sep, max_num_sep); - %} - -## Deprecated. Will be removed. -# Reason: the parameter ``other`` does nothing. -function split_complete%(str: string, - re: pattern, other: string_set, - incl_sep: bool, max_num_sep: count%): string_array - %{ - return do_split(str, re, other->AsTableVal(), incl_sep, max_num_sep); + return do_split(str, re, incl_sep, max_num_sep); %} ## Substitutes a given replacement string for the first occurrence of a pattern diff --git a/src/threading/Manager.cc b/src/threading/Manager.cc index 53c11f2ee9..cfc44596e1 100644 --- a/src/threading/Manager.cc +++ b/src/threading/Manager.cc @@ -80,8 +80,10 @@ double Manager::NextTimestamp(double* network_time) for ( msg_thread_list::iterator i = msg_threads.begin(); i != msg_threads.end(); i++ ) { - if ( (*i)->MightHaveOut() ) - return timer_mgr->Time(); + MsgThread* t = *i; + + if ( (*i)->MightHaveOut() && ! t->Killed() ) + return timer_mgr->Time(); } return -1.0; @@ -95,6 +97,12 @@ void Manager::KillThreads() (*i)->Kill(); } +void Manager::KillThread(BasicThread* thread) + { + DBG_LOG(DBG_THREADING, "Killing thread %s ...", thread->Name()); + thread->Kill(); + } + void Manager::Process() { bool do_beat = false; @@ -114,7 +122,7 @@ void Manager::Process() if ( do_beat ) t->Heartbeat(); - while ( t->HasOut() ) + while ( t->HasOut() && ! t->Killed() ) { Message* msg = t->RetrieveOut(); diff --git a/src/threading/Manager.h b/src/threading/Manager.h index be81c69ba0..b46a06a46e 100644 --- a/src/threading/Manager.h +++ b/src/threading/Manager.h @@ -74,6 +74,16 @@ public: */ void ForceProcessing() { Process(); } + /** + * Signals a specific threads to terminate immediately. + */ + void KillThread(BasicThread* thread); + + /** + * Signals all threads to terminate immediately. + */ + void KillThreads(); + protected: friend class BasicThread; friend class MsgThread; @@ -106,13 +116,6 @@ protected: */ virtual double NextTimestamp(double* network_time); - /** - * Kills all thread immediately. Note that this may cause race conditions - * if a child thread currently holds a lock that might block somebody - * else. - */ - virtual void KillThreads(); - /** * Part of the IOSource interface. */ diff --git a/src/threading/MsgThread.cc b/src/threading/MsgThread.cc index 48c7253885..6c63c5a287 100644 --- a/src/threading/MsgThread.cc +++ b/src/threading/MsgThread.cc @@ -70,6 +70,16 @@ private: Type type; }; +// A message from the the child to the main process, requesting suicide. +class KillMeMessage : public OutputMessage +{ +public: + KillMeMessage(MsgThread* thread) + : OutputMessage("ReporterMessage", thread) {} + + virtual bool Process() { thread_mgr->KillThread(Object()); return true; } +}; + #ifdef DEBUG // A debug message from the child to be passed on to the DebugLogger. class DebugMessage : public OutputMessage @@ -144,6 +154,7 @@ MsgThread::MsgThread() : BasicThread(), queue_in(this, 0), queue_out(0, this) { cnt_sent_in = cnt_sent_out = 0; finished = false; + failed = false; thread_mgr->AddMsgThread(this); } @@ -346,16 +357,21 @@ void MsgThread::Run() if ( ! result ) { - string s = Fmt("%s failed, terminating thread (MsgThread)", Name()); - Error(s.c_str()); - break; + Error("terminating thread"); + + // This will eventually kill this thread, but only + // after all other outgoing messages (in particular + // error messages have been processed by then main + // thread). + SendOut(new KillMeMessage(this)); + failed = true; } } // In case we haven't send the finish method yet, do it now. Reading // global network_time here should be fine, it isn't changing // anymore. - if ( ! finished ) + if ( ! finished && ! Killed() ) { OnFinish(network_time); Finished(); diff --git a/src/threading/MsgThread.h b/src/threading/MsgThread.h index da505de6be..e3e7c8500f 100644 --- a/src/threading/MsgThread.h +++ b/src/threading/MsgThread.h @@ -201,6 +201,12 @@ protected: */ void HeartbeatInChild(); + /** Returns true if a child command has reported a failure. In that case, we'll + * be in the process of killing this thread and no further activity + * should carried out. To be called only from this child thread. + */ + bool Failed() const { return failed; } + /** * Regulatly triggered for execution in the child thread. * @@ -294,6 +300,7 @@ private: uint64_t cnt_sent_out; // Counts message sent by child. bool finished; // Set to true by Finished message. + bool failed; // Set to true when a command failed. }; /** diff --git a/src/util.cc b/src/util.cc index be560928d6..3b6fcac76f 100644 --- a/src/util.cc +++ b/src/util.cc @@ -113,6 +113,9 @@ std::string get_escaped_string(const std::string& str, bool escape_all) char* copy_string(const char* s) { + if ( ! s ) + return 0; + char* c = new char[strlen(s)+1]; strcpy(c, s); return c; @@ -722,7 +725,7 @@ void init_random_seed(uint32 seed, const char* read_file, const char* write_file { int amt = read(fd, buf + pos, sizeof(uint32) * (bufsiz - pos)); - close(fd); + safe_close(fd); if ( amt > 0 ) pos += amt / sizeof(uint32); @@ -1204,7 +1207,7 @@ void _set_processing_status(const char* status) len -= n; } - close(fd); + safe_close(fd); errno = old_errno; } @@ -1353,9 +1356,40 @@ bool safe_write(int fd, const char* data, int len) return true; } +void safe_close(int fd) + { + /* + * Failure cases of close(2) are ... + * EBADF: Indicative of programming logic error that needs to be fixed, we + * should always be attempting to close a valid file descriptor. + * EINTR: Ignore signal interruptions, most implementations will actually + * reclaim the open descriptor and POSIX standard doesn't leave many + * options by declaring the state of the descriptor as "unspecified". + * Attempting to inspect actual state or re-attempt close() is not + * thread safe. + * EIO: Again the state of descriptor is "unspecified", but don't recover + * from an I/O error, safe_write() won't either. + * + * Note that we don't use the reporter here to allow use from different threads. + */ + if ( close(fd) < 0 && errno != EINTR ) + { + char buf[128]; + strerror_r(errno, buf, sizeof(buf)); + fprintf(stderr, "safe_close error %d: %s\n", errno, buf); + abort(); + } + } + void out_of_memory(const char* where) { - reporter->FatalError("out of memory in %s.\n", where); + fprintf(stderr, "out of memory in %s.\n", where); + + if ( reporter ) + // Guess that might fail here if memory is really tight ... + reporter->FatalError("out of memory in %s.\n", where); + + abort(); } void get_memory_usage(unsigned int* total, unsigned int* malloced) diff --git a/src/util.h b/src/util.h index 5d1bdf188a..e69167abce 100644 --- a/src/util.h +++ b/src/util.h @@ -297,6 +297,9 @@ inline size_t pad_size(size_t size) // thread-safe as long as no two threads write to the same descriptor. extern bool safe_write(int fd, const char* data, int len); +// Wraps close(2) to emit error messages and abort on unrecoverable errors. +extern void safe_close(int fd); + extern void out_of_memory(const char* where); inline void* safe_realloc(void* ptr, size_t size) diff --git a/testing/btest/Baseline/analyzers.conn-size-cc/conn.log b/testing/btest/Baseline/analyzers.conn-size-cc/conn.log deleted file mode 100644 index 2f703cbcd6..0000000000 --- a/testing/btest/Baseline/analyzers.conn-size-cc/conn.log +++ /dev/null @@ -1,5 +0,0 @@ -1128727430.350788 ? 141.42.64.125 125.190.109.199 other 56729 12345 tcp ? ? S0 X 1 60 0 0 cc=1 -1144876538.705610 5.921003 169.229.147.203 239.255.255.253 other 49370 427 udp 147 ? S0 X 3 231 0 0 -1144876599.397603 0.815763 192.150.186.169 194.64.249.244 http 53063 80 tcp 377 445 SF X 6 677 5 713 -1144876709.032670 9.000191 169.229.147.43 239.255.255.253 other 49370 427 udp 196 ? S0 X 4 308 0 0 -1144876697.068273 0.000650 192.150.186.169 192.150.186.15 icmp-unreach 3 3 icmp 56 ? OTH X 2 112 0 0 diff --git a/testing/btest/Baseline/analyzers.conn-size/conn.log b/testing/btest/Baseline/analyzers.conn-size/conn.log deleted file mode 100644 index 8129bc37f8..0000000000 --- a/testing/btest/Baseline/analyzers.conn-size/conn.log +++ /dev/null @@ -1,5 +0,0 @@ -1128727430.350788 ? 141.42.64.125 125.190.109.199 other 56729 12345 tcp ? ? S0 X 1 60 0 0 -1144876538.705610 5.921003 169.229.147.203 239.255.255.253 other 49370 427 udp 147 ? S0 X 3 231 0 0 -1144876599.397603 0.815763 192.150.186.169 194.64.249.244 http 53063 80 tcp 377 445 SF X 6 697 5 713 -1144876709.032670 9.000191 169.229.147.43 239.255.255.253 other 49370 427 udp 196 ? S0 X 4 308 0 0 -1144876697.068273 0.000650 192.150.186.169 192.150.186.15 icmp-unreach 3 3 icmp 56 ? OTH X 2 112 0 0 diff --git a/testing/btest/Baseline/bifs.analyzer_name/out b/testing/btest/Baseline/bifs.analyzer_name/out new file mode 100644 index 0000000000..84613e9dd1 --- /dev/null +++ b/testing/btest/Baseline/bifs.analyzer_name/out @@ -0,0 +1 @@ +PIA_TCP diff --git a/testing/btest/Baseline/bifs.capture_state_updates/out b/testing/btest/Baseline/bifs.capture_state_updates/out new file mode 100644 index 0000000000..62a6e3c9df --- /dev/null +++ b/testing/btest/Baseline/bifs.capture_state_updates/out @@ -0,0 +1 @@ +T diff --git a/testing/btest/Baseline/bifs.entropy_test/out b/testing/btest/Baseline/bifs.entropy_test/out new file mode 100644 index 0000000000..08a09de4e4 --- /dev/null +++ b/testing/btest/Baseline/bifs.entropy_test/out @@ -0,0 +1,2 @@ +[entropy=4.715374, chi_square=591.981818, mean=75.472727, monte_carlo_pi=4.0, serial_correlation=-0.11027] +[entropy=2.083189, chi_square=3906.018182, mean=69.054545, monte_carlo_pi=4.0, serial_correlation=0.849402] diff --git a/testing/btest/Baseline/bifs.global_sizes/out b/testing/btest/Baseline/bifs.global_sizes/out new file mode 100644 index 0000000000..76c40b297a --- /dev/null +++ b/testing/btest/Baseline/bifs.global_sizes/out @@ -0,0 +1 @@ +found bro_init diff --git a/testing/btest/Baseline/bifs.identify_data/out b/testing/btest/Baseline/bifs.identify_data/out new file mode 100644 index 0000000000..1cadefbf6e --- /dev/null +++ b/testing/btest/Baseline/bifs.identify_data/out @@ -0,0 +1,4 @@ +ASCII text, with no line terminators +text/plain; charset=us-ascii +PNG image +image/png; charset=binary diff --git a/testing/btest/Baseline/bifs.is_local_interface/out b/testing/btest/Baseline/bifs.is_local_interface/out new file mode 100644 index 0000000000..328bff6687 --- /dev/null +++ b/testing/btest/Baseline/bifs.is_local_interface/out @@ -0,0 +1,4 @@ +T +F +F +T diff --git a/testing/btest/Baseline/bifs.reading_traces/out1 b/testing/btest/Baseline/bifs.reading_traces/out1 new file mode 100644 index 0000000000..cf84443e49 --- /dev/null +++ b/testing/btest/Baseline/bifs.reading_traces/out1 @@ -0,0 +1 @@ +F diff --git a/testing/btest/Baseline/bifs.reading_traces/out2 b/testing/btest/Baseline/bifs.reading_traces/out2 new file mode 100644 index 0000000000..62a6e3c9df --- /dev/null +++ b/testing/btest/Baseline/bifs.reading_traces/out2 @@ -0,0 +1 @@ +T diff --git a/testing/btest/Baseline/bifs.strftime/out b/testing/btest/Baseline/bifs.strftime/out new file mode 100644 index 0000000000..b32393b332 --- /dev/null +++ b/testing/btest/Baseline/bifs.strftime/out @@ -0,0 +1,4 @@ +1970-01-01 00:00:00 +000000 19700101 +1973-11-29 21:33:09 +213309 19731129 diff --git a/testing/btest/Baseline/core.checksums/bad.out b/testing/btest/Baseline/core.checksums/bad.out index de4538e32b..94b141c9e1 100644 --- a/testing/btest/Baseline/core.checksums/bad.out +++ b/testing/btest/Baseline/core.checksums/bad.out @@ -3,101 +3,101 @@ #empty_field (empty) #unset_field - #path weird -#start 2012-03-26-18-03-01 +#open 2012-03-26-18-03-01 #fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p name addl notice peer #types time string addr port addr port string string bool string 1332784981.078396 - - - - - bad_IP_checksum - F bro -#end 2012-03-26-18-03-01 +#close 2012-03-26-18-03-01 #separator \x09 #set_separator , #empty_field (empty) #unset_field - #path weird -#start 2012-03-26-18-01-25 +#open 2012-03-26-18-01-25 #fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p name addl notice peer #types time string addr port addr port string string bool string 1332784885.686428 UWkUyAuUGXf 127.0.0.1 30000 127.0.0.1 80 bad_TCP_checksum - F bro -#end 2012-03-26-18-01-25 +#close 2012-03-26-18-01-25 #separator \x09 #set_separator , #empty_field (empty) #unset_field - #path weird -#start 2012-03-26-18-02-13 +#open 2012-03-26-18-02-13 #fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p name addl notice peer #types time string addr port addr port string string bool string 1332784933.501023 UWkUyAuUGXf 127.0.0.1 30000 127.0.0.1 13000 bad_UDP_checksum - F bro -#end 2012-03-26-18-02-13 +#close 2012-03-26-18-02-13 #separator \x09 #set_separator , #empty_field (empty) #unset_field - #path weird -#start 2012-04-10-16-29-23 +#open 2012-04-10-16-29-23 #fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p name addl notice peer #types time string addr port addr port string string bool string 1334075363.536871 UWkUyAuUGXf 192.168.1.100 8 192.168.1.101 0 bad_ICMP_checksum - F bro -#end 2012-04-10-16-29-23 +#close 2012-04-10-16-29-23 #separator \x09 #set_separator , #empty_field (empty) #unset_field - #path weird -#start 2012-03-26-18-06-50 +#open 2012-03-26-18-06-50 #fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p name addl notice peer #types time string addr port addr port string string bool string 1332785210.013051 - - - - - routing0_hdr - F bro 1332785210.013051 UWkUyAuUGXf 2001:4f8:4:7:2e0:81ff:fe52:ffff 30000 2001:78:1:32::2 80 bad_TCP_checksum - F bro -#end 2012-03-26-18-06-50 +#close 2012-03-26-18-06-50 #separator \x09 #set_separator , #empty_field (empty) #unset_field - #path weird -#start 2012-03-26-17-23-00 +#open 2012-03-26-17-23-00 #fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p name addl notice peer #types time string addr port addr port string string bool string 1332782580.798420 - - - - - routing0_hdr - F bro 1332782580.798420 UWkUyAuUGXf 2001:4f8:4:7:2e0:81ff:fe52:ffff 30000 2001:78:1:32::2 13000 bad_UDP_checksum - F bro -#end 2012-03-26-17-23-00 +#close 2012-03-26-17-23-00 #separator \x09 #set_separator , #empty_field (empty) #unset_field - #path weird -#start 2012-04-10-16-25-11 +#open 2012-04-10-16-25-11 #fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p name addl notice peer #types time string addr port addr port string string bool string 1334075111.800086 - - - - - routing0_hdr - F bro 1334075111.800086 UWkUyAuUGXf 2001:4f8:4:7:2e0:81ff:fe52:ffff 128 2001:78:1:32::1 129 bad_ICMP_checksum - F bro -#end 2012-04-10-16-25-11 +#close 2012-04-10-16-25-11 #separator \x09 #set_separator , #empty_field (empty) #unset_field - #path weird -#start 2012-03-26-18-07-30 +#open 2012-03-26-18-07-30 #fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p name addl notice peer #types time string addr port addr port string string bool string 1332785250.469132 UWkUyAuUGXf 2001:4f8:4:7:2e0:81ff:fe52:ffff 30000 2001:4f8:4:7:2e0:81ff:fe52:9a6b 80 bad_TCP_checksum - F bro -#end 2012-03-26-18-07-30 +#close 2012-03-26-18-07-30 #separator \x09 #set_separator , #empty_field (empty) #unset_field - #path weird -#start 2012-03-26-17-02-22 +#open 2012-03-26-17-02-22 #fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p name addl notice peer #types time string addr port addr port string string bool string 1332781342.923813 UWkUyAuUGXf 2001:4f8:4:7:2e0:81ff:fe52:ffff 30000 2001:4f8:4:7:2e0:81ff:fe52:9a6b 13000 bad_UDP_checksum - F bro -#end 2012-03-26-17-02-22 +#close 2012-03-26-17-02-22 #separator \x09 #set_separator , #empty_field (empty) #unset_field - #path weird -#start 2012-04-10-16-22-19 +#open 2012-04-10-16-22-19 #fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p name addl notice peer #types time string addr port addr port string string bool string 1334074939.467194 UWkUyAuUGXf 2001:4f8:4:7:2e0:81ff:fe52:ffff 128 2001:4f8:4:7:2e0:81ff:fe52:9a6b 129 bad_ICMP_checksum - F bro -#end 2012-04-10-16-22-19 +#close 2012-04-10-16-22-19 diff --git a/testing/btest/Baseline/core.checksums/good.out b/testing/btest/Baseline/core.checksums/good.out index ed6c071ffc..a47931a15c 100644 --- a/testing/btest/Baseline/core.checksums/good.out +++ b/testing/btest/Baseline/core.checksums/good.out @@ -3,68 +3,68 @@ #empty_field (empty) #unset_field - #path weird -#start 2012-04-10-16-22-19 +#open 2012-04-10-16-22-19 #fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p name addl notice peer #types time string addr port addr port string string bool string 1334074939.467194 UWkUyAuUGXf 2001:4f8:4:7:2e0:81ff:fe52:ffff 128 2001:4f8:4:7:2e0:81ff:fe52:9a6b 129 bad_ICMP_checksum - F bro -#end 2012-04-10-16-22-19 +#close 2012-04-10-16-22-19 #separator \x09 #set_separator , #empty_field (empty) #unset_field - #path weird -#start 2012-03-26-18-05-25 +#open 2012-03-26-18-05-25 #fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p name addl notice peer #types time string addr port addr port string string bool string 1332785125.596793 - - - - - routing0_hdr - F bro -#end 2012-03-26-18-05-25 +#close 2012-03-26-18-05-25 #separator \x09 #set_separator , #empty_field (empty) #unset_field - #path weird -#start 2012-03-26-17-21-48 +#open 2012-03-26-17-21-48 #fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p name addl notice peer #types time string addr port addr port string string bool string 1332782508.592037 - - - - - routing0_hdr - F bro -#end 2012-03-26-17-21-48 +#close 2012-03-26-17-21-48 #separator \x09 #set_separator , #empty_field (empty) #unset_field - #path weird -#start 2012-04-10-16-23-47 +#open 2012-04-10-16-23-47 #fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p name addl notice peer #types time string addr port addr port string string bool string 1334075027.053380 - - - - - routing0_hdr - F bro -#end 2012-04-10-16-23-47 +#close 2012-04-10-16-23-47 #separator \x09 #set_separator , #empty_field (empty) #unset_field - #path weird -#start 2012-04-10-16-23-47 +#open 2012-04-10-16-23-47 #fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p name addl notice peer #types time string addr port addr port string string bool string 1334075027.053380 - - - - - routing0_hdr - F bro -#end 2012-04-10-16-23-47 +#close 2012-04-10-16-23-47 #separator \x09 #set_separator , #empty_field (empty) #unset_field - #path weird -#start 2012-04-10-16-23-47 +#open 2012-04-10-16-23-47 #fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p name addl notice peer #types time string addr port addr port string string bool string 1334075027.053380 - - - - - routing0_hdr - F bro -#end 2012-04-10-16-23-47 +#close 2012-04-10-16-23-47 #separator \x09 #set_separator , #empty_field (empty) #unset_field - #path weird -#start 2012-04-10-16-23-47 +#open 2012-04-10-16-23-47 #fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p name addl notice peer #types time string addr port addr port string string bool string 1334075027.053380 - - - - - routing0_hdr - F bro -#end 2012-04-10-16-23-47 +#close 2012-04-10-16-23-47 diff --git a/testing/btest/Baseline/core.disable-mobile-ipv6/weird.log b/testing/btest/Baseline/core.disable-mobile-ipv6/weird.log index d29456f75f..9da1a8d3ba 100644 --- a/testing/btest/Baseline/core.disable-mobile-ipv6/weird.log +++ b/testing/btest/Baseline/core.disable-mobile-ipv6/weird.log @@ -3,8 +3,8 @@ #empty_field (empty) #unset_field - #path weird -#start 2012-04-05-21-56-51 +#open 2012-04-05-21-56-51 #fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p name addl notice peer #types time string addr port addr port string string bool string 1333663011.602839 - - - - - unknown_protocol_135 - F bro -#end 2012-04-05-21-56-51 +#close 2012-04-05-21-56-51 diff --git a/testing/btest/Baseline/core.expr-exception/reporter.log b/testing/btest/Baseline/core.expr-exception/reporter.log index f9e33d9718..d6e07b42b3 100644 --- a/testing/btest/Baseline/core.expr-exception/reporter.log +++ b/testing/btest/Baseline/core.expr-exception/reporter.log @@ -3,7 +3,7 @@ #empty_field (empty) #unset_field - #path reporter -#start 2011-03-18-19-06-08 +#open 2011-03-18-19-06-08 #fields ts level message location #types time enum string string 1300475168.783842 Reporter::ERROR field value missing [c$ftp] /da/home/robin/bro/master/testing/btest/.tmp/core.expr-exception/expr-exception.bro, line 10 @@ -15,4 +15,4 @@ 1300475168.954761 Reporter::ERROR field value missing [c$ftp] /da/home/robin/bro/master/testing/btest/.tmp/core.expr-exception/expr-exception.bro, line 10 1300475168.962628 Reporter::ERROR field value missing [c$ftp] /da/home/robin/bro/master/testing/btest/.tmp/core.expr-exception/expr-exception.bro, line 10 1300475169.780331 Reporter::ERROR field value missing [c$ftp] /da/home/robin/bro/master/testing/btest/.tmp/core.expr-exception/expr-exception.bro, line 10 -#end 2011-03-18-19-06-13 +#close 2011-03-18-19-06-13 diff --git a/testing/btest/Baseline/core.ipv6-frag/dns.log b/testing/btest/Baseline/core.ipv6-frag/dns.log index 2003d1f253..d763fc4fee 100644 --- a/testing/btest/Baseline/core.ipv6-frag/dns.log +++ b/testing/btest/Baseline/core.ipv6-frag/dns.log @@ -3,9 +3,9 @@ #empty_field (empty) #unset_field - #path dns -#start 2012-03-07-01-37-58 +#open 2012-03-07-01-37-58 #fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p proto trans_id query qclass qclass_name qtype qtype_name rcode rcode_name AA TC RD RA Z answers TTLs #types time string addr port addr port enum count string count string count string count string bool bool bool bool count vector[string] vector[interval] 1331084278.438444 UWkUyAuUGXf 2001:470:1f11:81f:d138:5f55:6d4:1fe2 51850 2607:f740:b::f93 53 udp 3903 txtpadding_323.n1.netalyzr.icsi.berkeley.edu 1 C_INTERNET 16 TXT 0 NOERROR T F T F 0 This TXT record should be ignored 1.000000 1331084293.592245 arKYeMETxOg 2001:470:1f11:81f:d138:5f55:6d4:1fe2 51851 2607:f740:b::f93 53 udp 40849 txtpadding_3230.n1.netalyzr.icsi.berkeley.edu 1 C_INTERNET 16 TXT 0 NOERROR T F T F 0 This TXT record should be ignored 1.000000 -#end 2012-03-07-01-38-18 +#close 2012-03-07-01-38-18 diff --git a/testing/btest/Baseline/core.ipv6_zero_len_ah/output b/testing/btest/Baseline/core.ipv6_zero_len_ah/output new file mode 100644 index 0000000000..d8db6a4c48 --- /dev/null +++ b/testing/btest/Baseline/core.ipv6_zero_len_ah/output @@ -0,0 +1,2 @@ +[orig_h=2000:1300::1, orig_p=128/icmp, resp_h=2000:1300::2, resp_p=129/icmp] +[ip=, ip6=[class=0, flow=0, len=166, nxt=51, hlim=255, src=2000:1300::1, dst=2000:1300::2, exts=[[id=51, hopopts=, dstopts=, routing=, fragment=, ah=[nxt=58, len=0, rsv=0, spi=0, seq=, data=], esp=, mobility=]]], tcp=, udp=, icmp=] diff --git a/testing/btest/Baseline/core.leaks.basic-cluster/manager-1.metrics.log b/testing/btest/Baseline/core.leaks.basic-cluster/manager-1.metrics.log index 42fcd6a526..cb1bd5af01 100644 --- a/testing/btest/Baseline/core.leaks.basic-cluster/manager-1.metrics.log +++ b/testing/btest/Baseline/core.leaks.basic-cluster/manager-1.metrics.log @@ -3,8 +3,10 @@ #empty_field (empty) #unset_field - #path metrics +#open 2012-07-20-01-50-41 #fields ts metric_id filter_name index.host index.str index.network value #types time enum string addr string subnet count -1331256494.591966 TEST_METRIC foo-bar 6.5.4.3 - - 4 -1331256494.591966 TEST_METRIC foo-bar 7.2.1.5 - - 2 -1331256494.591966 TEST_METRIC foo-bar 1.2.3.4 - - 6 +1342749041.601712 TEST_METRIC foo-bar 6.5.4.3 - - 4 +1342749041.601712 TEST_METRIC foo-bar 7.2.1.5 - - 2 +1342749041.601712 TEST_METRIC foo-bar 1.2.3.4 - - 6 +#close 2012-07-20-01-50-49 diff --git a/testing/btest/Baseline/core.leaks.remote/sender.test.failure.log b/testing/btest/Baseline/core.leaks.remote/sender.test.failure.log index 5a26f322f4..71e1d18c73 100644 --- a/testing/btest/Baseline/core.leaks.remote/sender.test.failure.log +++ b/testing/btest/Baseline/core.leaks.remote/sender.test.failure.log @@ -3,8 +3,10 @@ #empty_field (empty) #unset_field - #path test.failure +#open 2012-07-20-01-50-18 #fields t id.orig_h id.orig_p id.resp_h id.resp_p status country #types time addr port addr port string string -1331256472.375609 1.2.3.4 1234 2.3.4.5 80 failure US -1331256472.375609 1.2.3.4 1234 2.3.4.5 80 failure UK -1331256472.375609 1.2.3.4 1234 2.3.4.5 80 failure MX +1342749018.970682 1.2.3.4 1234 2.3.4.5 80 failure US +1342749018.970682 1.2.3.4 1234 2.3.4.5 80 failure UK +1342749018.970682 1.2.3.4 1234 2.3.4.5 80 failure MX +#close 2012-07-20-01-50-18 diff --git a/testing/btest/Baseline/core.leaks.remote/sender.test.log b/testing/btest/Baseline/core.leaks.remote/sender.test.log index 9d2ba26f48..bc3dac5a1a 100644 --- a/testing/btest/Baseline/core.leaks.remote/sender.test.log +++ b/testing/btest/Baseline/core.leaks.remote/sender.test.log @@ -3,10 +3,12 @@ #empty_field (empty) #unset_field - #path test +#open 2012-07-20-01-50-18 #fields t id.orig_h id.orig_p id.resp_h id.resp_p status country #types time addr port addr port string string -1331256472.375609 1.2.3.4 1234 2.3.4.5 80 success unknown -1331256472.375609 1.2.3.4 1234 2.3.4.5 80 failure US -1331256472.375609 1.2.3.4 1234 2.3.4.5 80 failure UK -1331256472.375609 1.2.3.4 1234 2.3.4.5 80 success BR -1331256472.375609 1.2.3.4 1234 2.3.4.5 80 failure MX +1342749018.970682 1.2.3.4 1234 2.3.4.5 80 success unknown +1342749018.970682 1.2.3.4 1234 2.3.4.5 80 failure US +1342749018.970682 1.2.3.4 1234 2.3.4.5 80 failure UK +1342749018.970682 1.2.3.4 1234 2.3.4.5 80 success BR +1342749018.970682 1.2.3.4 1234 2.3.4.5 80 failure MX +#close 2012-07-20-01-50-18 diff --git a/testing/btest/Baseline/core.leaks.remote/sender.test.success.log b/testing/btest/Baseline/core.leaks.remote/sender.test.success.log index 1b2ed452a0..f0b26454b4 100644 --- a/testing/btest/Baseline/core.leaks.remote/sender.test.success.log +++ b/testing/btest/Baseline/core.leaks.remote/sender.test.success.log @@ -3,7 +3,9 @@ #empty_field (empty) #unset_field - #path test.success +#open 2012-07-20-01-50-18 #fields t id.orig_h id.orig_p id.resp_h id.resp_p status country #types time addr port addr port string string -1331256472.375609 1.2.3.4 1234 2.3.4.5 80 success unknown -1331256472.375609 1.2.3.4 1234 2.3.4.5 80 success BR +1342749018.970682 1.2.3.4 1234 2.3.4.5 80 success unknown +1342749018.970682 1.2.3.4 1234 2.3.4.5 80 success BR +#close 2012-07-20-01-50-18 diff --git a/testing/btest/Baseline/core.print-bpf-filters/conn.log b/testing/btest/Baseline/core.print-bpf-filters/conn.log index 4033b64e2a..0fd86b8dc4 100644 --- a/testing/btest/Baseline/core.print-bpf-filters/conn.log +++ b/testing/btest/Baseline/core.print-bpf-filters/conn.log @@ -3,8 +3,8 @@ #empty_field (empty) #unset_field - #path conn -#start 2005-10-07-23-23-57 +#open 2005-10-07-23-23-57 #fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p proto service duration orig_bytes resp_bytes conn_state local_orig missed_bytes history orig_pkts orig_ip_bytes resp_pkts resp_ip_bytes tunnel_parents #types time string addr port addr port enum string interval count count string bool count string count count count count table[string] 1128727435.450898 UWkUyAuUGXf 141.42.64.125 56730 125.190.109.199 80 tcp http 1.733303 98 9417 SF - 0 ShADdFaf 12 730 10 9945 (empty) -#end 2005-10-07-23-23-57 +#close 2005-10-07-23-23-57 diff --git a/testing/btest/Baseline/core.print-bpf-filters/output b/testing/btest/Baseline/core.print-bpf-filters/output index e4bc04192a..c55952ffed 100644 --- a/testing/btest/Baseline/core.print-bpf-filters/output +++ b/testing/btest/Baseline/core.print-bpf-filters/output @@ -3,38 +3,38 @@ #empty_field (empty) #unset_field - #path packet_filter -#start 1970-01-01-00-00-00 +#open 2012-07-27-19-14-29 #fields ts node filter init success #types time string string bool bool -1342748953.570646 - ip or not ip T T -#end +1343416469.508262 - ip or not ip T T +#close 2012-07-27-19-14-29 #separator \x09 #set_separator , #empty_field (empty) #unset_field - #path packet_filter -#start 1970-01-01-00-00-00 +#open 2012-07-27-19-14-29 #fields ts node filter init success #types time string string bool bool -1342748953.898675 - (((((((((((((((((((((((((port 53) or (tcp port 989)) or (tcp port 443)) or (port 6669)) or (udp and port 5353)) or (port 6668)) or (tcp port 1080)) or (udp and port 5355)) or (tcp port 22)) or (tcp port 995)) or (port 21)) or (tcp port 25 or tcp port 587)) or (port 6667)) or (tcp port 614)) or (tcp port 990)) or (udp port 137)) or (tcp port 993)) or (tcp port 5223)) or (port 514)) or (tcp port 585)) or (tcp port 992)) or (tcp port 563)) or (tcp port 994)) or (tcp port 636)) or (tcp and port (80 or 81 or 631 or 1080 or 3138 or 8000 or 8080 or 8888))) or (port 6666) T T -#end +1343416469.888870 - (((((((((((((((((((((((((port 53) or (tcp port 989)) or (tcp port 443)) or (port 6669)) or (udp and port 5353)) or (port 6668)) or (tcp port 1080)) or (udp and port 5355)) or (tcp port 22)) or (tcp port 995)) or (port 21)) or (tcp port 25 or tcp port 587)) or (port 6667)) or (tcp port 614)) or (tcp port 990)) or (udp port 137)) or (tcp port 993)) or (tcp port 5223)) or (port 514)) or (tcp port 585)) or (tcp port 992)) or (tcp port 563)) or (tcp port 994)) or (tcp port 636)) or (tcp and port (80 or 81 or 631 or 1080 or 3138 or 8000 or 8080 or 8888))) or (port 6666) T T +#close 2012-07-27-19-14-29 #separator \x09 #set_separator , #empty_field (empty) #unset_field - #path packet_filter -#start 1970-01-01-00-00-00 +#open 2012-07-27-19-14-30 #fields ts node filter init success #types time string string bool bool -1342748954.278211 - port 42 T T -#end +1343416470.252918 - port 42 T T +#close 2012-07-27-19-14-30 #separator \x09 #set_separator , #empty_field (empty) #unset_field - #path packet_filter -#start 1970-01-01-00-00-00 +#open 2012-07-27-19-14-30 #fields ts node filter init success #types time string string bool bool -1342748954.883780 - port 56730 T T -#end 2005-10-07-23-23-57 +1343416470.614962 - port 56730 T T +#close 2012-07-27-19-14-30 diff --git a/testing/btest/Baseline/core.reporter-error-in-handler/output b/testing/btest/Baseline/core.reporter-error-in-handler/output index 83b310ab61..b20b1b2292 100644 --- a/testing/btest/Baseline/core.reporter-error-in-handler/output +++ b/testing/btest/Baseline/core.reporter-error-in-handler/output @@ -1,2 +1,3 @@ -error in /da/home/robin/bro/master/testing/btest/.tmp/core.reporter-error-in-handler/reporter-error-in-handler.bro, line 22: no such index (a[2]) +error in /home/jsiwek/bro/testing/btest/.tmp/core.reporter-error-in-handler/reporter-error-in-handler.bro, line 22: no such index (a[2]) +ERROR: no such index (a[1]) (/home/jsiwek/bro/testing/btest/.tmp/core.reporter-error-in-handler/reporter-error-in-handler.bro, line 28) 1st error printed on script level diff --git a/testing/btest/Baseline/core.reporter-runtime-error/output b/testing/btest/Baseline/core.reporter-runtime-error/output index 59bcc3ac9b..5a03f5feb2 100644 --- a/testing/btest/Baseline/core.reporter-runtime-error/output +++ b/testing/btest/Baseline/core.reporter-runtime-error/output @@ -1 +1,2 @@ -error in /da/home/robin/bro/master/testing/btest/.tmp/core.reporter-runtime-error/reporter-runtime-error.bro, line 12: no such index (a[1]) +error in /home/jsiwek/bro/testing/btest/.tmp/core.reporter-runtime-error/reporter-runtime-error.bro, line 12: no such index (a[1]) +ERROR: no such index (a[2]) (/home/jsiwek/bro/testing/btest/.tmp/core.reporter-runtime-error/reporter-runtime-error.bro, line 9) diff --git a/testing/btest/Baseline/core.reporter/logger-test.log b/testing/btest/Baseline/core.reporter/logger-test.log index 6f7ba1d8c7..5afd904b63 100644 --- a/testing/btest/Baseline/core.reporter/logger-test.log +++ b/testing/btest/Baseline/core.reporter/logger-test.log @@ -1,6 +1,6 @@ -reporter_info|init test-info|/da/home/robin/bro/master/testing/btest/.tmp/core.reporter/reporter.bro, line 8|0.000000 -reporter_warning|init test-warning|/da/home/robin/bro/master/testing/btest/.tmp/core.reporter/reporter.bro, line 9|0.000000 -reporter_error|init test-error|/da/home/robin/bro/master/testing/btest/.tmp/core.reporter/reporter.bro, line 10|0.000000 -reporter_info|done test-info|/da/home/robin/bro/master/testing/btest/.tmp/core.reporter/reporter.bro, line 15|0.000000 -reporter_warning|done test-warning|/da/home/robin/bro/master/testing/btest/.tmp/core.reporter/reporter.bro, line 16|0.000000 -reporter_error|done test-error|/da/home/robin/bro/master/testing/btest/.tmp/core.reporter/reporter.bro, line 17|0.000000 +reporter_info|init test-info|/home/jsiwek/bro/testing/btest/.tmp/core.reporter/reporter.bro, line 8|0.000000 +reporter_warning|init test-warning|/home/jsiwek/bro/testing/btest/.tmp/core.reporter/reporter.bro, line 9|0.000000 +reporter_error|init test-error|/home/jsiwek/bro/testing/btest/.tmp/core.reporter/reporter.bro, line 10|0.000000 +reporter_info|done test-info|/home/jsiwek/bro/testing/btest/.tmp/core.reporter/reporter.bro, line 15|0.000000 +reporter_warning|done test-warning|/home/jsiwek/bro/testing/btest/.tmp/core.reporter/reporter.bro, line 16|0.000000 +reporter_error|done test-error|/home/jsiwek/bro/testing/btest/.tmp/core.reporter/reporter.bro, line 17|0.000000 diff --git a/testing/btest/Baseline/core.reporter/output b/testing/btest/Baseline/core.reporter/output index 2735adc931..f2c59259c2 100644 --- a/testing/btest/Baseline/core.reporter/output +++ b/testing/btest/Baseline/core.reporter/output @@ -1,3 +1,7 @@ -/da/home/robin/bro/master/testing/btest/.tmp/core.reporter/reporter.bro, line 52: pre test-info -warning in /da/home/robin/bro/master/testing/btest/.tmp/core.reporter/reporter.bro, line 53: pre test-warning -error in /da/home/robin/bro/master/testing/btest/.tmp/core.reporter/reporter.bro, line 54: pre test-error +/home/jsiwek/bro/testing/btest/.tmp/core.reporter/reporter.bro, line 52: pre test-info +warning in /home/jsiwek/bro/testing/btest/.tmp/core.reporter/reporter.bro, line 53: pre test-warning +error in /home/jsiwek/bro/testing/btest/.tmp/core.reporter/reporter.bro, line 54: pre test-error +WARNING: init test-warning (/home/jsiwek/bro/testing/btest/.tmp/core.reporter/reporter.bro, line 9) +ERROR: init test-error (/home/jsiwek/bro/testing/btest/.tmp/core.reporter/reporter.bro, line 10) +WARNING: done test-warning (/home/jsiwek/bro/testing/btest/.tmp/core.reporter/reporter.bro, line 16) +ERROR: done test-error (/home/jsiwek/bro/testing/btest/.tmp/core.reporter/reporter.bro, line 17) diff --git a/testing/btest/Baseline/core.truncation/output b/testing/btest/Baseline/core.truncation/output index 836f9170d4..9243c2f873 100644 --- a/testing/btest/Baseline/core.truncation/output +++ b/testing/btest/Baseline/core.truncation/output @@ -3,38 +3,38 @@ #empty_field (empty) #unset_field - #path weird -#start 2012-04-11-16-01-35 +#open 2012-04-11-16-01-35 #fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p name addl notice peer #types time string addr port addr port string string bool string 1334160095.895421 - - - - - truncated_IP - F bro -#end 2012-04-11-16-01-35 +#close 2012-04-11-16-01-35 #separator \x09 #set_separator , #empty_field (empty) #unset_field - #path weird -#start 2012-04-11-14-57-21 +#open 2012-04-11-14-57-21 #fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p name addl notice peer #types time string addr port addr port string string bool string 1334156241.519125 - - - - - truncated_IP - F bro -#end 2012-04-11-14-57-21 +#close 2012-04-11-14-57-21 #separator \x09 #set_separator , #empty_field (empty) #unset_field - #path weird -#start 2012-04-10-21-50-48 +#open 2012-04-10-21-50-48 #fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p name addl notice peer #types time string addr port addr port string string bool string 1334094648.590126 - - - - - truncated_IP - F bro -#end 2012-04-10-21-50-48 +#close 2012-04-10-21-50-48 #separator \x09 #set_separator , #empty_field (empty) #unset_field - #path weird -#start 2012-05-29-22-02-34 +#open 2012-05-29-22-02-34 #fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p name addl notice peer #types time string addr port addr port string string bool string 1338328954.078361 - - - - - internally_truncated_header - F bro -#end 2012-05-29-22-02-34 +#close 2012-05-29-22-02-34 diff --git a/testing/btest/Baseline/core.tunnels.ayiya/conn.log b/testing/btest/Baseline/core.tunnels.ayiya/conn.log index 82a3828f0d..7646fa574a 100644 --- a/testing/btest/Baseline/core.tunnels.ayiya/conn.log +++ b/testing/btest/Baseline/core.tunnels.ayiya/conn.log @@ -3,7 +3,7 @@ #empty_field (empty) #unset_field - #path conn -#start 2009-11-08-04-41-57 +#open 2009-11-08-04-41-57 #fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p proto service duration orig_bytes resp_bytes conn_state local_orig missed_bytes history orig_pkts orig_ip_bytes resp_pkts resp_ip_bytes tunnel_parents #types time string addr port addr port enum string interval count count string bool count string count count count count table[string] 1257655301.595604 5OKnoww6xl4 2001:4978:f:4c::2 53382 2001:4860:b002::68 80 tcp http 2.101052 2981 4665 S1 - 0 ShADad 10 3605 11 5329 k6kgXLOoSKl @@ -14,4 +14,4 @@ 1257655296.585188 TEfuqmmG4bh fe80::216:cbff:fe9a:4cb9 131 ff02::1:ff00:2 130 icmp - 0.919988 32 0 OTH - 0 - 2 144 0 0 k6kgXLOoSKl 1257655296.585151 j4u32Pc5bif fe80::216:cbff:fe9a:4cb9 131 ff02::2:f901:d225 130 icmp - 0.719947 32 0 OTH - 0 - 2 144 0 0 k6kgXLOoSKl 1257655296.585034 nQcgTWjvg4c fe80::216:cbff:fe9a:4cb9 131 ff02::1:ff9a:4cb9 130 icmp - 4.922880 32 0 OTH - 0 - 2 144 0 0 k6kgXLOoSKl -#end 2009-11-08-04-41-57 +#close 2009-11-08-04-41-57 diff --git a/testing/btest/Baseline/core.tunnels.ayiya/http.log b/testing/btest/Baseline/core.tunnels.ayiya/http.log index 4fbcd508f4..2a97fd9b69 100644 --- a/testing/btest/Baseline/core.tunnels.ayiya/http.log +++ b/testing/btest/Baseline/core.tunnels.ayiya/http.log @@ -3,10 +3,10 @@ #empty_field (empty) #unset_field - #path http -#start 2009-11-08-04-41-41 +#open 2009-11-08-04-41-41 #fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p trans_depth method host uri referrer user_agent request_body_len response_body_len status_code status_msg info_code info_msg filename tags username password proxied mime_type md5 extraction_file #types time string addr port addr port count string string string string string count count count string count string string table[enum] string string table[string] string string file 1257655301.652206 5OKnoww6xl4 2001:4978:f:4c::2 53382 2001:4860:b002::68 80 1 GET ipv6.google.com / - Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en; rv:1.9.0.15pre) Gecko/2009091516 Camino/2.0b4 (like Firefox/3.0.15pre) 0 10102 200 OK - - - (empty) - - - text/html - - 1257655302.514424 5OKnoww6xl4 2001:4978:f:4c::2 53382 2001:4860:b002::68 80 2 GET ipv6.google.com /csi?v=3&s=webhp&action=&tran=undefined&e=17259,19771,21517,21766,21887,22212&ei=BUz2Su7PMJTglQfz3NzCAw&rt=prt.77,xjs.565,ol.645 http://ipv6.google.com/ Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en; rv:1.9.0.15pre) Gecko/2009091516 Camino/2.0b4 (like Firefox/3.0.15pre) 0 0 204 No Content - - - (empty) - - - - - - 1257655303.603569 5OKnoww6xl4 2001:4978:f:4c::2 53382 2001:4860:b002::68 80 3 GET ipv6.google.com /gen_204?atyp=i&ct=fade&cad=1254&ei=BUz2Su7PMJTglQfz3NzCAw&zx=1257655303600 http://ipv6.google.com/ Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en; rv:1.9.0.15pre) Gecko/2009091516 Camino/2.0b4 (like Firefox/3.0.15pre) 0 0 204 No Content - - - (empty) - - - - - - -#end 2009-11-08-04-41-57 +#close 2009-11-08-04-41-57 diff --git a/testing/btest/Baseline/core.tunnels.ayiya/tunnel.log b/testing/btest/Baseline/core.tunnels.ayiya/tunnel.log index 123ea8a792..60e0a4a108 100644 --- a/testing/btest/Baseline/core.tunnels.ayiya/tunnel.log +++ b/testing/btest/Baseline/core.tunnels.ayiya/tunnel.log @@ -3,11 +3,11 @@ #empty_field (empty) #unset_field - #path tunnel -#start 2009-11-08-04-41-33 +#open 2009-11-08-04-41-33 #fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p tunnel_type action #types time string addr port addr port enum enum 1257655293.629048 UWkUyAuUGXf 192.168.3.101 53796 216.14.98.22 5072 Tunnel::AYIYA Tunnel::DISCOVER 1257655296.585034 k6kgXLOoSKl 192.168.3.101 53859 216.14.98.22 5072 Tunnel::AYIYA Tunnel::DISCOVER 1257655317.464035 k6kgXLOoSKl 192.168.3.101 53859 216.14.98.22 5072 Tunnel::AYIYA Tunnel::CLOSE 1257655317.464035 UWkUyAuUGXf 192.168.3.101 53796 216.14.98.22 5072 Tunnel::AYIYA Tunnel::CLOSE -#end 2009-11-08-04-41-57 +#close 2009-11-08-04-41-57 diff --git a/testing/btest/Baseline/core.tunnels.false-teredo/dpd.log b/testing/btest/Baseline/core.tunnels.false-teredo/dpd.log deleted file mode 100644 index 63a0437445..0000000000 --- a/testing/btest/Baseline/core.tunnels.false-teredo/dpd.log +++ /dev/null @@ -1,15 +0,0 @@ -#separator \x09 -#set_separator , -#empty_field (empty) -#unset_field - -#path dpd -#start 2009-11-18-17-59-51 -#fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p proto analyzer failure_reason -#types time string addr port addr port enum string string -1258567191.486869 UWkUyAuUGXf 192.168.1.105 57696 192.168.1.1 53 udp TEREDO Teredo payload length [c\x1d\x81\x80\x00\x01\x00\x02\x00\x02\x00\x00\x04amch\x0equestionmarket\x03com\x00\x00\x01\x00...] -1258578181.516140 nQcgTWjvg4c 192.168.1.104 64838 192.168.1.1 53 udp TEREDO Teredo payload length [h\xfd\x81\x80\x00\x01\x00\x02\x00\x03\x00\x02\x08football\x02uk\x07reuters\x03com\x00\x00\x01\x00...] -1258579063.784919 j4u32Pc5bif 192.168.1.104 55778 192.168.1.1 53 udp TEREDO Teredo payload length [j\x12\x81\x80\x00\x01\x00\x02\x00\x04\x00\x00\x08fastflip\x0agooglelabs\x03com\x00\x00\x01\x00...] -1258581768.898165 TEfuqmmG4bh 192.168.1.104 50798 192.168.1.1 53 udp TEREDO Teredo payload length [o\xe3\x81\x80\x00\x01\x00\x02\x00\x04\x00\x04\x03www\x0fnashuatelegraph\x03com\x00\x00\x01\x00...] -1258584478.989528 FrJExwHcSal 192.168.1.104 64963 192.168.1.1 53 udp TEREDO Teredo payload length [e\xbd\x81\x80\x00\x01\x00\x08\x00\x06\x00\x06\x08wellness\x05blogs\x04time\x03com\x00\x00\x01\x00...] -1258600683.934672 5OKnoww6xl4 192.168.1.103 59838 192.168.1.1 53 udp TEREDO Teredo payload length [h\xf0\x81\x80\x00\x01\x00\x01\x00\x02\x00\x00\x06update\x0csanasecurity\x03com\x00\x00\x01\x00...] -#end 2009-11-19-03-18-03 diff --git a/testing/btest/Baseline/core.tunnels.false-teredo/weird.log b/testing/btest/Baseline/core.tunnels.false-teredo/weird.log index eb4319c7eb..a84d469660 100644 --- a/testing/btest/Baseline/core.tunnels.false-teredo/weird.log +++ b/testing/btest/Baseline/core.tunnels.false-teredo/weird.log @@ -3,7 +3,7 @@ #empty_field (empty) #unset_field - #path weird -#start 2009-11-18-17-59-51 +#open 2009-11-18-17-59-51 #fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p name addl notice peer #types time string addr port addr port string string bool string 1258567191.405770 - - - - - truncated_header_in_tunnel - F bro @@ -12,4 +12,4 @@ 1258581768.568451 - - - - - truncated_header_in_tunnel - F bro 1258584478.859853 - - - - - truncated_header_in_tunnel - F bro 1258600683.934458 - - - - - truncated_header_in_tunnel - F bro -#end 2009-11-19-03-18-03 +#close 2009-11-19-03-18-03 diff --git a/testing/btest/Baseline/core.tunnels.teredo-known-services/known_services.log b/testing/btest/Baseline/core.tunnels.teredo-known-services/known_services.log new file mode 100644 index 0000000000..705cd0e956 --- /dev/null +++ b/testing/btest/Baseline/core.tunnels.teredo-known-services/known_services.log @@ -0,0 +1,10 @@ +#separator \x09 +#set_separator , +#empty_field (empty) +#unset_field - +#path known_services +#open 2012-10-02-20-10-05 +#fields ts host port_num port_proto service +#types time addr port enum table[string] +1258567191.405770 192.168.1.1 53 udp TEREDO +#close 2012-10-02-20-10-05 diff --git a/testing/btest/Baseline/core.tunnels.teredo/conn.log b/testing/btest/Baseline/core.tunnels.teredo/conn.log index 2342953339..b71e56f073 100644 --- a/testing/btest/Baseline/core.tunnels.teredo/conn.log +++ b/testing/btest/Baseline/core.tunnels.teredo/conn.log @@ -3,7 +3,7 @@ #empty_field (empty) #unset_field - #path conn -#start 2008-05-16-15-50-57 +#open 2008-05-16-15-50-57 #fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p proto service duration orig_bytes resp_bytes conn_state local_orig missed_bytes history orig_pkts orig_ip_bytes resp_pkts resp_ip_bytes tunnel_parents #types time string addr port addr port enum string interval count count string bool count string count count count count table[string] 1210953047.736921 arKYeMETxOg 192.168.2.16 1576 75.126.130.163 80 tcp - 0.000357 0 0 SHR - 0 fA 1 40 1 40 (empty) @@ -22,9 +22,9 @@ 1210953052.202579 nQcgTWjvg4c 192.168.2.16 3797 65.55.158.80 3544 udp teredo 8.928880 129 48 SF - 0 Dd 2 185 1 76 (empty) 1210953060.829233 GSxOnSLghOa 192.168.2.16 3797 83.170.1.38 32900 udp teredo 13.293994 2359 11243 SF - 0 Dd 12 2695 13 11607 (empty) 1210953058.933954 iE6yhOq3SF 0.0.0.0 68 255.255.255.255 67 udp - - - - S0 - 0 D 1 328 0 0 (empty) -1210953052.324629 TEfuqmmG4bh 192.168.2.16 3797 65.55.158.81 3544 udp teredo - - - SHR - 0 d 0 0 1 137 (empty) +1210953052.324629 TEfuqmmG4bh 192.168.2.16 3797 65.55.158.81 3544 udp - - - - SHR - 0 d 0 0 1 137 (empty) 1210953046.591933 UWkUyAuUGXf 192.168.2.16 138 192.168.2.255 138 udp - 28.448321 416 0 S0 - 0 D 2 472 0 0 (empty) 1210953052.324629 FrJExwHcSal fe80::8000:f227:bec8:61af 134 fe80::8000:ffff:ffff:fffd 133 icmp - - - - OTH - 0 - 1 88 0 0 TEfuqmmG4bh 1210953060.829303 qCaWGmzFtM5 2001:0:4137:9e50:8000:f12a:b9c8:2815 128 2001:4860:0:2001::68 129 icmp - 0.463615 4 4 OTH - 0 - 1 52 1 52 GSxOnSLghOa,nQcgTWjvg4c 1210953052.202579 j4u32Pc5bif fe80::8000:ffff:ffff:fffd 133 ff02::2 134 icmp - - - - OTH - 0 - 1 64 0 0 nQcgTWjvg4c -#end 2008-05-16-15-51-16 +#close 2008-05-16-15-51-16 diff --git a/testing/btest/Baseline/core.tunnels.teredo/http.log b/testing/btest/Baseline/core.tunnels.teredo/http.log index c0db5fc146..c77297c58d 100644 --- a/testing/btest/Baseline/core.tunnels.teredo/http.log +++ b/testing/btest/Baseline/core.tunnels.teredo/http.log @@ -3,11 +3,11 @@ #empty_field (empty) #unset_field - #path http -#start 2008-05-16-15-50-58 +#open 2008-05-16-15-50-58 #fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p trans_depth method host uri referrer user_agent request_body_len response_body_len status_code status_msg info_code info_msg filename tags username password proxied mime_type md5 extraction_file #types time string addr port addr port count string string string string string count count count string count string string table[enum] string string table[string] string string file 1210953057.917183 3PKsZ2Uye21 192.168.2.16 1578 75.126.203.78 80 1 POST download913.avast.com /cgi-bin/iavs4stats.cgi - Syncer/4.80 (av_pro-1169;f) 589 0 204 - - - (empty) - - - text/plain - - 1210953061.585996 70MGiRM1Qf4 2001:0:4137:9e50:8000:f12a:b9c8:2815 1286 2001:4860:0:2001::68 80 1 GET ipv6.google.com / - Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9b5) Gecko/2008032620 Firefox/3.0b5 0 6640 200 OK - - - (empty) - - - text/html - - 1210953073.381474 70MGiRM1Qf4 2001:0:4137:9e50:8000:f12a:b9c8:2815 1286 2001:4860:0:2001::68 80 2 GET ipv6.google.com /search?hl=en&q=Wireshark+!&btnG=Google+Search http://ipv6.google.com/ Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9b5) Gecko/2008032620 Firefox/3.0b5 0 25119 200 OK - - - (empty) - - - text/html - - 1210953074.674817 c4Zw9TmAE05 192.168.2.16 1580 67.228.110.120 80 1 GET www.wireshark.org / http://ipv6.google.com/search?hl=en&q=Wireshark+%21&btnG=Google+Search Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9b5) Gecko/2008032620 Firefox/3.0b5 0 11845 200 OK - - - (empty) - - - text/xml - - -#end 2008-05-16-15-51-16 +#close 2008-05-16-15-51-16 diff --git a/testing/btest/Baseline/core.tunnels.teredo/tunnel.log b/testing/btest/Baseline/core.tunnels.teredo/tunnel.log index ab14bf68bc..120089caa0 100644 --- a/testing/btest/Baseline/core.tunnels.teredo/tunnel.log +++ b/testing/btest/Baseline/core.tunnels.teredo/tunnel.log @@ -3,7 +3,7 @@ #empty_field (empty) #unset_field - #path tunnel -#start 2008-05-16-15-50-52 +#open 2008-05-16-15-50-52 #fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p tunnel_type action #types time string addr port addr port enum enum 1210953052.202579 nQcgTWjvg4c 192.168.2.16 3797 65.55.158.80 3544 Tunnel::TEREDO Tunnel::DISCOVER @@ -12,4 +12,4 @@ 1210953076.058333 nQcgTWjvg4c 192.168.2.16 3797 65.55.158.80 3544 Tunnel::TEREDO Tunnel::CLOSE 1210953076.058333 GSxOnSLghOa 192.168.2.16 3797 83.170.1.38 32900 Tunnel::TEREDO Tunnel::CLOSE 1210953076.058333 TEfuqmmG4bh 192.168.2.16 3797 65.55.158.81 3544 Tunnel::TEREDO Tunnel::CLOSE -#end 2008-05-16-15-51-16 +#close 2008-05-16-15-51-16 diff --git a/testing/btest/Baseline/core.tunnels.teredo_bubble_with_payload/conn.log b/testing/btest/Baseline/core.tunnels.teredo_bubble_with_payload/conn.log index 7b9ff58624..9d4bf86d57 100644 --- a/testing/btest/Baseline/core.tunnels.teredo_bubble_with_payload/conn.log +++ b/testing/btest/Baseline/core.tunnels.teredo_bubble_with_payload/conn.log @@ -3,14 +3,14 @@ #empty_field (empty) #unset_field - #path conn -#start 2012-06-19-17-39-37 +#open 2012-06-19-17-39-37 #fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p proto service duration orig_bytes resp_bytes conn_state local_orig missed_bytes history orig_pkts orig_ip_bytes resp_pkts resp_ip_bytes tunnel_parents #types time string addr port addr port enum string interval count count string bool count string count count count count table[string] 1340127577.354166 FrJExwHcSal 2001:0:4137:9e50:8000:f12a:b9c8:2815 1286 2001:4860:0:2001::68 80 tcp http 0.052829 1675 10467 S1 - 0 ShADad 10 2279 12 11191 j4u32Pc5bif 1340127577.336558 UWkUyAuUGXf 192.168.2.16 3797 65.55.158.80 3544 udp teredo 0.010291 129 52 SF - 0 Dd 2 185 1 80 (empty) 1340127577.341510 j4u32Pc5bif 192.168.2.16 3797 83.170.1.38 32900 udp teredo 0.065485 2367 11243 SF - 0 Dd 12 2703 13 11607 (empty) -1340127577.339015 k6kgXLOoSKl 192.168.2.16 3797 65.55.158.81 3544 udp teredo - - - SHR - 0 d 0 0 1 137 (empty) +1340127577.339015 k6kgXLOoSKl 192.168.2.16 3797 65.55.158.81 3544 udp - - - - SHR - 0 d 0 0 1 137 (empty) 1340127577.339015 nQcgTWjvg4c fe80::8000:f227:bec8:61af 134 fe80::8000:ffff:ffff:fffd 133 icmp - - - - OTH - 0 - 1 88 0 0 k6kgXLOoSKl 1340127577.343969 TEfuqmmG4bh 2001:0:4137:9e50:8000:f12a:b9c8:2815 128 2001:4860:0:2001::68 129 icmp - 0.007778 4 4 OTH - 0 - 1 52 1 52 UWkUyAuUGXf,j4u32Pc5bif 1340127577.336558 arKYeMETxOg fe80::8000:ffff:ffff:fffd 133 ff02::2 134 icmp - - - - OTH - 0 - 1 64 0 0 UWkUyAuUGXf -#end 2012-06-19-17-39-37 +#close 2012-06-19-17-39-37 diff --git a/testing/btest/Baseline/core.tunnels.teredo_bubble_with_payload/http.log b/testing/btest/Baseline/core.tunnels.teredo_bubble_with_payload/http.log index 12f0d7be7a..e0b223d114 100644 --- a/testing/btest/Baseline/core.tunnels.teredo_bubble_with_payload/http.log +++ b/testing/btest/Baseline/core.tunnels.teredo_bubble_with_payload/http.log @@ -3,9 +3,9 @@ #empty_field (empty) #unset_field - #path http -#start 2012-06-19-17-39-37 +#open 2012-06-19-17-39-37 #fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p trans_depth method host uri referrer user_agent request_body_len response_body_len status_code status_msg info_code info_msg filename tags username password proxied mime_type md5 extraction_file #types time string addr port addr port count string string string string string count count count string count string string table[enum] string string table[string] string string file 1340127577.361683 FrJExwHcSal 2001:0:4137:9e50:8000:f12a:b9c8:2815 1286 2001:4860:0:2001::68 80 1 GET ipv6.google.com / - Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9b5) Gecko/2008032620 Firefox/3.0b5 0 6640 200 OK - - - (empty) - - - text/html - - 1340127577.379360 FrJExwHcSal 2001:0:4137:9e50:8000:f12a:b9c8:2815 1286 2001:4860:0:2001::68 80 2 GET ipv6.google.com /search?hl=en&q=Wireshark+!&btnG=Google+Search http://ipv6.google.com/ Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9b5) Gecko/2008032620 Firefox/3.0b5 0 25119 200 OK - - - (empty) - - - text/html - - -#end 2012-06-19-17-39-37 +#close 2012-06-19-17-39-37 diff --git a/testing/btest/Baseline/core.tunnels.teredo_bubble_with_payload/tunnel.log b/testing/btest/Baseline/core.tunnels.teredo_bubble_with_payload/tunnel.log index 1a14b3edb7..86c2c94c04 100644 --- a/testing/btest/Baseline/core.tunnels.teredo_bubble_with_payload/tunnel.log +++ b/testing/btest/Baseline/core.tunnels.teredo_bubble_with_payload/tunnel.log @@ -3,7 +3,7 @@ #empty_field (empty) #unset_field - #path tunnel -#start 2012-06-19-17-39-37 +#open 2012-06-19-17-39-37 #fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p tunnel_type action #types time string addr port addr port enum enum 1340127577.336558 UWkUyAuUGXf 192.168.2.16 3797 65.55.158.80 3544 Tunnel::TEREDO Tunnel::DISCOVER @@ -12,4 +12,4 @@ 1340127577.406995 UWkUyAuUGXf 192.168.2.16 3797 65.55.158.80 3544 Tunnel::TEREDO Tunnel::CLOSE 1340127577.406995 j4u32Pc5bif 192.168.2.16 3797 83.170.1.38 32900 Tunnel::TEREDO Tunnel::CLOSE 1340127577.406995 k6kgXLOoSKl 192.168.2.16 3797 65.55.158.81 3544 Tunnel::TEREDO Tunnel::CLOSE -#end 2012-06-19-17-39-37 +#close 2012-06-19-17-39-37 diff --git a/testing/btest/Baseline/core.tunnels.teredo_bubble_with_payload/weird.log b/testing/btest/Baseline/core.tunnels.teredo_bubble_with_payload/weird.log index 8b252a5819..764b78656a 100644 --- a/testing/btest/Baseline/core.tunnels.teredo_bubble_with_payload/weird.log +++ b/testing/btest/Baseline/core.tunnels.teredo_bubble_with_payload/weird.log @@ -3,9 +3,9 @@ #empty_field (empty) #unset_field - #path weird -#start 2012-06-19-17-39-37 +#open 2012-10-02-16-53-03 #fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p name addl notice peer #types time string addr port addr port string string bool string +1340127577.341510 j4u32Pc5bif 192.168.2.16 3797 83.170.1.38 32900 Teredo_bubble_with_payload - F bro 1340127577.346849 UWkUyAuUGXf 192.168.2.16 3797 65.55.158.80 3544 Teredo_bubble_with_payload - F bro -1340127577.349292 j4u32Pc5bif 192.168.2.16 3797 83.170.1.38 32900 Teredo_bubble_with_payload - F bro -#end 2012-06-19-17-39-37 +#close 2012-10-02-16-53-03 diff --git a/testing/btest/Baseline/core.vlan-mpls/conn.log b/testing/btest/Baseline/core.vlan-mpls/conn.log index 72e13ee9b4..d4cc8370a5 100644 --- a/testing/btest/Baseline/core.vlan-mpls/conn.log +++ b/testing/btest/Baseline/core.vlan-mpls/conn.log @@ -3,10 +3,10 @@ #empty_field (empty) #unset_field - #path conn -#start 2005-10-07-23-23-55 +#open 2005-10-07-23-23-55 #fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p proto service duration orig_bytes resp_bytes conn_state local_orig missed_bytes history orig_pkts orig_ip_bytes resp_pkts resp_ip_bytes tunnel_parents #types time string addr port addr port enum string interval count count string bool count string count count count count table[string] 952109346.874907 UWkUyAuUGXf 10.1.2.1 11001 10.34.0.1 23 tcp - 2.102560 26 0 SH - 0 SADF 11 470 0 0 (empty) 1128727435.450898 arKYeMETxOg 141.42.64.125 56730 125.190.109.199 80 tcp http 1.733303 98 9417 SF - 0 ShADdFaf 12 730 10 9945 (empty) 1278600802.069419 k6kgXLOoSKl 10.20.80.1 50343 10.0.0.15 80 tcp - 0.004152 9 3429 SF - 0 ShADadfF 7 381 7 3801 (empty) -#end 2010-07-08-14-53-22 +#close 2010-07-08-14-53-22 diff --git a/testing/btest/Baseline/coverage.bare-load-baseline/canonified_loaded_scripts.log b/testing/btest/Baseline/coverage.bare-load-baseline/canonified_loaded_scripts.log index ca8749956f..41209a4084 100644 --- a/testing/btest/Baseline/coverage.bare-load-baseline/canonified_loaded_scripts.log +++ b/testing/btest/Baseline/coverage.bare-load-baseline/canonified_loaded_scripts.log @@ -3,7 +3,7 @@ #empty_field (empty) #unset_field - #path loaded_scripts -#start 2012-07-20-14-34-11 +#open 2012-07-20-14-34-11 #fields name #types string scripts/base/init-bare.bro @@ -30,4 +30,4 @@ scripts/base/init-bare.bro scripts/base/frameworks/input/./readers/raw.bro scripts/base/frameworks/input/./readers/benchmark.bro scripts/policy/misc/loaded-scripts.bro -#end 2012-07-20-14-34-11 +#close 2012-07-20-14-34-11 diff --git a/testing/btest/Baseline/coverage.bare-mode-errors/unique_errors_no_elasticsearch b/testing/btest/Baseline/coverage.bare-mode-errors/unique_errors_no_elasticsearch new file mode 100644 index 0000000000..e95f88e74b --- /dev/null +++ b/testing/btest/Baseline/coverage.bare-mode-errors/unique_errors_no_elasticsearch @@ -0,0 +1 @@ +error: unknown writer type requested diff --git a/testing/btest/Baseline/coverage.default-load-baseline/canonified_loaded_scripts.log b/testing/btest/Baseline/coverage.default-load-baseline/canonified_loaded_scripts.log index b464c916f2..b2afadc0fe 100644 --- a/testing/btest/Baseline/coverage.default-load-baseline/canonified_loaded_scripts.log +++ b/testing/btest/Baseline/coverage.default-load-baseline/canonified_loaded_scripts.log @@ -3,7 +3,7 @@ #empty_field (empty) #unset_field - #path loaded_scripts -#start 2012-07-20-14-34-40 +#open 2012-07-20-14-34-40 #fields name #types string scripts/base/init-bare.bro @@ -110,4 +110,4 @@ scripts/base/init-default.bro scripts/base/protocols/syslog/./consts.bro scripts/base/protocols/syslog/./main.bro scripts/policy/misc/loaded-scripts.bro -#end 2012-07-20-14-34-40 +#close 2012-07-20-14-34-40 diff --git a/testing/btest/Baseline/istate.events-ssl/receiver.http.log b/testing/btest/Baseline/istate.events-ssl/receiver.http.log index c9a996ef5b..3fc7f1b66f 100644 --- a/testing/btest/Baseline/istate.events-ssl/receiver.http.log +++ b/testing/btest/Baseline/istate.events-ssl/receiver.http.log @@ -3,8 +3,8 @@ #empty_field (empty) #unset_field - #path http -#start 2012-07-20-01-53-03 +#open 2012-07-20-01-53-03 #fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p trans_depth method host uri referrer user_agent request_body_len response_body_len status_code status_msg info_code info_msg filename tags username password proxied mime_type md5 extraction_file #types time string addr port addr port count string string string string string count count count string count string string table[enum] string string table[string] string string file 1342749182.906082 arKYeMETxOg 141.42.64.125 56730 125.190.109.199 80 1 GET www.icir.org / - Wget/1.10 0 9130 200 OK - - - (empty) - - - text/html - - -#end 2012-07-20-01-53-04 +#close 2012-07-20-01-53-04 diff --git a/testing/btest/Baseline/istate.events-ssl/sender.http.log b/testing/btest/Baseline/istate.events-ssl/sender.http.log index c9a996ef5b..3fc7f1b66f 100644 --- a/testing/btest/Baseline/istate.events-ssl/sender.http.log +++ b/testing/btest/Baseline/istate.events-ssl/sender.http.log @@ -3,8 +3,8 @@ #empty_field (empty) #unset_field - #path http -#start 2012-07-20-01-53-03 +#open 2012-07-20-01-53-03 #fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p trans_depth method host uri referrer user_agent request_body_len response_body_len status_code status_msg info_code info_msg filename tags username password proxied mime_type md5 extraction_file #types time string addr port addr port count string string string string string count count count string count string string table[enum] string string table[string] string string file 1342749182.906082 arKYeMETxOg 141.42.64.125 56730 125.190.109.199 80 1 GET www.icir.org / - Wget/1.10 0 9130 200 OK - - - (empty) - - - text/html - - -#end 2012-07-20-01-53-04 +#close 2012-07-20-01-53-04 diff --git a/testing/btest/Baseline/istate.events/receiver.http.log b/testing/btest/Baseline/istate.events/receiver.http.log index 566457b996..6862c08b98 100644 --- a/testing/btest/Baseline/istate.events/receiver.http.log +++ b/testing/btest/Baseline/istate.events/receiver.http.log @@ -3,8 +3,8 @@ #empty_field (empty) #unset_field - #path http -#start 2012-07-20-01-53-12 +#open 2012-07-20-01-53-12 #fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p trans_depth method host uri referrer user_agent request_body_len response_body_len status_code status_msg info_code info_msg filename tags username password proxied mime_type md5 extraction_file #types time string addr port addr port count string string string string string count count count string count string string table[enum] string string table[string] string string file 1342749191.765740 arKYeMETxOg 141.42.64.125 56730 125.190.109.199 80 1 GET www.icir.org / - Wget/1.10 0 9130 200 OK - - - (empty) - - - text/html - - -#end 2012-07-20-01-53-13 +#close 2012-07-20-01-53-13 diff --git a/testing/btest/Baseline/istate.events/sender.http.log b/testing/btest/Baseline/istate.events/sender.http.log index 566457b996..6862c08b98 100644 --- a/testing/btest/Baseline/istate.events/sender.http.log +++ b/testing/btest/Baseline/istate.events/sender.http.log @@ -3,8 +3,8 @@ #empty_field (empty) #unset_field - #path http -#start 2012-07-20-01-53-12 +#open 2012-07-20-01-53-12 #fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p trans_depth method host uri referrer user_agent request_body_len response_body_len status_code status_msg info_code info_msg filename tags username password proxied mime_type md5 extraction_file #types time string addr port addr port count string string string string string count count count string count string string table[enum] string string table[string] string string file 1342749191.765740 arKYeMETxOg 141.42.64.125 56730 125.190.109.199 80 1 GET www.icir.org / - Wget/1.10 0 9130 200 OK - - - (empty) - - - text/html - - -#end 2012-07-20-01-53-13 +#close 2012-07-20-01-53-13 diff --git a/testing/btest/Baseline/language.addr/out b/testing/btest/Baseline/language.addr/out new file mode 100644 index 0000000000..b04aac5ce3 --- /dev/null +++ b/testing/btest/Baseline/language.addr/out @@ -0,0 +1,15 @@ +IPv4 address inequality (PASS) +IPv4 address equality (PASS) +IPv4 address comparison (PASS) +IPv4 address comparison (PASS) +size of IPv4 address (PASS) +IPv4 address type inference (PASS) +IPv6 address inequality (PASS) +IPv6 address equality (PASS) +IPv6 address equality (PASS) +IPv6 address comparison (PASS) +IPv6 address comparison (PASS) +IPv6 address not case-sensitive (PASS) +size of IPv6 address (PASS) +IPv6 address type inference (PASS) +IPv4 and IPv6 address inequality (PASS) diff --git a/testing/btest/Baseline/language.any/out b/testing/btest/Baseline/language.any/out new file mode 100644 index 0000000000..4072ce3745 --- /dev/null +++ b/testing/btest/Baseline/language.any/out @@ -0,0 +1,14 @@ +count (PASS) +string (PASS) +pattern (PASS) +bool (PASS) +string (PASS) +count (PASS) +int (PASS) +double (PASS) +pattern (PASS) +addr (PASS) +addr (PASS) +subnet (PASS) +subnet (PASS) +port (PASS) diff --git a/testing/btest/Baseline/language.at-if/out b/testing/btest/Baseline/language.at-if/out new file mode 100644 index 0000000000..b63cbbb714 --- /dev/null +++ b/testing/btest/Baseline/language.at-if/out @@ -0,0 +1,3 @@ +@if (PASS) +@if...@else (PASS) +@if...@else (PASS) diff --git a/testing/btest/Baseline/language.at-ifdef/out b/testing/btest/Baseline/language.at-ifdef/out new file mode 100644 index 0000000000..644a42d407 --- /dev/null +++ b/testing/btest/Baseline/language.at-ifdef/out @@ -0,0 +1,3 @@ +@ifdef (PASS) +@ifdef...@else (PASS) +@ifdef...@else (PASS) diff --git a/testing/btest/Baseline/language.at-ifndef/out b/testing/btest/Baseline/language.at-ifndef/out new file mode 100644 index 0000000000..70abba9b3f --- /dev/null +++ b/testing/btest/Baseline/language.at-ifndef/out @@ -0,0 +1,3 @@ +@ifndef (PASS) +@ifndef...@else (PASS) +@ifndef...@else (PASS) diff --git a/testing/btest/Baseline/language.at-load/out b/testing/btest/Baseline/language.at-load/out new file mode 100644 index 0000000000..5b011543b5 --- /dev/null +++ b/testing/btest/Baseline/language.at-load/out @@ -0,0 +1,4 @@ +function (PASS) +global variable (PASS) +const (PASS) +event (PASS) diff --git a/testing/btest/Baseline/language.bool/out b/testing/btest/Baseline/language.bool/out new file mode 100644 index 0000000000..9e4c6c3d6e --- /dev/null +++ b/testing/btest/Baseline/language.bool/out @@ -0,0 +1,9 @@ +equality operator (PASS) +inequality operator (PASS) +logical or operator (PASS) +logical and operator (PASS) +negation operator (PASS) +absolute value (PASS) +absolute value (PASS) +type inference (PASS) +type inference (PASS) diff --git a/testing/btest/Baseline/language.conditional-expression/out b/testing/btest/Baseline/language.conditional-expression/out new file mode 100644 index 0000000000..0dcbdbd7c7 --- /dev/null +++ b/testing/btest/Baseline/language.conditional-expression/out @@ -0,0 +1,7 @@ +true condition (PASS) +false condition (PASS) +true condition (PASS) +false condition (PASS) +associativity (PASS) +associativity (PASS) +associativity (PASS) diff --git a/testing/btest/Baseline/language.copy/out b/testing/btest/Baseline/language.copy/out new file mode 100644 index 0000000000..675d38aa5d --- /dev/null +++ b/testing/btest/Baseline/language.copy/out @@ -0,0 +1,2 @@ +direct assignment (PASS) +using copy (PASS) diff --git a/testing/btest/Baseline/language.count/out b/testing/btest/Baseline/language.count/out new file mode 100644 index 0000000000..4ef65b6098 --- /dev/null +++ b/testing/btest/Baseline/language.count/out @@ -0,0 +1,18 @@ +type inference (PASS) +counter alias (PASS) +hexadecimal (PASS) +inequality operator (PASS) +relational operator (PASS) +relational operator (PASS) +relational operator (PASS) +relational operator (PASS) +absolute value (PASS) +absolute value (PASS) +pre-increment operator (PASS) +pre-decrement operator (PASS) +modulus operator (PASS) +division operator (PASS) +assignment operator (PASS) +assignment operator (PASS) +max count value = 18446744073709551615 (PASS) +max count value = 18446744073709551615 (PASS) diff --git a/testing/btest/Baseline/language.double/out b/testing/btest/Baseline/language.double/out new file mode 100644 index 0000000000..3f70635588 --- /dev/null +++ b/testing/btest/Baseline/language.double/out @@ -0,0 +1,28 @@ +type inference (PASS) +type inference (PASS) +type inference (PASS) +double representations (PASS) +double representations (PASS) +double representations (PASS) +double representations (PASS) +double representations (PASS) +double representations (PASS) +double representations (PASS) +double representations (PASS) +double representations (PASS) +double representations (PASS) +double representations (PASS) +double representations (PASS) +double representations (PASS) +double representations (PASS) +double representations (PASS) +inequality operator (PASS) +absolute value (PASS) +assignment operator (PASS) +assignment operator (PASS) +relational operator (PASS) +relational operator (PASS) +relational operator (PASS) +relational operator (PASS) +division operator (PASS) +max double value = 1.7976931348623157e+308 (PASS) diff --git a/testing/btest/Baseline/language.enum/out b/testing/btest/Baseline/language.enum/out new file mode 100644 index 0000000000..1bafdd73b0 --- /dev/null +++ b/testing/btest/Baseline/language.enum/out @@ -0,0 +1,4 @@ +enum equality comparison (PASS) +enum equality comparison (PASS) +enum equality comparison (PASS) +type inference (PASS) diff --git a/testing/btest/Baseline/language.event/out b/testing/btest/Baseline/language.event/out new file mode 100644 index 0000000000..d5a22b3745 --- /dev/null +++ b/testing/btest/Baseline/language.event/out @@ -0,0 +1,4 @@ +event statement +event part1 +event part2 +schedule statement diff --git a/testing/btest/Baseline/language.file/out1 b/testing/btest/Baseline/language.file/out1 new file mode 100644 index 0000000000..5ff4194027 --- /dev/null +++ b/testing/btest/Baseline/language.file/out1 @@ -0,0 +1,2 @@ +20 +12 diff --git a/testing/btest/Baseline/language.file/out2 b/testing/btest/Baseline/language.file/out2 new file mode 100644 index 0000000000..12be2d6723 --- /dev/null +++ b/testing/btest/Baseline/language.file/out2 @@ -0,0 +1 @@ +test, 123, 456 diff --git a/testing/btest/Baseline/language.for/out b/testing/btest/Baseline/language.for/out new file mode 100644 index 0000000000..dccc00ce3e --- /dev/null +++ b/testing/btest/Baseline/language.for/out @@ -0,0 +1,3 @@ +for loop (PASS) +for loop with break (PASS) +for loop with next (PASS) diff --git a/testing/btest/Baseline/language.function/out b/testing/btest/Baseline/language.function/out new file mode 100644 index 0000000000..f530024370 --- /dev/null +++ b/testing/btest/Baseline/language.function/out @@ -0,0 +1,11 @@ +no args without return value (PASS) +no args no return value, empty return (PASS) +no args with return value (PASS) +args without return value (PASS) +args with return value (PASS) +multiple args with return value (PASS) +anonymous function without args or return value (PASS) +anonymous function with return value (PASS) +anonymous function with args and return value (PASS) +assign function variable (PASS) +reassign function variable (PASS) diff --git a/testing/btest/Baseline/language.if/out b/testing/btest/Baseline/language.if/out new file mode 100644 index 0000000000..510b66b0cf --- /dev/null +++ b/testing/btest/Baseline/language.if/out @@ -0,0 +1,12 @@ +if T (PASS) +if T else (PASS) +if F else (PASS) +if T else if F (PASS) +if F else if T (PASS) +if T else if T (PASS) +if T else if F else (PASS) +if F else if T else (PASS) +if T else if T else (PASS) +if F else if F else (PASS) +if F else if F else if T else (PASS) +if F else if F else if F else (PASS) diff --git a/testing/btest/Baseline/language.int/out b/testing/btest/Baseline/language.int/out new file mode 100644 index 0000000000..01f018acbe --- /dev/null +++ b/testing/btest/Baseline/language.int/out @@ -0,0 +1,23 @@ +type inference (PASS) +optional '+' sign (PASS) +negative vs. positive (PASS) +negative vs. positive (PASS) +hexadecimal (PASS) +hexadecimal (PASS) +hexadecimal (PASS) +relational operator (PASS) +relational operator (PASS) +relational operator (PASS) +relational operator (PASS) +absolute value (PASS) +absolute value (PASS) +pre-increment operator (PASS) +pre-decrement operator (PASS) +modulus operator (PASS) +division operator (PASS) +assignment operator (PASS) +assignment operator (PASS) +max int value = 9223372036854775807 (PASS) +min int value = -9223372036854775808 (PASS) +max int value = 9223372036854775807 (PASS) +min int value = -9223372036854775808 (PASS) diff --git a/testing/btest/Baseline/language.interval/out b/testing/btest/Baseline/language.interval/out new file mode 100644 index 0000000000..ae9ed5d74e --- /dev/null +++ b/testing/btest/Baseline/language.interval/out @@ -0,0 +1,27 @@ +type inference (PASS) +type inference (PASS) +type inference (PASS) +optional space (PASS) +plural/singular interval are same (PASS) +different units with same numeric value (PASS) +compare different time units (PASS) +compare different time units (PASS) +compare different time units (PASS) +compare different time units (PASS) +compare different time units (PASS) +compare different time units (PASS) +compare different time units (PASS) +add different time units (PASS) +subtract different time units (PASS) +absolute value (PASS) +absolute value (PASS) +assignment operator (PASS) +assignment operator (PASS) +multiplication operator (PASS) +division operator (PASS) +division operator (PASS) +relative size of units (PASS) +relative size of units (PASS) +relative size of units (PASS) +relative size of units (PASS) +relative size of units (PASS) diff --git a/testing/btest/Baseline/language.module/out b/testing/btest/Baseline/language.module/out new file mode 100644 index 0000000000..5b011543b5 --- /dev/null +++ b/testing/btest/Baseline/language.module/out @@ -0,0 +1,4 @@ +function (PASS) +global variable (PASS) +const (PASS) +event (PASS) diff --git a/testing/btest/Baseline/language.no-module/out b/testing/btest/Baseline/language.no-module/out new file mode 100644 index 0000000000..5b011543b5 --- /dev/null +++ b/testing/btest/Baseline/language.no-module/out @@ -0,0 +1,4 @@ +function (PASS) +global variable (PASS) +const (PASS) +event (PASS) diff --git a/testing/btest/Baseline/language.null-statement/out b/testing/btest/Baseline/language.null-statement/out new file mode 100644 index 0000000000..19f86f493a --- /dev/null +++ b/testing/btest/Baseline/language.null-statement/out @@ -0,0 +1 @@ +done diff --git a/testing/btest/Baseline/language.pattern/out b/testing/btest/Baseline/language.pattern/out new file mode 100644 index 0000000000..4a5b8de670 --- /dev/null +++ b/testing/btest/Baseline/language.pattern/out @@ -0,0 +1,8 @@ +type inference (PASS) +equality operator (PASS) +equality operator (order of operands) (PASS) +inequality operator (PASS) +inequality operator (order of operands) (PASS) +in operator (PASS) +in operator (PASS) +!in operator (PASS) diff --git a/testing/btest/Baseline/language.port/out b/testing/btest/Baseline/language.port/out new file mode 100644 index 0000000000..b307388c35 --- /dev/null +++ b/testing/btest/Baseline/language.port/out @@ -0,0 +1,9 @@ +type inference (PASS) +protocol ordering (PASS) +protocol ordering (PASS) +protocol ordering (PASS) +protocol ordering (PASS) +protocol ordering (PASS) +different protocol but same numeric value (PASS) +different protocol but same numeric value (PASS) +equality operator (PASS) diff --git a/testing/btest/Baseline/language.precedence/out b/testing/btest/Baseline/language.precedence/out new file mode 100644 index 0000000000..263ca83529 --- /dev/null +++ b/testing/btest/Baseline/language.precedence/out @@ -0,0 +1,31 @@ +++ and * (PASS) +++ and * (PASS) +* and ++ (PASS) +* and % (PASS) +* and % (PASS) +* and % (PASS) +% and * (PASS) +% and * (PASS) +% and * (PASS) ++ and * (PASS) ++ and * (PASS) ++ and * (PASS) +< and + (PASS) +< and + (PASS) ++ and < (PASS) ++ and < (PASS) ++= and + (PASS) ++= and + (PASS) ++= and + (PASS) +&& and || (PASS) +&& and || (PASS) +&& and || (PASS) +|| and && (PASS) +|| and && (PASS) +|| and && (PASS) +|| and conditional operator (PASS) +|| and conditional operator (PASS) +|| and conditional operator (PASS) +conditional operator and || (PASS) +conditional operator and || (PASS) +conditional operator and || (PASS) diff --git a/testing/btest/Baseline/language.set/out b/testing/btest/Baseline/language.set/out new file mode 100644 index 0000000000..fc157cf7d9 --- /dev/null +++ b/testing/btest/Baseline/language.set/out @@ -0,0 +1,44 @@ +type inference (PASS) +type inference (PASS) +type inference (PASS) +cardinality (PASS) +cardinality (PASS) +cardinality (PASS) +cardinality (PASS) +cardinality (PASS) +cardinality (PASS) +cardinality (PASS) +cardinality (PASS) +cardinality (PASS) +cardinality (PASS) +cardinality (PASS) +iterate over set (PASS) +iterate over set (PASS) +iterate over set (PASS) +iterate over set (PASS) +add element (PASS) +in operator (PASS) +add element (PASS) +add element (PASS) +in operator (PASS) +in operator (PASS) +add element (PASS) +in operator (PASS) +add element (PASS) +in operator (PASS) +add element (PASS) +in operator (PASS) +add element (PASS) +in operator (PASS) +remove element (PASS) +!in operator (PASS) +remove element (PASS) +!in operator (PASS) +remove element (PASS) +!in operator (PASS) +remove element (PASS) +!in operator (PASS) +remove element (PASS) +!in operator (PASS) +remove element (PASS) +!in operator (PASS) diff --git a/testing/btest/Baseline/language.short-circuit/out b/testing/btest/Baseline/language.short-circuit/out new file mode 100644 index 0000000000..c92995ea7c --- /dev/null +++ b/testing/btest/Baseline/language.short-circuit/out @@ -0,0 +1,4 @@ +&& operator (eval. both operands) (PASS) +&& operator (eval. 1st operand) (PASS) +|| operator (eval. 1st operand) (PASS) +|| operator (eval. both operands) (PASS) diff --git a/testing/btest/Baseline/language.string/out b/testing/btest/Baseline/language.string/out new file mode 100644 index 0000000000..5595445ffc --- /dev/null +++ b/testing/btest/Baseline/language.string/out @@ -0,0 +1,29 @@ +type inference (PASS) +tab escape sequence (PASS) +newline escape sequence (PASS) +double quote escape sequence (PASS) +backslash escape sequence (PASS) +1-digit hex escape sequence (PASS) +2-digit hex escape sequence (PASS) +2-digit hex escape sequence (PASS) +2-digit hex escape sequence (PASS) +3-digit octal escape sequence (PASS) +2-digit octal escape sequence (PASS) +1-digit octal escape sequence (PASS) +tab escape sequence (PASS) +tab escape sequence (PASS) +newline escape sequence (PASS) +newline escape sequence (PASS) +double quote escape sequence (PASS) +null escape sequence (PASS) +empty string (PASS) +nonempty string (PASS) +string comparison (PASS) +string comparison (PASS) +string comparison (PASS) +string comparison (PASS) +string concatenation (PASS) +string concatenation (PASS) +multi-line string initialization (PASS) +in operator (PASS) +!in operator (PASS) diff --git a/testing/btest/Baseline/language.subnet/out b/testing/btest/Baseline/language.subnet/out new file mode 100644 index 0000000000..45900a291e --- /dev/null +++ b/testing/btest/Baseline/language.subnet/out @@ -0,0 +1,12 @@ +IPv4 subnet equality (PASS) +IPv4 subnet inequality (PASS) +IPv4 subnet in operator (PASS) +IPv4 subnet !in operator (PASS) +IPv4 subnet type inference (PASS) +IPv6 subnet equality (PASS) +IPv6 subnet inequality (PASS) +IPv6 subnet in operator (PASS) +IPv6 subnet !in operator (PASS) +IPv6 subnet type inference (PASS) +IPv4 and IPv6 subnet inequality (PASS) +IPv4 address and IPv6 subnet (PASS) diff --git a/testing/btest/Baseline/language.table/out b/testing/btest/Baseline/language.table/out new file mode 100644 index 0000000000..514cb6b02d --- /dev/null +++ b/testing/btest/Baseline/language.table/out @@ -0,0 +1,42 @@ +type inference (PASS) +type inference (PASS) +type inference (PASS) +cardinality (PASS) +cardinality (PASS) +cardinality (PASS) +cardinality (PASS) +cardinality (PASS) +cardinality (PASS) +cardinality (PASS) +cardinality (PASS) +cardinality (PASS) +cardinality (PASS) +cardinality (PASS) +iterate over table (PASS) +iterate over table (PASS) +iterate over table (PASS) +iterate over table (PASS) +iterate over table (PASS) +overwrite element (PASS) +add element (PASS) +in operator (PASS) +add element (PASS) +add element (PASS) +in operator (PASS) +in operator (PASS) +add element (PASS) +in operator (PASS) +add element (PASS) +in operator (PASS) +add element (PASS) +in operator (PASS) +remove element (PASS) +!in operator (PASS) +remove element (PASS) +!in operator (PASS) +remove element (PASS) +!in operator (PASS) +remove element (PASS) +!in operator (PASS) +remove element (PASS) +!in operator (PASS) diff --git a/testing/btest/Baseline/language.time/out b/testing/btest/Baseline/language.time/out new file mode 100644 index 0000000000..5e1c8e6b26 --- /dev/null +++ b/testing/btest/Baseline/language.time/out @@ -0,0 +1,7 @@ +type inference (PASS) +add interval (PASS) +subtract interval (PASS) +inequality (PASS) +equality (PASS) +subtract time (PASS) +size operator (PASS) diff --git a/testing/btest/Baseline/language.timeout/out b/testing/btest/Baseline/language.timeout/out new file mode 100644 index 0000000000..790851a6bb --- /dev/null +++ b/testing/btest/Baseline/language.timeout/out @@ -0,0 +1 @@ +timeout diff --git a/testing/btest/Baseline/language.vector/out b/testing/btest/Baseline/language.vector/out new file mode 100644 index 0000000000..0aa3ab0a8f --- /dev/null +++ b/testing/btest/Baseline/language.vector/out @@ -0,0 +1,59 @@ +type inference (PASS) +type inference (PASS) +type inference (PASS) +cardinality (PASS) +cardinality (PASS) +cardinality (PASS) +cardinality (PASS) +cardinality (PASS) +cardinality (PASS) +cardinality (PASS) +cardinality (PASS) +cardinality (PASS) +cardinality (PASS) +cardinality (PASS) +cardinality (PASS) +cardinality (PASS) +cardinality (PASS) +cardinality (PASS) +cardinality (PASS) +zero-based indexing (PASS) +iterate over vector (PASS) +iterate over vector (PASS) +iterate over vector (PASS) +add element (PASS) +access element (PASS) +add element (PASS) +add element (PASS) +access element (PASS) +access element (PASS) +add element (PASS) +access element (PASS) +add element (PASS) +access element (PASS) +add element (PASS) +access element (PASS) +add element (PASS) +access element (PASS) +overwrite element (PASS) +access element (PASS) +overwrite element (PASS) +access element (PASS) +access element (PASS) +overwrite element (PASS) +access element (PASS) +overwrite element (PASS) +access element (PASS) +overwrite element (PASS) +access element (PASS) +overwrite element (PASS) +access element (PASS) +++ operator (PASS) +-- operator (PASS) ++ operator (PASS) +- operator (PASS) +* operator (PASS) +/ operator (PASS) +% operator (PASS) +&& operator (PASS) +|| operator (PASS) diff --git a/testing/btest/Baseline/language.when/out b/testing/btest/Baseline/language.when/out new file mode 100644 index 0000000000..3a052217ab --- /dev/null +++ b/testing/btest/Baseline/language.when/out @@ -0,0 +1,2 @@ +done +lookup successful diff --git a/testing/btest/Baseline/scripts.base.frameworks.communication.communication_log_baseline/send.log b/testing/btest/Baseline/scripts.base.frameworks.communication.communication_log_baseline/send.log index 7e21ff86b7..c6a19029b6 100644 --- a/testing/btest/Baseline/scripts.base.frameworks.communication.communication_log_baseline/send.log +++ b/testing/btest/Baseline/scripts.base.frameworks.communication.communication_log_baseline/send.log @@ -3,7 +3,7 @@ #empty_field (empty) #unset_field - #path communication -#start 2012-07-20-01-49-40 +#open 2012-07-20-01-49-40 #fields ts peer src_name connected_peer_desc connected_peer_addr connected_peer_port level message #types time string string string addr port string string 1342748980.737451 bro parent - - - info [#1/127.0.0.1:47757] added peer @@ -21,4 +21,4 @@ 1342748980.793108 bro parent - - - info terminating... 1342748980.796454 bro child - - - info terminating 1342748980.797536 bro parent - - - info [#1/127.0.0.1:47757] closing connection -#end 2012-07-20-01-49-40 +#close 2012-07-20-01-49-40 diff --git a/testing/btest/Baseline/scripts.base.frameworks.input.basic/out b/testing/btest/Baseline/scripts.base.frameworks.input.basic/out index ebac1866b6..c456298062 100644 --- a/testing/btest/Baseline/scripts.base.frameworks.input.basic/out +++ b/testing/btest/Baseline/scripts.base.frameworks.input.basic/out @@ -1,5 +1,5 @@ { -[-42] = [b=T, e=SSH::LOG, c=21, p=123/unknown, sn=10.0.0.0/24, a=1.2.3.4, d=3.14, t=1315801931.273616, iv=100.0, s=hurz, sc={ +[-42] = [b=T, e=SSH::LOG, c=21, p=123/unknown, sn=10.0.0.0/24, a=1.2.3.4, d=3.14, t=1315801931.273616, iv=100.0, s=hurz, ns=4242, sc={ 2, 4, 1, @@ -12,3 +12,4 @@ BB }, vc=[10, 20, 30], ve=[]] } +4242 diff --git a/testing/btest/Baseline/scripts.base.frameworks.input.bignumber/out b/testing/btest/Baseline/scripts.base.frameworks.input.bignumber/out new file mode 100644 index 0000000000..8b95ed8b19 --- /dev/null +++ b/testing/btest/Baseline/scripts.base.frameworks.input.bignumber/out @@ -0,0 +1,4 @@ +{ +[9223372036854775800] = [c=18446744073709551612], +[-9223372036854775800] = [c=18446744073709551612] +} diff --git a/testing/btest/Baseline/scripts.base.frameworks.input.empty-values-hashing/out b/testing/btest/Baseline/scripts.base.frameworks.input.empty-values-hashing/out new file mode 100644 index 0000000000..474ef45cc2 --- /dev/null +++ b/testing/btest/Baseline/scripts.base.frameworks.input.empty-values-hashing/out @@ -0,0 +1,155 @@ +============PREDICATE============ +Input::EVENT_NEW +[i=1] +[s=, ss=TEST] +============PREDICATE============ +Input::EVENT_NEW +[i=2] +[s=, ss=] +============EVENT============ +Description +[source=../input.log, reader=Input::READER_ASCII, mode=Input::REREAD, name=ssh, destination={ +[2] = [s=, ss=], +[1] = [s=, ss=TEST] +}, idx=, val=, want_record=T, ev=line +{ +print A::outfile, ============EVENT============; +print A::outfile, Description; +print A::outfile, A::description; +print A::outfile, Type; +print A::outfile, A::tpe; +print A::outfile, Left; +print A::outfile, A::left; +print A::outfile, Right; +print A::outfile, A::right; +}, pred=anonymous-function +{ +print A::outfile, ============PREDICATE============; +print A::outfile, A::typ; +print A::outfile, A::left; +print A::outfile, A::right; +return (T); +}, config={ + +}] +Type +Input::EVENT_NEW +Left +[i=1] +Right +[s=, ss=TEST] +============EVENT============ +Description +[source=../input.log, reader=Input::READER_ASCII, mode=Input::REREAD, name=ssh, destination={ +[2] = [s=, ss=], +[1] = [s=, ss=TEST] +}, idx=, val=, want_record=T, ev=line +{ +print A::outfile, ============EVENT============; +print A::outfile, Description; +print A::outfile, A::description; +print A::outfile, Type; +print A::outfile, A::tpe; +print A::outfile, Left; +print A::outfile, A::left; +print A::outfile, Right; +print A::outfile, A::right; +}, pred=anonymous-function +{ +print A::outfile, ============PREDICATE============; +print A::outfile, A::typ; +print A::outfile, A::left; +print A::outfile, A::right; +return (T); +}, config={ + +}] +Type +Input::EVENT_NEW +Left +[i=2] +Right +[s=, ss=] +==========SERVERS============ +{ +[2] = [s=, ss=], +[1] = [s=, ss=TEST] +} +============PREDICATE============ +Input::EVENT_CHANGED +[i=1] +[s=TEST, ss=] +============PREDICATE============ +Input::EVENT_CHANGED +[i=2] +[s=TEST, ss=TEST] +============EVENT============ +Description +[source=../input.log, reader=Input::READER_ASCII, mode=Input::REREAD, name=ssh, destination={ +[2] = [s=TEST, ss=TEST], +[1] = [s=TEST, ss=] +}, idx=, val=, want_record=T, ev=line +{ +print A::outfile, ============EVENT============; +print A::outfile, Description; +print A::outfile, A::description; +print A::outfile, Type; +print A::outfile, A::tpe; +print A::outfile, Left; +print A::outfile, A::left; +print A::outfile, Right; +print A::outfile, A::right; +}, pred=anonymous-function +{ +print A::outfile, ============PREDICATE============; +print A::outfile, A::typ; +print A::outfile, A::left; +print A::outfile, A::right; +return (T); +}, config={ + +}] +Type +Input::EVENT_CHANGED +Left +[i=1] +Right +[s=, ss=TEST] +============EVENT============ +Description +[source=../input.log, reader=Input::READER_ASCII, mode=Input::REREAD, name=ssh, destination={ +[2] = [s=TEST, ss=TEST], +[1] = [s=TEST, ss=] +}, idx=, val=, want_record=T, ev=line +{ +print A::outfile, ============EVENT============; +print A::outfile, Description; +print A::outfile, A::description; +print A::outfile, Type; +print A::outfile, A::tpe; +print A::outfile, Left; +print A::outfile, A::left; +print A::outfile, Right; +print A::outfile, A::right; +}, pred=anonymous-function +{ +print A::outfile, ============PREDICATE============; +print A::outfile, A::typ; +print A::outfile, A::left; +print A::outfile, A::right; +return (T); +}, config={ + +}] +Type +Input::EVENT_CHANGED +Left +[i=2] +Right +[s=, ss=] +==========SERVERS============ +{ +[2] = [s=TEST, ss=TEST], +[1] = [s=TEST, ss=] +} +done diff --git a/testing/btest/Baseline/scripts.base.frameworks.input.invalidnumbers/.stderrwithoutfirstline b/testing/btest/Baseline/scripts.base.frameworks.input.invalidnumbers/.stderrwithoutfirstline new file mode 100644 index 0000000000..3ef51e40f2 --- /dev/null +++ b/testing/btest/Baseline/scripts.base.frameworks.input.invalidnumbers/.stderrwithoutfirstline @@ -0,0 +1,8 @@ +error: ../input.log/Input::READER_ASCII: Number '12129223372036854775800' out of supported range. +error: ../input.log/Input::READER_ASCII: Could not convert line '12129223372036854775800 121218446744073709551612' to Val. Ignoring line. +warning: ../input.log/Input::READER_ASCII: Number '9223372036854775801TEXTHERE' contained non-numeric trailing characters. Ignored trailing characters 'TEXTHERE' +warning: ../input.log/Input::READER_ASCII: Number '1Justtext' contained non-numeric trailing characters. Ignored trailing characters 'Justtext' +error: ../input.log/Input::READER_ASCII: String 'Justtext' contained no parseable number +error: ../input.log/Input::READER_ASCII: Could not convert line 'Justtext 1' to Val. Ignoring line. +received termination signal +>>> diff --git a/testing/btest/Baseline/scripts.base.frameworks.input.invalidnumbers/out b/testing/btest/Baseline/scripts.base.frameworks.input.invalidnumbers/out new file mode 100644 index 0000000000..56b2736006 --- /dev/null +++ b/testing/btest/Baseline/scripts.base.frameworks.input.invalidnumbers/out @@ -0,0 +1,4 @@ +{ +[9223372036854775800] = [c=4], +[9223372036854775801] = [c=1] +} diff --git a/testing/btest/Baseline/scripts.base.frameworks.input.missing-file/bro..stderr b/testing/btest/Baseline/scripts.base.frameworks.input.missing-file/bro..stderr new file mode 100644 index 0000000000..4380007b93 --- /dev/null +++ b/testing/btest/Baseline/scripts.base.frameworks.input.missing-file/bro..stderr @@ -0,0 +1,5 @@ +error: does-not-exist.dat/Input::READER_ASCII: Init: cannot open does-not-exist.dat +error: does-not-exist.dat/Input::READER_ASCII: Init failed +warning: Stream input is already queued for removal. Ignoring remove. +error: does-not-exist.dat/Input::READER_ASCII: terminating thread +received termination signal diff --git a/testing/btest/Baseline/scripts.base.frameworks.input.predicaterefusesecondsamerecord/out b/testing/btest/Baseline/scripts.base.frameworks.input.predicaterefusesecondsamerecord/out new file mode 100644 index 0000000000..f752ff451a --- /dev/null +++ b/testing/btest/Baseline/scripts.base.frameworks.input.predicaterefusesecondsamerecord/out @@ -0,0 +1,3 @@ +{ +[1.228.83.33] = [asn=9318 HANARO-AS Hanaro Telecom Inc., severity=medium, confidence=95, detecttime=1342569600.0] +} diff --git a/testing/btest/Baseline/scripts.base.frameworks.input.reread/out b/testing/btest/Baseline/scripts.base.frameworks.input.reread/out index 8b55ced2ac..538a6dec18 100644 --- a/testing/btest/Baseline/scripts.base.frameworks.input.reread/out +++ b/testing/btest/Baseline/scripts.base.frameworks.input.reread/out @@ -1084,7 +1084,7 @@ BB } ============PREDICATE============ Input::EVENT_REMOVED -[i=-43] +[i=-44] [b=F, e=SSH::LOG, c=21, p=123/unknown, sn=10.0.0.0/24, a=1.2.3.4, d=3.14, t=1315801931.273616, iv=100.0, s=hurz, sc={ 2, 4, @@ -1096,6 +1096,21 @@ AA, BB }, se={ +}, vc=[10, 20, 30], ve=[]] +============PREDICATE============ +Input::EVENT_REMOVED +[i=-42] +[b=T, e=SSH::LOG, c=21, p=123/unknown, sn=10.0.0.0/24, a=1.2.3.4, d=3.14, t=1315801931.273616, iv=100.0, s=hurz, sc={ +2, +4, +1, +3 +}, ss={ +CC, +AA, +BB +}, se={ + }, vc=[10, 20, 30], ve=[]] ============PREDICATE============ Input::EVENT_REMOVED @@ -1111,21 +1126,6 @@ AA, BB }, se={ -}, vc=[10, 20, 30], ve=[]] -============PREDICATE============ -Input::EVENT_REMOVED -[i=-44] -[b=F, e=SSH::LOG, c=21, p=123/unknown, sn=10.0.0.0/24, a=1.2.3.4, d=3.14, t=1315801931.273616, iv=100.0, s=hurz, sc={ -2, -4, -1, -3 -}, ss={ -CC, -AA, -BB -}, se={ - }, vc=[10, 20, 30], ve=[]] ============PREDICATE============ Input::EVENT_REMOVED @@ -1159,25 +1159,7 @@ BB }, vc=[10, 20, 30], ve=[]] ============PREDICATE============ Input::EVENT_REMOVED -[i=-42] -[b=T, e=SSH::LOG, c=21, p=123/unknown, sn=10.0.0.0/24, a=1.2.3.4, d=3.14, t=1315801931.273616, iv=100.0, s=hurz, sc={ -2, -4, -1, -3 -}, ss={ -CC, -AA, -BB -}, se={ - -}, vc=[10, 20, 30], ve=[]] -============EVENT============ -Description -Input::EVENT_REMOVED -Type [i=-43] -Left [b=F, e=SSH::LOG, c=21, p=123/unknown, sn=10.0.0.0/24, a=1.2.3.4, d=3.14, t=1315801931.273616, iv=100.0, s=hurz, sc={ 2, 4, @@ -1190,14 +1172,10 @@ BB }, se={ }, vc=[10, 20, 30], ve=[]] -Right ============EVENT============ Description -Input::EVENT_REMOVED -Type -[i=-46] -Left -[b=F, e=SSH::LOG, c=21, p=123/unknown, sn=10.0.0.0/24, a=1.2.3.4, d=3.14, t=1315801931.273616, iv=100.0, s=hurz, sc={ +[source=../input.log, reader=Input::READER_ASCII, mode=Input::REREAD, name=ssh, destination={ +[-48] = [b=F, e=SSH::LOG, c=21, p=123/unknown, sn=10.0.0.0/24, a=1.2.3.4, d=3.14, t=1315801931.273616, iv=100.0, s=hurz, sc={ 2, 4, 1, @@ -1209,13 +1187,32 @@ BB }, se={ }, vc=[10, 20, 30], ve=[]] -Right -============EVENT============ -Description -Input::EVENT_REMOVED +}, idx=, val=, want_record=T, ev=line +{ +print A::outfile, ============EVENT============; +print A::outfile, Description; +print A::outfile, A::description; +print A::outfile, Type; +print A::outfile, A::tpe; +print A::outfile, Left; +print A::outfile, A::left; +print A::outfile, Right; +print A::outfile, A::right; +}, pred=anonymous-function +{ +print A::outfile, ============PREDICATE============; +print A::outfile, A::typ; +print A::outfile, A::left; +print A::outfile, A::right; +return (T); +}, config={ + +}] Type +Input::EVENT_REMOVED +Left [i=-44] -Left +Right [b=F, e=SSH::LOG, c=21, p=123/unknown, sn=10.0.0.0/24, a=1.2.3.4, d=3.14, t=1315801931.273616, iv=100.0, s=hurz, sc={ 2, 4, @@ -1228,14 +1225,10 @@ BB }, se={ }, vc=[10, 20, 30], ve=[]] -Right ============EVENT============ Description -Input::EVENT_REMOVED -Type -[i=-47] -Left -[b=F, e=SSH::LOG, c=21, p=123/unknown, sn=10.0.0.0/24, a=1.2.3.4, d=3.14, t=1315801931.273616, iv=100.0, s=hurz, sc={ +[source=../input.log, reader=Input::READER_ASCII, mode=Input::REREAD, name=ssh, destination={ +[-48] = [b=F, e=SSH::LOG, c=21, p=123/unknown, sn=10.0.0.0/24, a=1.2.3.4, d=3.14, t=1315801931.273616, iv=100.0, s=hurz, sc={ 2, 4, 1, @@ -1247,32 +1240,32 @@ BB }, se={ }, vc=[10, 20, 30], ve=[]] -Right -============EVENT============ -Description -Input::EVENT_REMOVED -Type -[i=-45] -Left -[b=F, e=SSH::LOG, c=21, p=123/unknown, sn=10.0.0.0/24, a=1.2.3.4, d=3.14, t=1315801931.273616, iv=100.0, s=hurz, sc={ -2, -4, -1, -3 -}, ss={ -CC, -AA, -BB -}, se={ +}, idx=, val=, want_record=T, ev=line +{ +print A::outfile, ============EVENT============; +print A::outfile, Description; +print A::outfile, A::description; +print A::outfile, Type; +print A::outfile, A::tpe; +print A::outfile, Left; +print A::outfile, A::left; +print A::outfile, Right; +print A::outfile, A::right; +}, pred=anonymous-function +{ +print A::outfile, ============PREDICATE============; +print A::outfile, A::typ; +print A::outfile, A::left; +print A::outfile, A::right; +return (T); +}, config={ -}, vc=[10, 20, 30], ve=[]] -Right -============EVENT============ -Description -Input::EVENT_REMOVED +}] Type +Input::EVENT_REMOVED +Left [i=-42] -Left +Right [b=T, e=SSH::LOG, c=21, p=123/unknown, sn=10.0.0.0/24, a=1.2.3.4, d=3.14, t=1315801931.273616, iv=100.0, s=hurz, sc={ 2, 4, @@ -1285,7 +1278,218 @@ BB }, se={ }, vc=[10, 20, 30], ve=[]] +============EVENT============ +Description +[source=../input.log, reader=Input::READER_ASCII, mode=Input::REREAD, name=ssh, destination={ +[-48] = [b=F, e=SSH::LOG, c=21, p=123/unknown, sn=10.0.0.0/24, a=1.2.3.4, d=3.14, t=1315801931.273616, iv=100.0, s=hurz, sc={ +2, +4, +1, +3 +}, ss={ +CC, +AA, +BB +}, se={ + +}, vc=[10, 20, 30], ve=[]] +}, idx=, val=, want_record=T, ev=line +{ +print A::outfile, ============EVENT============; +print A::outfile, Description; +print A::outfile, A::description; +print A::outfile, Type; +print A::outfile, A::tpe; +print A::outfile, Left; +print A::outfile, A::left; +print A::outfile, Right; +print A::outfile, A::right; +}, pred=anonymous-function +{ +print A::outfile, ============PREDICATE============; +print A::outfile, A::typ; +print A::outfile, A::left; +print A::outfile, A::right; +return (T); +}, config={ + +}] +Type +Input::EVENT_REMOVED +Left +[i=-46] Right +[b=F, e=SSH::LOG, c=21, p=123/unknown, sn=10.0.0.0/24, a=1.2.3.4, d=3.14, t=1315801931.273616, iv=100.0, s=hurz, sc={ +2, +4, +1, +3 +}, ss={ +CC, +AA, +BB +}, se={ + +}, vc=[10, 20, 30], ve=[]] +============EVENT============ +Description +[source=../input.log, reader=Input::READER_ASCII, mode=Input::REREAD, name=ssh, destination={ +[-48] = [b=F, e=SSH::LOG, c=21, p=123/unknown, sn=10.0.0.0/24, a=1.2.3.4, d=3.14, t=1315801931.273616, iv=100.0, s=hurz, sc={ +2, +4, +1, +3 +}, ss={ +CC, +AA, +BB +}, se={ + +}, vc=[10, 20, 30], ve=[]] +}, idx=, val=, want_record=T, ev=line +{ +print A::outfile, ============EVENT============; +print A::outfile, Description; +print A::outfile, A::description; +print A::outfile, Type; +print A::outfile, A::tpe; +print A::outfile, Left; +print A::outfile, A::left; +print A::outfile, Right; +print A::outfile, A::right; +}, pred=anonymous-function +{ +print A::outfile, ============PREDICATE============; +print A::outfile, A::typ; +print A::outfile, A::left; +print A::outfile, A::right; +return (T); +}, config={ + +}] +Type +Input::EVENT_REMOVED +Left +[i=-47] +Right +[b=F, e=SSH::LOG, c=21, p=123/unknown, sn=10.0.0.0/24, a=1.2.3.4, d=3.14, t=1315801931.273616, iv=100.0, s=hurz, sc={ +2, +4, +1, +3 +}, ss={ +CC, +AA, +BB +}, se={ + +}, vc=[10, 20, 30], ve=[]] +============EVENT============ +Description +[source=../input.log, reader=Input::READER_ASCII, mode=Input::REREAD, name=ssh, destination={ +[-48] = [b=F, e=SSH::LOG, c=21, p=123/unknown, sn=10.0.0.0/24, a=1.2.3.4, d=3.14, t=1315801931.273616, iv=100.0, s=hurz, sc={ +2, +4, +1, +3 +}, ss={ +CC, +AA, +BB +}, se={ + +}, vc=[10, 20, 30], ve=[]] +}, idx=, val=, want_record=T, ev=line +{ +print A::outfile, ============EVENT============; +print A::outfile, Description; +print A::outfile, A::description; +print A::outfile, Type; +print A::outfile, A::tpe; +print A::outfile, Left; +print A::outfile, A::left; +print A::outfile, Right; +print A::outfile, A::right; +}, pred=anonymous-function +{ +print A::outfile, ============PREDICATE============; +print A::outfile, A::typ; +print A::outfile, A::left; +print A::outfile, A::right; +return (T); +}, config={ + +}] +Type +Input::EVENT_REMOVED +Left +[i=-45] +Right +[b=F, e=SSH::LOG, c=21, p=123/unknown, sn=10.0.0.0/24, a=1.2.3.4, d=3.14, t=1315801931.273616, iv=100.0, s=hurz, sc={ +2, +4, +1, +3 +}, ss={ +CC, +AA, +BB +}, se={ + +}, vc=[10, 20, 30], ve=[]] +============EVENT============ +Description +[source=../input.log, reader=Input::READER_ASCII, mode=Input::REREAD, name=ssh, destination={ +[-48] = [b=F, e=SSH::LOG, c=21, p=123/unknown, sn=10.0.0.0/24, a=1.2.3.4, d=3.14, t=1315801931.273616, iv=100.0, s=hurz, sc={ +2, +4, +1, +3 +}, ss={ +CC, +AA, +BB +}, se={ + +}, vc=[10, 20, 30], ve=[]] +}, idx=, val=, want_record=T, ev=line +{ +print A::outfile, ============EVENT============; +print A::outfile, Description; +print A::outfile, A::description; +print A::outfile, Type; +print A::outfile, A::tpe; +print A::outfile, Left; +print A::outfile, A::left; +print A::outfile, Right; +print A::outfile, A::right; +}, pred=anonymous-function +{ +print A::outfile, ============PREDICATE============; +print A::outfile, A::typ; +print A::outfile, A::left; +print A::outfile, A::right; +return (T); +}, config={ + +}] +Type +Input::EVENT_REMOVED +Left +[i=-43] +Right +[b=F, e=SSH::LOG, c=21, p=123/unknown, sn=10.0.0.0/24, a=1.2.3.4, d=3.14, t=1315801931.273616, iv=100.0, s=hurz, sc={ +2, +4, +1, +3 +}, ss={ +CC, +AA, +BB +}, se={ + +}, vc=[10, 20, 30], ve=[]] ==========SERVERS============ { [-48] = [b=F, e=SSH::LOG, c=21, p=123/unknown, sn=10.0.0.0/24, a=1.2.3.4, d=3.14, t=1315801931.273616, iv=100.0, s=hurz, sc={ diff --git a/testing/btest/Baseline/scripts.base.frameworks.input.set/out b/testing/btest/Baseline/scripts.base.frameworks.input.set/out new file mode 100644 index 0000000000..998244cf3f --- /dev/null +++ b/testing/btest/Baseline/scripts.base.frameworks.input.set/out @@ -0,0 +1,7 @@ +{ +192.168.17.7, +192.168.17.42, +192.168.17.14, +192.168.17.1, +192.168.17.2 +} diff --git a/testing/btest/Baseline/scripts.base.frameworks.input.setseparator/out b/testing/btest/Baseline/scripts.base.frameworks.input.setseparator/out new file mode 100644 index 0000000000..d0e0f53310 --- /dev/null +++ b/testing/btest/Baseline/scripts.base.frameworks.input.setseparator/out @@ -0,0 +1,10 @@ +{ +[1] = [s={ +b, +e, +d, +c, +f, +a +}, ss=[1, 2, 3, 4, 5, 6]] +} diff --git a/testing/btest/Baseline/scripts.base.frameworks.input.setspecialcases/out b/testing/btest/Baseline/scripts.base.frameworks.input.setspecialcases/out new file mode 100644 index 0000000000..62229f7f37 --- /dev/null +++ b/testing/btest/Baseline/scripts.base.frameworks.input.setspecialcases/out @@ -0,0 +1,23 @@ +{ +[2] = [s={ +, +testing +}, s=[testing, , testing]], +[4] = [s={ +, +testing +}, s=[testing, ]], +[6] = [s={ + +}, s=[]], +[1] = [s={ +testing,testing,testing, +}, s=[testing,testing,testing,]], +[5] = [s={ + +}, s=[, , , ]], +[3] = [s={ +, +testing +}, s=[, testing]] +} diff --git a/testing/btest/Baseline/scripts.base.frameworks.input.subrecord-event/out b/testing/btest/Baseline/scripts.base.frameworks.input.subrecord-event/out new file mode 100644 index 0000000000..197cb54df9 --- /dev/null +++ b/testing/btest/Baseline/scripts.base.frameworks.input.subrecord-event/out @@ -0,0 +1,12 @@ +[sub=[b=T, e=SSH::LOG, c=21, p=123/unknown, sn=10.0.0.0/24, two=[a=1.2.3.4, d=3.14]], t=1315801931.273616, iv=100.0, s=hurz, sc={ +2, +4, +1, +3 +}, ss={ +CC, +AA, +BB +}, se={ + +}, vc=[10, 20, 30], ve=[]] diff --git a/testing/btest/Baseline/scripts.base.frameworks.input.subrecord/out b/testing/btest/Baseline/scripts.base.frameworks.input.subrecord/out new file mode 100644 index 0000000000..c7e46dfacd --- /dev/null +++ b/testing/btest/Baseline/scripts.base.frameworks.input.subrecord/out @@ -0,0 +1,14 @@ +{ +[-42] = [sub=[b=T, e=SSH::LOG, c=21, p=123/unknown, sn=10.0.0.0/24, two=[a=1.2.3.4, d=3.14]], t=1315801931.273616, iv=100.0, s=hurz, sc={ +2, +4, +1, +3 +}, ss={ +CC, +AA, +BB +}, se={ + +}, vc=[10, 20, 30], ve=[]] +} diff --git a/testing/btest/Baseline/scripts.base.frameworks.input.unsupported_types/out b/testing/btest/Baseline/scripts.base.frameworks.input.unsupported_types/out new file mode 100644 index 0000000000..7ef82cf368 --- /dev/null +++ b/testing/btest/Baseline/scripts.base.frameworks.input.unsupported_types/out @@ -0,0 +1,14 @@ +{ +[-42] = [fi=, b=T, e=SSH::LOG, c=21, p=123/unknown, sn=10.0.0.0/24, a=1.2.3.4, d=3.14, t=1315801931.273616, iv=100.0, s=hurz, sc={ +2, +4, +1, +3 +}, ss={ +CC, +AA, +BB +}, se={ + +}, vc=[10, 20, 30], ve=[]] +} diff --git a/testing/btest/Baseline/scripts.base.frameworks.logging.adapt-filter/ssh-new-default.log b/testing/btest/Baseline/scripts.base.frameworks.logging.adapt-filter/ssh-new-default.log index a0359c2d70..655d9a5fbd 100644 --- a/testing/btest/Baseline/scripts.base.frameworks.logging.adapt-filter/ssh-new-default.log +++ b/testing/btest/Baseline/scripts.base.frameworks.logging.adapt-filter/ssh-new-default.log @@ -3,9 +3,9 @@ #empty_field (empty) #unset_field - #path ssh-new-default -#start 2012-07-20-01-49-19 +#open 2012-07-20-01-49-19 #fields t id.orig_h id.orig_p id.resp_h id.resp_p status country #types time addr port addr port string string 1342748959.430282 1.2.3.4 1234 2.3.4.5 80 success unknown 1342748959.430282 1.2.3.4 1234 2.3.4.5 80 failure US -#end 2012-07-20-01-49-19 +#close 2012-07-20-01-49-19 diff --git a/testing/btest/Baseline/scripts.base.frameworks.logging.ascii-binary/ssh.log b/testing/btest/Baseline/scripts.base.frameworks.logging.ascii-binary/ssh.log index 0c826f9694..b2528467a1 100644 --- a/testing/btest/Baseline/scripts.base.frameworks.logging.ascii-binary/ssh.log +++ b/testing/btest/Baseline/scripts.base.frameworks.logging.ascii-binary/ssh.log @@ -3,10 +3,10 @@ #empty_field|(empty) #unset_field|- #path|ssh -#start|2012-07-20-01-49-19 +#open|2012-07-20-01-49-19 #fields|data|data2 #types|string|string abc\x0a\xffdef|DATA2 abc\x7c\xffdef|DATA2 abc\xff\x7cdef|DATA2 -#end|2012-07-20-01-49-19 +#close|2012-07-20-01-49-19 diff --git a/testing/btest/Baseline/scripts.base.frameworks.logging.ascii-escape-notset-str/test.log b/testing/btest/Baseline/scripts.base.frameworks.logging.ascii-escape-notset-str/test.log index b1a4ba52e2..b77541d35e 100644 --- a/testing/btest/Baseline/scripts.base.frameworks.logging.ascii-escape-notset-str/test.log +++ b/testing/btest/Baseline/scripts.base.frameworks.logging.ascii-escape-notset-str/test.log @@ -3,8 +3,8 @@ #empty_field (empty) #unset_field - #path test -#start 2012-07-20-01-49-19 +#open 2012-07-20-01-49-19 #fields x y z #types string string string \x2d - (empty) -#end 2012-07-20-01-49-19 +#close 2012-07-20-01-49-19 diff --git a/testing/btest/Baseline/scripts.base.frameworks.logging.ascii-escape-odd-url/http.log b/testing/btest/Baseline/scripts.base.frameworks.logging.ascii-escape-odd-url/http.log index 683f149317..f1ff4db3b8 100644 --- a/testing/btest/Baseline/scripts.base.frameworks.logging.ascii-escape-odd-url/http.log +++ b/testing/btest/Baseline/scripts.base.frameworks.logging.ascii-escape-odd-url/http.log @@ -3,8 +3,8 @@ #empty_field (empty) #unset_field - #path http -#start 2011-09-12-03-57-36 +#open 2011-09-12-03-57-36 #fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p trans_depth method host uri referrer user_agent request_body_len response_body_len status_code status_msg info_code info_msg filename tags username password proxied mime_type md5 extraction_file #types time string addr port addr port count string string string string string count count count string count string string table[enum] string string table[string] string string file 1315799856.264750 UWkUyAuUGXf 10.0.1.104 64216 193.40.5.162 80 1 GET lepo.it.da.ut.ee /~cect/teoreetilised seminarid_2010/arheoloogia_uurimisr\xfchma_seminar/Joyce et al - The Languages of Archaeology ~ Dialogue, Narrative and Writing.pdf - Wget/1.12 (darwin10.8.0) 0 346 404 Not Found - - - (empty) - - - text/html - - -#end 2011-09-12-03-57-37 +#close 2011-09-12-03-57-37 diff --git a/testing/btest/Baseline/scripts.base.frameworks.logging.ascii-escape-set-separator/test.log b/testing/btest/Baseline/scripts.base.frameworks.logging.ascii-escape-set-separator/test.log index a03c6f954b..25e9319eec 100644 --- a/testing/btest/Baseline/scripts.base.frameworks.logging.ascii-escape-set-separator/test.log +++ b/testing/btest/Baseline/scripts.base.frameworks.logging.ascii-escape-set-separator/test.log @@ -3,8 +3,8 @@ #empty_field (empty) #unset_field - #path test -#start 2012-07-20-01-49-19 +#open 2012-07-20-01-49-19 #fields ss #types table[string] CC,AA,\x2c,\x2c\x2c -#end 2012-07-20-01-49-19 +#close 2012-07-20-01-49-19 diff --git a/testing/btest/Baseline/scripts.base.frameworks.logging.ascii-escape/ssh.log b/testing/btest/Baseline/scripts.base.frameworks.logging.ascii-escape/ssh.log index 0c6a266de0..d61eae873a 100644 --- a/testing/btest/Baseline/scripts.base.frameworks.logging.ascii-escape/ssh.log +++ b/testing/btest/Baseline/scripts.base.frameworks.logging.ascii-escape/ssh.log @@ -5,8 +5,8 @@ #path||ssh #fields||t||id.orig_h||id.orig_p||id.resp_h||id.resp_p||status||country #types||time||addr||port||addr||port||string||string -1342759749.586006||1.2.3.4||1234||2.3.4.5||80||success||unknown -1342759749.586006||1.2.3.4||1234||2.3.4.5||80||failure||US -1342759749.586006||1.2.3.4||1234||2.3.4.5||80||fa\x7c\x7cure||UK -1342759749.586006||1.2.3.4||1234||2.3.4.5||80||su\x7c\x7cess||BR -1342759749.586006||1.2.3.4||1234||2.3.4.5||80||failure||MX +1343417536.767956||1.2.3.4||1234||2.3.4.5||80||success||unknown +1343417536.767956||1.2.3.4||1234||2.3.4.5||80||failure||US +1343417536.767956||1.2.3.4||1234||2.3.4.5||80||fa\x7c\x7cure||UK +1343417536.767956||1.2.3.4||1234||2.3.4.5||80||su\x7c\x7cess||BR +1343417536.767956||1.2.3.4||1234||2.3.4.5||80||failure||MX diff --git a/testing/btest/Baseline/scripts.base.frameworks.logging.ascii-line-like-comment/test.log b/testing/btest/Baseline/scripts.base.frameworks.logging.ascii-line-like-comment/test.log index 21b81abf95..0f825462ab 100644 --- a/testing/btest/Baseline/scripts.base.frameworks.logging.ascii-line-like-comment/test.log +++ b/testing/btest/Baseline/scripts.base.frameworks.logging.ascii-line-like-comment/test.log @@ -3,10 +3,10 @@ #empty_field (empty) #unset_field - #path test -#start 2012-07-20-01-49-22 +#open 2012-07-20-01-49-22 #fields data c #types string count Test1 42 \x23Kaputt 42 Test2 42 -#end 2012-07-20-01-49-22 +#close 2012-07-20-01-49-22 diff --git a/testing/btest/Baseline/scripts.base.frameworks.logging.ascii-timestamps/test.log b/testing/btest/Baseline/scripts.base.frameworks.logging.ascii-timestamps/test.log index 5fba268afa..c644dab007 100644 --- a/testing/btest/Baseline/scripts.base.frameworks.logging.ascii-timestamps/test.log +++ b/testing/btest/Baseline/scripts.base.frameworks.logging.ascii-timestamps/test.log @@ -3,7 +3,7 @@ #empty_field (empty) #unset_field - #path test -#start 2012-07-20-01-49-20 +#open 2012-07-20-01-49-20 #fields data #types time 1234567890.000000 @@ -14,4 +14,4 @@ 1234567890.000010 1234567890.000001 1234567890.000000 -#end 2012-07-20-01-49-20 +#close 2012-07-20-01-49-20 diff --git a/testing/btest/Baseline/scripts.base.frameworks.logging.attr-extend/ssh.log b/testing/btest/Baseline/scripts.base.frameworks.logging.attr-extend/ssh.log index 7d3bbc0774..9eb2f0e663 100644 --- a/testing/btest/Baseline/scripts.base.frameworks.logging.attr-extend/ssh.log +++ b/testing/btest/Baseline/scripts.base.frameworks.logging.attr-extend/ssh.log @@ -3,8 +3,8 @@ #empty_field (empty) #unset_field - #path ssh -#start 2012-07-20-01-49-20 +#open 2012-07-20-01-49-20 #fields status country a1 b1 b2 #types string string count count count success unknown 1 3 4 -#end 2012-07-20-01-49-20 +#close 2012-07-20-01-49-20 diff --git a/testing/btest/Baseline/scripts.base.frameworks.logging.attr/ssh.log b/testing/btest/Baseline/scripts.base.frameworks.logging.attr/ssh.log index c3163dba6f..bcedd1174e 100644 --- a/testing/btest/Baseline/scripts.base.frameworks.logging.attr/ssh.log +++ b/testing/btest/Baseline/scripts.base.frameworks.logging.attr/ssh.log @@ -3,7 +3,7 @@ #empty_field (empty) #unset_field - #path ssh -#start 2012-07-20-01-49-20 +#open 2012-07-20-01-49-20 #fields status country #types string string success unknown @@ -11,4 +11,4 @@ failure US failure UK success BR failure MX -#end 2012-07-20-01-49-20 +#close 2012-07-20-01-49-20 diff --git a/testing/btest/Baseline/scripts.base.frameworks.logging.empty-event/ssh.log b/testing/btest/Baseline/scripts.base.frameworks.logging.empty-event/ssh.log index 42f945bf0c..b255ac3489 100644 --- a/testing/btest/Baseline/scripts.base.frameworks.logging.empty-event/ssh.log +++ b/testing/btest/Baseline/scripts.base.frameworks.logging.empty-event/ssh.log @@ -3,7 +3,7 @@ #empty_field (empty) #unset_field - #path ssh -#start 2012-07-20-01-49-20 +#open 2012-07-20-01-49-20 #fields t id.orig_h id.orig_p id.resp_h id.resp_p status country #types time addr port addr port string string 1342748960.468458 1.2.3.4 1234 2.3.4.5 80 success unknown @@ -11,4 +11,4 @@ 1342748960.468458 1.2.3.4 1234 2.3.4.5 80 failure UK 1342748960.468458 1.2.3.4 1234 2.3.4.5 80 success BR 1342748960.468458 1.2.3.4 1234 2.3.4.5 80 failure MX -#end 2012-07-20-01-49-20 +#close 2012-07-20-01-49-20 diff --git a/testing/btest/Baseline/scripts.base.frameworks.logging.exclude/ssh.log b/testing/btest/Baseline/scripts.base.frameworks.logging.exclude/ssh.log index 3fe01ff913..f795159a16 100644 --- a/testing/btest/Baseline/scripts.base.frameworks.logging.exclude/ssh.log +++ b/testing/btest/Baseline/scripts.base.frameworks.logging.exclude/ssh.log @@ -3,7 +3,7 @@ #empty_field (empty) #unset_field - #path ssh -#start 2012-07-20-01-49-20 +#open 2012-07-20-01-49-20 #fields id.orig_p id.resp_h id.resp_p status country #types port addr port string string 1234 2.3.4.5 80 success unknown @@ -11,4 +11,4 @@ 1234 2.3.4.5 80 failure UK 1234 2.3.4.5 80 success BR 1234 2.3.4.5 80 failure MX -#end 2012-07-20-01-49-20 +#close 2012-07-20-01-49-20 diff --git a/testing/btest/Baseline/scripts.base.frameworks.logging.file/ssh.log b/testing/btest/Baseline/scripts.base.frameworks.logging.file/ssh.log index 205f37243f..34d5f28b82 100644 --- a/testing/btest/Baseline/scripts.base.frameworks.logging.file/ssh.log +++ b/testing/btest/Baseline/scripts.base.frameworks.logging.file/ssh.log @@ -3,8 +3,8 @@ #empty_field (empty) #unset_field - #path ssh -#start 2012-07-20-01-49-20 +#open 2012-07-20-01-49-20 #fields t f #types time file 1342748960.757056 Foo.log -#end 2012-07-20-01-49-20 +#close 2012-07-20-01-49-20 diff --git a/testing/btest/Baseline/scripts.base.frameworks.logging.include/ssh.log b/testing/btest/Baseline/scripts.base.frameworks.logging.include/ssh.log index cafacf9c4e..8935046687 100644 --- a/testing/btest/Baseline/scripts.base.frameworks.logging.include/ssh.log +++ b/testing/btest/Baseline/scripts.base.frameworks.logging.include/ssh.log @@ -3,7 +3,7 @@ #empty_field (empty) #unset_field - #path ssh -#start 2012-07-20-01-49-20 +#open 2012-07-20-01-49-20 #fields t id.orig_h #types time addr 1342748960.796093 1.2.3.4 @@ -11,4 +11,4 @@ 1342748960.796093 1.2.3.4 1342748960.796093 1.2.3.4 1342748960.796093 1.2.3.4 -#end 2012-07-20-01-49-20 +#close 2012-07-20-01-49-20 diff --git a/testing/btest/Baseline/scripts.base.frameworks.logging.path-func-column-demote/local.log b/testing/btest/Baseline/scripts.base.frameworks.logging.path-func-column-demote/local.log index 3240e9f824..819b7b9bc2 100644 --- a/testing/btest/Baseline/scripts.base.frameworks.logging.path-func-column-demote/local.log +++ b/testing/btest/Baseline/scripts.base.frameworks.logging.path-func-column-demote/local.log @@ -3,7 +3,7 @@ #empty_field (empty) #unset_field - #path local -#start 2011-03-18-19-06-13 +#open 2011-03-18-19-06-13 #fields ts id.orig_h #types time addr 1300475168.859163 141.142.220.118 @@ -36,4 +36,4 @@ 1300475168.902195 141.142.220.118 1300475168.894787 141.142.220.118 1300475168.901749 141.142.220.118 -#end 2011-03-18-19-06-13 +#close 2011-03-18-19-06-13 diff --git a/testing/btest/Baseline/scripts.base.frameworks.logging.path-func-column-demote/remote.log b/testing/btest/Baseline/scripts.base.frameworks.logging.path-func-column-demote/remote.log index 84980836c4..41f575ef63 100644 --- a/testing/btest/Baseline/scripts.base.frameworks.logging.path-func-column-demote/remote.log +++ b/testing/btest/Baseline/scripts.base.frameworks.logging.path-func-column-demote/remote.log @@ -3,11 +3,11 @@ #empty_field (empty) #unset_field - #path remote -#start 2011-03-18-19-06-13 +#open 2011-03-18-19-06-13 #fields ts id.orig_h #types time addr 1300475169.780331 173.192.163.128 1300475167.097012 fe80::217:f2ff:fed7:cf65 1300475171.675372 fe80::3074:17d5:2052:c324 1300475173.116749 fe80::3074:17d5:2052:c324 -#end 2011-03-18-19-06-13 +#close 2011-03-18-19-06-13 diff --git a/testing/btest/Baseline/scripts.base.frameworks.logging.path-func/output b/testing/btest/Baseline/scripts.base.frameworks.logging.path-func/output index 1c67ff52b6..c67a12e1d9 100644 --- a/testing/btest/Baseline/scripts.base.frameworks.logging.path-func/output +++ b/testing/btest/Baseline/scripts.base.frameworks.logging.path-func/output @@ -10,68 +10,68 @@ static-prefix-2-UK.log #empty_field (empty) #unset_field - #path static-prefix-0-BR -#start 2012-07-20-01-49-21 +#open 2012-07-20-01-49-21 #fields t id.orig_h id.orig_p id.resp_h id.resp_p status country #types time addr port addr port string string 1342748961.180156 1.2.3.4 1234 2.3.4.5 80 success BR -#end 2012-07-20-01-49-21 +#close 2012-07-20-01-49-21 #separator \x09 #set_separator , #empty_field (empty) #unset_field - #path static-prefix-0-MX3 -#start 2012-07-20-01-49-21 +#open 2012-07-20-01-49-21 #fields t id.orig_h id.orig_p id.resp_h id.resp_p status country #types time addr port addr port string string 1342748961.180156 1.2.3.4 1234 2.3.4.5 80 failure MX3 -#end 2012-07-20-01-49-21 +#close 2012-07-20-01-49-21 #separator \x09 #set_separator , #empty_field (empty) #unset_field - #path static-prefix-0-unknown -#start 2012-07-20-01-49-21 +#open 2012-07-20-01-49-21 #fields t id.orig_h id.orig_p id.resp_h id.resp_p status country #types time addr port addr port string string 1342748961.180156 1.2.3.4 1234 2.3.4.5 80 success unknown -#end 2012-07-20-01-49-21 +#close 2012-07-20-01-49-21 #separator \x09 #set_separator , #empty_field (empty) #unset_field - #path static-prefix-1-MX -#start 2012-07-20-01-49-21 +#open 2012-07-20-01-49-21 #fields t id.orig_h id.orig_p id.resp_h id.resp_p status country #types time addr port addr port string string 1342748961.180156 1.2.3.4 1234 2.3.4.5 80 failure MX -#end 2012-07-20-01-49-21 +#close 2012-07-20-01-49-21 #separator \x09 #set_separator , #empty_field (empty) #unset_field - #path static-prefix-1-US -#start 2012-07-20-01-49-21 +#open 2012-07-20-01-49-21 #fields t id.orig_h id.orig_p id.resp_h id.resp_p status country #types time addr port addr port string string 1342748961.180156 1.2.3.4 1234 2.3.4.5 80 failure US -#end 2012-07-20-01-49-21 +#close 2012-07-20-01-49-21 #separator \x09 #set_separator , #empty_field (empty) #unset_field - #path static-prefix-2-MX2 -#start 2012-07-20-01-49-21 +#open 2012-07-20-01-49-21 #fields t id.orig_h id.orig_p id.resp_h id.resp_p status country #types time addr port addr port string string 1342748961.180156 1.2.3.4 1234 2.3.4.5 80 failure MX2 -#end 2012-07-20-01-49-21 +#close 2012-07-20-01-49-21 #separator \x09 #set_separator , #empty_field (empty) #unset_field - #path static-prefix-2-UK -#start 2012-07-20-01-49-21 +#open 2012-07-20-01-49-21 #fields t id.orig_h id.orig_p id.resp_h id.resp_p status country #types time addr port addr port string string 1342748961.180156 1.2.3.4 1234 2.3.4.5 80 failure UK -#end 2012-07-20-01-49-21 +#close 2012-07-20-01-49-21 diff --git a/testing/btest/Baseline/scripts.base.frameworks.logging.pred/test.failure.log b/testing/btest/Baseline/scripts.base.frameworks.logging.pred/test.failure.log index 96dede8965..a362135318 100644 --- a/testing/btest/Baseline/scripts.base.frameworks.logging.pred/test.failure.log +++ b/testing/btest/Baseline/scripts.base.frameworks.logging.pred/test.failure.log @@ -3,8 +3,8 @@ #empty_field (empty) #unset_field - #path test.failure -#start 2012-07-20-01-49-21 +#open 2012-07-20-01-49-21 #fields t id.orig_h id.orig_p id.resp_h id.resp_p status country #types time addr port addr port string string 1342748961.488370 1.2.3.4 1234 2.3.4.5 80 failure US -#end 2012-07-20-01-49-21 +#close 2012-07-20-01-49-21 diff --git a/testing/btest/Baseline/scripts.base.frameworks.logging.pred/test.success.log b/testing/btest/Baseline/scripts.base.frameworks.logging.pred/test.success.log index 85b5ca9f45..dd9c300429 100644 --- a/testing/btest/Baseline/scripts.base.frameworks.logging.pred/test.success.log +++ b/testing/btest/Baseline/scripts.base.frameworks.logging.pred/test.success.log @@ -3,8 +3,8 @@ #empty_field (empty) #unset_field - #path test.success -#start 2012-07-20-01-49-21 +#open 2012-07-20-01-49-21 #fields t id.orig_h id.orig_p id.resp_h id.resp_p status country #types time addr port addr port string string 1342748961.488370 1.2.3.4 1234 2.3.4.5 80 success unknown -#end 2012-07-20-01-49-21 +#close 2012-07-20-01-49-21 diff --git a/testing/btest/Baseline/scripts.base.frameworks.logging.remote-types/receiver.test.log b/testing/btest/Baseline/scripts.base.frameworks.logging.remote-types/receiver.test.log index aa18822daf..13364f8e77 100644 --- a/testing/btest/Baseline/scripts.base.frameworks.logging.remote-types/receiver.test.log +++ b/testing/btest/Baseline/scripts.base.frameworks.logging.remote-types/receiver.test.log @@ -3,8 +3,8 @@ #empty_field EMPTY #unset_field - #path test -#start 1970-01-01-00-00-00 +#open 1970-01-01-00-00-00 #fields b i e c p sn a d t iv s sc ss se vc ve #types bool int enum count port subnet addr double time interval string table[count] table[string] table[string] vector[count] vector[string] T -42 Test::LOG 21 123 10.0.0.0/24 1.2.3.4 3.14 1342749004.579242 100.000000 hurz 2,4,1,3 CC,AA,BB EMPTY 10,20,30 EMPTY -#end 2012-07-20-01-50-05 +#close 2012-07-20-01-50-05 diff --git a/testing/btest/Baseline/scripts.base.frameworks.logging.remote/sender.test.failure.log b/testing/btest/Baseline/scripts.base.frameworks.logging.remote/sender.test.failure.log index 36b88e496d..71e1d18c73 100644 --- a/testing/btest/Baseline/scripts.base.frameworks.logging.remote/sender.test.failure.log +++ b/testing/btest/Baseline/scripts.base.frameworks.logging.remote/sender.test.failure.log @@ -3,10 +3,10 @@ #empty_field (empty) #unset_field - #path test.failure -#start 2012-07-20-01-50-18 +#open 2012-07-20-01-50-18 #fields t id.orig_h id.orig_p id.resp_h id.resp_p status country #types time addr port addr port string string 1342749018.970682 1.2.3.4 1234 2.3.4.5 80 failure US 1342749018.970682 1.2.3.4 1234 2.3.4.5 80 failure UK 1342749018.970682 1.2.3.4 1234 2.3.4.5 80 failure MX -#end 2012-07-20-01-50-18 +#close 2012-07-20-01-50-18 diff --git a/testing/btest/Baseline/scripts.base.frameworks.logging.remote/sender.test.log b/testing/btest/Baseline/scripts.base.frameworks.logging.remote/sender.test.log index 22d354fce4..bc3dac5a1a 100644 --- a/testing/btest/Baseline/scripts.base.frameworks.logging.remote/sender.test.log +++ b/testing/btest/Baseline/scripts.base.frameworks.logging.remote/sender.test.log @@ -3,7 +3,7 @@ #empty_field (empty) #unset_field - #path test -#start 2012-07-20-01-50-18 +#open 2012-07-20-01-50-18 #fields t id.orig_h id.orig_p id.resp_h id.resp_p status country #types time addr port addr port string string 1342749018.970682 1.2.3.4 1234 2.3.4.5 80 success unknown @@ -11,4 +11,4 @@ 1342749018.970682 1.2.3.4 1234 2.3.4.5 80 failure UK 1342749018.970682 1.2.3.4 1234 2.3.4.5 80 success BR 1342749018.970682 1.2.3.4 1234 2.3.4.5 80 failure MX -#end 2012-07-20-01-50-18 +#close 2012-07-20-01-50-18 diff --git a/testing/btest/Baseline/scripts.base.frameworks.logging.remote/sender.test.success.log b/testing/btest/Baseline/scripts.base.frameworks.logging.remote/sender.test.success.log index 888dc424b5..f0b26454b4 100644 --- a/testing/btest/Baseline/scripts.base.frameworks.logging.remote/sender.test.success.log +++ b/testing/btest/Baseline/scripts.base.frameworks.logging.remote/sender.test.success.log @@ -3,9 +3,9 @@ #empty_field (empty) #unset_field - #path test.success -#start 2012-07-20-01-50-18 +#open 2012-07-20-01-50-18 #fields t id.orig_h id.orig_p id.resp_h id.resp_p status country #types time addr port addr port string string 1342749018.970682 1.2.3.4 1234 2.3.4.5 80 success unknown 1342749018.970682 1.2.3.4 1234 2.3.4.5 80 success BR -#end 2012-07-20-01-50-18 +#close 2012-07-20-01-50-18 diff --git a/testing/btest/Baseline/scripts.base.frameworks.logging.remove/ssh.failure.log b/testing/btest/Baseline/scripts.base.frameworks.logging.remove/ssh.failure.log index 5a23ad2066..de324c337f 100644 --- a/testing/btest/Baseline/scripts.base.frameworks.logging.remove/ssh.failure.log +++ b/testing/btest/Baseline/scripts.base.frameworks.logging.remove/ssh.failure.log @@ -3,9 +3,9 @@ #empty_field (empty) #unset_field - #path ssh.failure -#start 2012-07-20-01-49-21 +#open 2012-07-20-01-49-21 #fields t id.orig_h id.orig_p id.resp_h id.resp_p status country #types time addr port addr port string string 1342748961.521536 1.2.3.4 1234 2.3.4.5 80 failure US 1342748961.521536 1.2.3.4 1234 2.3.4.5 80 failure UK -#end 2012-07-20-01-49-21 +#close 2012-07-20-01-49-21 diff --git a/testing/btest/Baseline/scripts.base.frameworks.logging.remove/ssh.log b/testing/btest/Baseline/scripts.base.frameworks.logging.remove/ssh.log index cea1069748..ed0a118cac 100644 --- a/testing/btest/Baseline/scripts.base.frameworks.logging.remove/ssh.log +++ b/testing/btest/Baseline/scripts.base.frameworks.logging.remove/ssh.log @@ -3,10 +3,10 @@ #empty_field (empty) #unset_field - #path ssh -#start 2012-07-20-01-49-21 +#open 2012-07-20-01-49-21 #fields t id.orig_h id.orig_p id.resp_h id.resp_p status country #types time addr port addr port string string 1342748961.521536 1.2.3.4 1234 2.3.4.5 80 failure US 1342748961.521536 1.2.3.4 1234 2.3.4.5 80 failure UK 1342748961.521536 1.2.3.4 1234 2.3.4.5 80 failure BR -#end 2012-07-20-01-49-21 +#close 2012-07-20-01-49-21 diff --git a/testing/btest/Baseline/scripts.base.frameworks.logging.rotate-custom/.stderr b/testing/btest/Baseline/scripts.base.frameworks.logging.rotate-custom/.stderr index e1958d67ad..e69de29bb2 100644 --- a/testing/btest/Baseline/scripts.base.frameworks.logging.rotate-custom/.stderr +++ b/testing/btest/Baseline/scripts.base.frameworks.logging.rotate-custom/.stderr @@ -1,10 +0,0 @@ -1st test.2011-03-07-03-00-05.log test 11-03-07_03.00.05 11-03-07_04.00.05 0 ascii -1st test.2011-03-07-04-00-05.log test 11-03-07_04.00.05 11-03-07_05.00.05 0 ascii -1st test.2011-03-07-05-00-05.log test 11-03-07_05.00.05 11-03-07_06.00.05 0 ascii -1st test.2011-03-07-06-00-05.log test 11-03-07_06.00.05 11-03-07_07.00.05 0 ascii -1st test.2011-03-07-07-00-05.log test 11-03-07_07.00.05 11-03-07_08.00.05 0 ascii -1st test.2011-03-07-08-00-05.log test 11-03-07_08.00.05 11-03-07_09.00.05 0 ascii -1st test.2011-03-07-09-00-05.log test 11-03-07_09.00.05 11-03-07_10.00.05 0 ascii -1st test.2011-03-07-10-00-05.log test 11-03-07_10.00.05 11-03-07_11.00.05 0 ascii -1st test.2011-03-07-11-00-05.log test 11-03-07_11.00.05 11-03-07_12.00.05 0 ascii -1st test.2011-03-07-12-00-05.log test 11-03-07_12.00.05 11-03-07_12.59.55 1 ascii diff --git a/testing/btest/Baseline/scripts.base.frameworks.logging.rotate-custom/out b/testing/btest/Baseline/scripts.base.frameworks.logging.rotate-custom/out index 91b6f5de7a..3acce6f1ce 100644 --- a/testing/btest/Baseline/scripts.base.frameworks.logging.rotate-custom/out +++ b/testing/btest/Baseline/scripts.base.frameworks.logging.rotate-custom/out @@ -1,3 +1,13 @@ +1st test.2011-03-07-03-00-05.log test 11-03-07_03.00.05 11-03-07_04.00.05 0 ascii +1st test.2011-03-07-04-00-05.log test 11-03-07_04.00.05 11-03-07_05.00.05 0 ascii +1st test.2011-03-07-05-00-05.log test 11-03-07_05.00.05 11-03-07_06.00.05 0 ascii +1st test.2011-03-07-06-00-05.log test 11-03-07_06.00.05 11-03-07_07.00.05 0 ascii +1st test.2011-03-07-07-00-05.log test 11-03-07_07.00.05 11-03-07_08.00.05 0 ascii +1st test.2011-03-07-08-00-05.log test 11-03-07_08.00.05 11-03-07_09.00.05 0 ascii +1st test.2011-03-07-09-00-05.log test 11-03-07_09.00.05 11-03-07_10.00.05 0 ascii +1st test.2011-03-07-10-00-05.log test 11-03-07_10.00.05 11-03-07_11.00.05 0 ascii +1st test.2011-03-07-11-00-05.log test 11-03-07_11.00.05 11-03-07_12.00.05 0 ascii +1st test.2011-03-07-12-00-05.log test 11-03-07_12.00.05 11-03-07_12.59.55 1 ascii custom rotate, [writer=Log::WRITER_ASCII, fname=test2-11-03-07_03.00.05.log, path=test2, open=1299466805.0, close=1299470395.0, terminating=F] custom rotate, [writer=Log::WRITER_ASCII, fname=test2-11-03-07_03.59.55.log, path=test2, open=1299470395.0, close=1299470405.0, terminating=F] custom rotate, [writer=Log::WRITER_ASCII, fname=test2-11-03-07_04.00.05.log, path=test2, open=1299470405.0, close=1299473995.0, terminating=F] @@ -18,32 +28,14 @@ custom rotate, [writer=Log::WRITER_ASCII, fname=test2-11-03-07_11.00.05.log, pat custom rotate, [writer=Log::WRITER_ASCII, fname=test2-11-03-07_11.59.55.log, path=test2, open=1299499195.0, close=1299499205.0, terminating=F] custom rotate, [writer=Log::WRITER_ASCII, fname=test2-11-03-07_12.00.05.log, path=test2, open=1299499205.0, close=1299502795.0, terminating=F] custom rotate, [writer=Log::WRITER_ASCII, fname=test2-11-03-07_12.59.55.log, path=test2, open=1299502795.0, close=1299502795.0, terminating=T] +#close 2012-07-27-19-14-39 #empty_field (empty) -#end 2011-03-07-03-59-55 -#end 2011-03-07-04-00-05 -#end 2011-03-07-04-59-55 -#end 2011-03-07-05-00-05 -#end 2011-03-07-05-59-55 -#end 2011-03-07-06-00-05 -#end 2011-03-07-06-59-55 -#end 2011-03-07-07-00-05 -#end 2011-03-07-07-59-55 -#end 2011-03-07-08-00-05 -#end 2011-03-07-08-59-55 -#end 2011-03-07-09-00-05 -#end 2011-03-07-09-59-55 -#end 2011-03-07-10-00-05 -#end 2011-03-07-10-59-55 -#end 2011-03-07-11-00-05 -#end 2011-03-07-11-59-55 -#end 2011-03-07-12-00-05 -#end 2011-03-07-12-59-55 #fields t id.orig_h id.orig_p id.resp_h id.resp_p +#open 2012-07-27-19-14-39 #path test #path test2 #separator \x09 #set_separator , -#start 2011-03-07-03-00-05 #types time addr port addr port #unset_field - 1299466805.000000 10.0.0.1 20 10.0.0.2 1024 diff --git a/testing/btest/Baseline/scripts.base.frameworks.logging.rotate/out b/testing/btest/Baseline/scripts.base.frameworks.logging.rotate/out index 4764ff23d0..b26d2fcd1b 100644 --- a/testing/btest/Baseline/scripts.base.frameworks.logging.rotate/out +++ b/testing/btest/Baseline/scripts.base.frameworks.logging.rotate/out @@ -14,117 +14,117 @@ test.2011-03-07-12-00-05.log test 11-03-07_12.00.05 11-03-07_12.59.55 1 ascii #empty_field (empty) #unset_field - #path test -#start 2011-03-07-03-00-05 +#open 2011-03-07-03-00-05 #fields t id.orig_h id.orig_p id.resp_h id.resp_p #types time addr port addr port 1299466805.000000 10.0.0.1 20 10.0.0.2 1024 1299470395.000000 10.0.0.2 20 10.0.0.3 0 -#end 2011-03-07-04-00-05 +#close 2011-03-07-04-00-05 > test.2011-03-07-04-00-05.log #separator \x09 #set_separator , #empty_field (empty) #unset_field - #path test -#start 2011-03-07-03-00-05 +#open 2011-03-07-03-00-05 #fields t id.orig_h id.orig_p id.resp_h id.resp_p #types time addr port addr port 1299470405.000000 10.0.0.1 20 10.0.0.2 1025 1299473995.000000 10.0.0.2 20 10.0.0.3 1 -#end 2011-03-07-05-00-05 +#close 2011-03-07-05-00-05 > test.2011-03-07-05-00-05.log #separator \x09 #set_separator , #empty_field (empty) #unset_field - #path test -#start 2011-03-07-03-00-05 +#open 2011-03-07-03-00-05 #fields t id.orig_h id.orig_p id.resp_h id.resp_p #types time addr port addr port 1299474005.000000 10.0.0.1 20 10.0.0.2 1026 1299477595.000000 10.0.0.2 20 10.0.0.3 2 -#end 2011-03-07-06-00-05 +#close 2011-03-07-06-00-05 > test.2011-03-07-06-00-05.log #separator \x09 #set_separator , #empty_field (empty) #unset_field - #path test -#start 2011-03-07-03-00-05 +#open 2011-03-07-03-00-05 #fields t id.orig_h id.orig_p id.resp_h id.resp_p #types time addr port addr port 1299477605.000000 10.0.0.1 20 10.0.0.2 1027 1299481195.000000 10.0.0.2 20 10.0.0.3 3 -#end 2011-03-07-07-00-05 +#close 2011-03-07-07-00-05 > test.2011-03-07-07-00-05.log #separator \x09 #set_separator , #empty_field (empty) #unset_field - #path test -#start 2011-03-07-03-00-05 +#open 2011-03-07-03-00-05 #fields t id.orig_h id.orig_p id.resp_h id.resp_p #types time addr port addr port 1299481205.000000 10.0.0.1 20 10.0.0.2 1028 1299484795.000000 10.0.0.2 20 10.0.0.3 4 -#end 2011-03-07-08-00-05 +#close 2011-03-07-08-00-05 > test.2011-03-07-08-00-05.log #separator \x09 #set_separator , #empty_field (empty) #unset_field - #path test -#start 2011-03-07-03-00-05 +#open 2011-03-07-03-00-05 #fields t id.orig_h id.orig_p id.resp_h id.resp_p #types time addr port addr port 1299484805.000000 10.0.0.1 20 10.0.0.2 1029 1299488395.000000 10.0.0.2 20 10.0.0.3 5 -#end 2011-03-07-09-00-05 +#close 2011-03-07-09-00-05 > test.2011-03-07-09-00-05.log #separator \x09 #set_separator , #empty_field (empty) #unset_field - #path test -#start 2011-03-07-03-00-05 +#open 2011-03-07-03-00-05 #fields t id.orig_h id.orig_p id.resp_h id.resp_p #types time addr port addr port 1299488405.000000 10.0.0.1 20 10.0.0.2 1030 1299491995.000000 10.0.0.2 20 10.0.0.3 6 -#end 2011-03-07-10-00-05 +#close 2011-03-07-10-00-05 > test.2011-03-07-10-00-05.log #separator \x09 #set_separator , #empty_field (empty) #unset_field - #path test -#start 2011-03-07-03-00-05 +#open 2011-03-07-03-00-05 #fields t id.orig_h id.orig_p id.resp_h id.resp_p #types time addr port addr port 1299492005.000000 10.0.0.1 20 10.0.0.2 1031 1299495595.000000 10.0.0.2 20 10.0.0.3 7 -#end 2011-03-07-11-00-05 +#close 2011-03-07-11-00-05 > test.2011-03-07-11-00-05.log #separator \x09 #set_separator , #empty_field (empty) #unset_field - #path test -#start 2011-03-07-03-00-05 +#open 2011-03-07-03-00-05 #fields t id.orig_h id.orig_p id.resp_h id.resp_p #types time addr port addr port 1299495605.000000 10.0.0.1 20 10.0.0.2 1032 1299499195.000000 10.0.0.2 20 10.0.0.3 8 -#end 2011-03-07-12-00-05 +#close 2011-03-07-12-00-05 > test.2011-03-07-12-00-05.log #separator \x09 #set_separator , #empty_field (empty) #unset_field - #path test -#start 2011-03-07-03-00-05 +#open 2011-03-07-03-00-05 #fields t id.orig_h id.orig_p id.resp_h id.resp_p #types time addr port addr port 1299499205.000000 10.0.0.1 20 10.0.0.2 1033 1299502795.000000 10.0.0.2 20 10.0.0.3 9 -#end 2011-03-07-12-59-55 +#close 2011-03-07-12-59-55 diff --git a/testing/btest/Baseline/scripts.base.frameworks.logging.stdout/output b/testing/btest/Baseline/scripts.base.frameworks.logging.stdout/output index 110cef054a..6ff5237afa 100644 --- a/testing/btest/Baseline/scripts.base.frameworks.logging.stdout/output +++ b/testing/btest/Baseline/scripts.base.frameworks.logging.stdout/output @@ -3,7 +3,7 @@ #empty_field (empty) #unset_field - #path /dev/stdout -#start 2012-07-20-01-49-21 +#open 2012-07-20-01-49-21 #fields t id.orig_h id.orig_p id.resp_h id.resp_p status country #types time addr port addr port string string 1342748961.732599 1.2.3.4 1234 2.3.4.5 80 success unknown @@ -11,4 +11,4 @@ 1342748961.732599 1.2.3.4 1234 2.3.4.5 80 failure UK 1342748961.732599 1.2.3.4 1234 2.3.4.5 80 success BR 1342748961.732599 1.2.3.4 1234 2.3.4.5 80 failure MX -#end 2012-07-20-01-49-21 +#close 2012-07-20-01-49-21 diff --git a/testing/btest/Baseline/scripts.base.frameworks.logging.test-logging/ssh.log b/testing/btest/Baseline/scripts.base.frameworks.logging.test-logging/ssh.log index c9191b666e..d2d484e02f 100644 --- a/testing/btest/Baseline/scripts.base.frameworks.logging.test-logging/ssh.log +++ b/testing/btest/Baseline/scripts.base.frameworks.logging.test-logging/ssh.log @@ -3,7 +3,7 @@ #empty_field (empty) #unset_field - #path ssh -#start 2012-07-20-01-49-21 +#open 2012-07-20-01-49-21 #fields t id.orig_h id.orig_p id.resp_h id.resp_p status country #types time addr port addr port string string 1342748961.748481 1.2.3.4 1234 2.3.4.5 80 success unknown @@ -11,4 +11,4 @@ 1342748961.748481 1.2.3.4 1234 2.3.4.5 80 failure UK 1342748961.748481 1.2.3.4 1234 2.3.4.5 80 success BR 1342748961.748481 1.2.3.4 1234 2.3.4.5 80 failure MX -#end 2012-07-20-01-49-21 +#close 2012-07-20-01-49-21 diff --git a/testing/btest/Baseline/scripts.base.frameworks.logging.types/ssh.log b/testing/btest/Baseline/scripts.base.frameworks.logging.types/ssh.log index 1fc29dbb4e..6b75d056cf 100644 --- a/testing/btest/Baseline/scripts.base.frameworks.logging.types/ssh.log +++ b/testing/btest/Baseline/scripts.base.frameworks.logging.types/ssh.log @@ -3,8 +3,8 @@ #empty_field EMPTY #unset_field - #path ssh -#start 2012-07-20-01-49-22 +#open 2012-07-20-01-49-22 #fields b i e c p sn a d t iv s sc ss se vc ve f #types bool int enum count port subnet addr double time interval string table[count] table[string] table[string] vector[count] vector[string] func T -42 SSH::LOG 21 123 10.0.0.0/24 1.2.3.4 3.14 1342748962.114672 100.000000 hurz 2,4,1,3 CC,AA,BB EMPTY 10,20,30 EMPTY SSH::foo\x0a{ \x0aif (0 < SSH::i) \x0a\x09return (Foo);\x0aelse\x0a\x09return (Bar);\x0a\x0a} -#end 2012-07-20-01-49-22 +#close 2012-07-20-01-49-22 diff --git a/testing/btest/Baseline/scripts.base.frameworks.logging.unset-record/testing.log b/testing/btest/Baseline/scripts.base.frameworks.logging.unset-record/testing.log index b4089aeee8..0ebe8838ad 100644 --- a/testing/btest/Baseline/scripts.base.frameworks.logging.unset-record/testing.log +++ b/testing/btest/Baseline/scripts.base.frameworks.logging.unset-record/testing.log @@ -3,9 +3,9 @@ #empty_field (empty) #unset_field - #path testing -#start 2012-07-20-01-49-22 +#open 2012-07-20-01-49-22 #fields a.val1 a.val2 b #types count count count - - 6 1 2 3 -#end 2012-07-20-01-49-22 +#close 2012-07-20-01-49-22 diff --git a/testing/btest/Baseline/scripts.base.frameworks.logging.vec/ssh.log b/testing/btest/Baseline/scripts.base.frameworks.logging.vec/ssh.log index ae5d6d246e..3e8e1e737e 100644 --- a/testing/btest/Baseline/scripts.base.frameworks.logging.vec/ssh.log +++ b/testing/btest/Baseline/scripts.base.frameworks.logging.vec/ssh.log @@ -3,8 +3,8 @@ #empty_field (empty) #unset_field - #path ssh -#start 2012-07-20-01-49-22 +#open 2012-07-20-01-49-22 #fields vec #types vector[string] -,2,-,-,5 -#end 2012-07-20-01-49-22 +#close 2012-07-20-01-49-22 diff --git a/testing/btest/Baseline/scripts.base.frameworks.logging.writer-path-conflict/http-2-2.log b/testing/btest/Baseline/scripts.base.frameworks.logging.writer-path-conflict/http-2-2.log new file mode 100644 index 0000000000..cbc90d9926 --- /dev/null +++ b/testing/btest/Baseline/scripts.base.frameworks.logging.writer-path-conflict/http-2-2.log @@ -0,0 +1,23 @@ +#separator \x09 +#set_separator , +#empty_field (empty) +#unset_field - +#path http-2-2 +#open 2011-03-18-19-06-08 +#fields status_code +#types count +304 +304 +304 +304 +304 +304 +304 +304 +304 +304 +304 +304 +304 +304 +#close 2011-03-18-19-06-13 diff --git a/testing/btest/Baseline/scripts.base.frameworks.logging.writer-path-conflict/http-2.log b/testing/btest/Baseline/scripts.base.frameworks.logging.writer-path-conflict/http-2.log new file mode 100644 index 0000000000..8f66184146 --- /dev/null +++ b/testing/btest/Baseline/scripts.base.frameworks.logging.writer-path-conflict/http-2.log @@ -0,0 +1,23 @@ +#separator \x09 +#set_separator , +#empty_field (empty) +#unset_field - +#path http-2 +#open 2011-03-18-19-06-08 +#fields host +#types string +bits.wikimedia.org +upload.wikimedia.org +upload.wikimedia.org +upload.wikimedia.org +upload.wikimedia.org +upload.wikimedia.org +upload.wikimedia.org +meta.wikimedia.org +upload.wikimedia.org +upload.wikimedia.org +upload.wikimedia.org +upload.wikimedia.org +upload.wikimedia.org +upload.wikimedia.org +#close 2011-03-18-19-06-13 diff --git a/testing/btest/Baseline/scripts.base.frameworks.logging.writer-path-conflict/http-3.log b/testing/btest/Baseline/scripts.base.frameworks.logging.writer-path-conflict/http-3.log new file mode 100644 index 0000000000..d64b9aa128 --- /dev/null +++ b/testing/btest/Baseline/scripts.base.frameworks.logging.writer-path-conflict/http-3.log @@ -0,0 +1,23 @@ +#separator \x09 +#set_separator , +#empty_field (empty) +#unset_field - +#path http-3 +#open 2011-03-18-19-06-08 +#fields uri +#types string +/skins-1.5/monobook/main.css +/wikipedia/commons/6/63/Wikipedia-logo.png +/wikipedia/commons/thumb/b/bb/Wikipedia_wordmark.svg/174px-Wikipedia_wordmark.svg.png +/wikipedia/commons/b/bd/Bookshelf-40x201_6.png +/wikipedia/commons/thumb/8/8a/Wikinews-logo.png/35px-Wikinews-logo.png +/wikipedia/commons/4/4a/Wiktionary-logo-en-35px.png +/wikipedia/commons/thumb/f/fa/Wikiquote-logo.svg/35px-Wikiquote-logo.svg.png +/images/wikimedia-button.png +/wikipedia/commons/thumb/f/fa/Wikibooks-logo.svg/35px-Wikibooks-logo.svg.png +/wikipedia/commons/thumb/d/df/Wikispecies-logo.svg/35px-Wikispecies-logo.svg.png +/wikipedia/commons/thumb/4/4c/Wikisource-logo.svg/35px-Wikisource-logo.svg.png +/wikipedia/commons/thumb/4/4a/Commons-logo.svg/35px-Commons-logo.svg.png +/wikipedia/commons/thumb/9/91/Wikiversity-logo.svg/35px-Wikiversity-logo.svg.png +/wikipedia/commons/thumb/7/75/Wikimedia_Community_Logo.svg/35px-Wikimedia_Community_Logo.svg.png +#close 2011-03-18-19-06-13 diff --git a/testing/btest/Baseline/scripts.base.frameworks.logging.writer-path-conflict/http.log b/testing/btest/Baseline/scripts.base.frameworks.logging.writer-path-conflict/http.log index 9ac9b6304c..97273995bc 100644 --- a/testing/btest/Baseline/scripts.base.frameworks.logging.writer-path-conflict/http.log +++ b/testing/btest/Baseline/scripts.base.frameworks.logging.writer-path-conflict/http.log @@ -3,7 +3,7 @@ #empty_field (empty) #unset_field - #path http -#start 2011-03-18-19-06-08 +#open 2011-03-18-19-06-08 #fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p trans_depth method host uri referrer user_agent request_body_len response_body_len status_code status_msg info_code info_msg filename tags username password proxied mime_type md5 extraction_file #types time string addr port addr port count string string string string string count count count string count string string table[enum] string string table[string] string string file 1300475168.784020 j4u32Pc5bif 141.142.220.118 48649 208.80.152.118 80 1 GET bits.wikimedia.org /skins-1.5/monobook/main.css http://www.wikipedia.org/ Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.15) Gecko/20110303 Ubuntu/10.04 (lucid) Firefox/3.6.15 0 0 304 Not Modified - - - (empty) - - - - - - @@ -20,4 +20,4 @@ 1300475169.014619 Tw8jXtpTGu6 141.142.220.118 50000 208.80.152.3 80 2 GET upload.wikimedia.org /wikipedia/commons/thumb/4/4a/Commons-logo.svg/35px-Commons-logo.svg.png http://www.wikipedia.org/ Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.15) Gecko/20110303 Ubuntu/10.04 (lucid) Firefox/3.6.15 0 0 304 Not Modified - - - (empty) - - - - - - 1300475169.014593 P654jzLoe3a 141.142.220.118 49999 208.80.152.3 80 2 GET upload.wikimedia.org /wikipedia/commons/thumb/9/91/Wikiversity-logo.svg/35px-Wikiversity-logo.svg.png http://www.wikipedia.org/ Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.15) Gecko/20110303 Ubuntu/10.04 (lucid) Firefox/3.6.15 0 0 304 Not Modified - - - (empty) - - - - - - 1300475169.014927 0Q4FH8sESw5 141.142.220.118 50001 208.80.152.3 80 2 GET upload.wikimedia.org /wikipedia/commons/thumb/7/75/Wikimedia_Community_Logo.svg/35px-Wikimedia_Community_Logo.svg.png http://www.wikipedia.org/ Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.15) Gecko/20110303 Ubuntu/10.04 (lucid) Firefox/3.6.15 0 0 304 Not Modified - - - (empty) - - - - - - -#end 2011-03-18-19-06-13 +#close 2011-03-18-19-06-13 diff --git a/testing/btest/Baseline/scripts.base.frameworks.logging.writer-path-conflict/reporter.log b/testing/btest/Baseline/scripts.base.frameworks.logging.writer-path-conflict/reporter.log old mode 100755 new mode 100644 index 7a4225d718..35e9134583 --- a/testing/btest/Baseline/scripts.base.frameworks.logging.writer-path-conflict/reporter.log +++ b/testing/btest/Baseline/scripts.base.frameworks.logging.writer-path-conflict/reporter.log @@ -3,21 +3,10 @@ #empty_field (empty) #unset_field - #path reporter -#start 2011-03-18-19-06-08 +#open 2011-03-18-19-06-08 #fields ts level message location #types time enum string string -1300475168.843894 Reporter::WARNING Skipping write to filter 'host-only' on path 'http' because filter 'default' has already instantiated the same writer type for that path (empty) -1300475168.975800 Reporter::WARNING Skipping write to filter 'host-only' on path 'http' because filter 'default' has already instantiated the same writer type for that path (empty) -1300475168.976327 Reporter::WARNING Skipping write to filter 'host-only' on path 'http' because filter 'default' has already instantiated the same writer type for that path (empty) -1300475168.979160 Reporter::WARNING Skipping write to filter 'host-only' on path 'http' because filter 'default' has already instantiated the same writer type for that path (empty) -1300475169.012666 Reporter::WARNING Skipping write to filter 'host-only' on path 'http' because filter 'default' has already instantiated the same writer type for that path (empty) -1300475169.012730 Reporter::WARNING Skipping write to filter 'host-only' on path 'http' because filter 'default' has already instantiated the same writer type for that path (empty) -1300475169.014860 Reporter::WARNING Skipping write to filter 'host-only' on path 'http' because filter 'default' has already instantiated the same writer type for that path (empty) -1300475169.022665 Reporter::WARNING Skipping write to filter 'host-only' on path 'http' because filter 'default' has already instantiated the same writer type for that path (empty) -1300475169.036294 Reporter::WARNING Skipping write to filter 'host-only' on path 'http' because filter 'default' has already instantiated the same writer type for that path (empty) -1300475169.036798 Reporter::WARNING Skipping write to filter 'host-only' on path 'http' because filter 'default' has already instantiated the same writer type for that path (empty) -1300475169.039923 Reporter::WARNING Skipping write to filter 'host-only' on path 'http' because filter 'default' has already instantiated the same writer type for that path (empty) -1300475169.074793 Reporter::WARNING Skipping write to filter 'host-only' on path 'http' because filter 'default' has already instantiated the same writer type for that path (empty) -1300475169.074938 Reporter::WARNING Skipping write to filter 'host-only' on path 'http' because filter 'default' has already instantiated the same writer type for that path (empty) -1300475169.075065 Reporter::WARNING Skipping write to filter 'host-only' on path 'http' because filter 'default' has already instantiated the same writer type for that path (empty) -#end 2011-03-18-19-06-13 +1300475168.843894 Reporter::WARNING Write using filter 'host-only' on path 'http' changed to use new path 'http-2' to avoid conflict with filter 'default' (empty) +1300475168.843894 Reporter::WARNING Write using filter 'uri-only' on path 'http' changed to use new path 'http-3' to avoid conflict with filter 'default' (empty) +1300475168.843894 Reporter::WARNING Write using filter 'status-only' on path 'http-2' changed to use new path 'http-2-2' to avoid conflict with filter 'host-only' (empty) +#close 2011-03-18-19-06-13 diff --git a/testing/btest/Baseline/scripts.base.frameworks.metrics.basic-cluster/manager-1.metrics.log b/testing/btest/Baseline/scripts.base.frameworks.metrics.basic-cluster/manager-1.metrics.log index a3f476c1fb..cb1bd5af01 100644 --- a/testing/btest/Baseline/scripts.base.frameworks.metrics.basic-cluster/manager-1.metrics.log +++ b/testing/btest/Baseline/scripts.base.frameworks.metrics.basic-cluster/manager-1.metrics.log @@ -3,10 +3,10 @@ #empty_field (empty) #unset_field - #path metrics -#start 2012-07-20-01-50-41 +#open 2012-07-20-01-50-41 #fields ts metric_id filter_name index.host index.str index.network value #types time enum string addr string subnet count 1342749041.601712 TEST_METRIC foo-bar 6.5.4.3 - - 4 1342749041.601712 TEST_METRIC foo-bar 7.2.1.5 - - 2 1342749041.601712 TEST_METRIC foo-bar 1.2.3.4 - - 6 -#end 2012-07-20-01-50-49 +#close 2012-07-20-01-50-49 diff --git a/testing/btest/Baseline/scripts.base.frameworks.metrics.basic/metrics.log b/testing/btest/Baseline/scripts.base.frameworks.metrics.basic/metrics.log index b497da5194..fb6476ee88 100644 --- a/testing/btest/Baseline/scripts.base.frameworks.metrics.basic/metrics.log +++ b/testing/btest/Baseline/scripts.base.frameworks.metrics.basic/metrics.log @@ -3,10 +3,10 @@ #empty_field (empty) #unset_field - #path metrics -#start 2012-07-20-01-49-22 +#open 2012-07-20-01-49-22 #fields ts metric_id filter_name index.host index.str index.network value #types time enum string addr string subnet count 1342748962.841548 TEST_METRIC foo-bar 6.5.4.3 - - 2 1342748962.841548 TEST_METRIC foo-bar 7.2.1.5 - - 1 1342748962.841548 TEST_METRIC foo-bar 1.2.3.4 - - 3 -#end 2012-07-20-01-49-22 +#close 2012-07-20-01-49-22 diff --git a/testing/btest/Baseline/scripts.base.frameworks.metrics.cluster-intermediate-update/manager-1.notice.log b/testing/btest/Baseline/scripts.base.frameworks.metrics.cluster-intermediate-update/manager-1.notice.log index 8f3a9dc70c..217b3ed49b 100644 --- a/testing/btest/Baseline/scripts.base.frameworks.metrics.cluster-intermediate-update/manager-1.notice.log +++ b/testing/btest/Baseline/scripts.base.frameworks.metrics.cluster-intermediate-update/manager-1.notice.log @@ -3,8 +3,8 @@ #empty_field (empty) #unset_field - #path notice -#start 2012-07-20-01-50-59 +#open 2012-07-20-01-50-59 #fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p proto note msg sub src dst p n peer_descr actions policy_items suppress_for dropped remote_location.country_code remote_location.region remote_location.city remote_location.latitude remote_location.longitude metric_index.host metric_index.str metric_index.network #types time string addr port addr port enum enum string string addr addr port count string table[enum] table[count] interval bool string string string double double addr string subnet 1342749059.978651 - - - - - - Test_Notice Threshold crossed by metric_index(host=1.2.3.4) 100/100 - 1.2.3.4 - - 100 manager-1 Notice::ACTION_LOG 6 3600.000000 F - - - - - 1.2.3.4 - - -#end 2012-07-20-01-51-08 +#close 2012-07-20-01-51-08 diff --git a/testing/btest/Baseline/scripts.base.frameworks.metrics.notice/notice.log b/testing/btest/Baseline/scripts.base.frameworks.metrics.notice/notice.log index 5a214b26cc..ba6c680e27 100644 --- a/testing/btest/Baseline/scripts.base.frameworks.metrics.notice/notice.log +++ b/testing/btest/Baseline/scripts.base.frameworks.metrics.notice/notice.log @@ -3,9 +3,9 @@ #empty_field (empty) #unset_field - #path notice -#start 2012-07-20-01-49-23 +#open 2012-07-20-01-49-23 #fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p proto note msg sub src dst p n peer_descr actions policy_items suppress_for dropped remote_location.country_code remote_location.region remote_location.city remote_location.latitude remote_location.longitude metric_index.host metric_index.str metric_index.network #types time string addr port addr port enum enum string string addr addr port count string table[enum] table[count] interval bool string string string double double addr string subnet 1342748963.085888 - - - - - - Test_Notice Threshold crossed by metric_index(host=1.2.3.4) 3/2 - 1.2.3.4 - - 3 bro Notice::ACTION_LOG 6 3600.000000 F - - - - - 1.2.3.4 - - 1342748963.085888 - - - - - - Test_Notice Threshold crossed by metric_index(host=6.5.4.3) 2/2 - 6.5.4.3 - - 2 bro Notice::ACTION_LOG 6 3600.000000 F - - - - - 6.5.4.3 - - -#end 2012-07-20-01-49-23 +#close 2012-07-20-01-49-23 diff --git a/testing/btest/Baseline/scripts.base.frameworks.notice.cluster/manager-1.notice.log b/testing/btest/Baseline/scripts.base.frameworks.notice.cluster/manager-1.notice.log index 4903ec0c01..6c93cb875e 100644 --- a/testing/btest/Baseline/scripts.base.frameworks.notice.cluster/manager-1.notice.log +++ b/testing/btest/Baseline/scripts.base.frameworks.notice.cluster/manager-1.notice.log @@ -3,8 +3,8 @@ #empty_field (empty) #unset_field - #path notice -#start 2012-07-20-01-51-18 +#open 2012-07-20-01-51-18 #fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p proto note msg sub src dst p n peer_descr actions policy_items suppress_for dropped remote_location.country_code remote_location.region remote_location.city remote_location.latitude remote_location.longitude metric_index.host metric_index.str metric_index.network #types time string addr port addr port enum enum string string addr addr port count string table[enum] table[count] interval bool string string string double double addr string subnet 1342749078.270791 - - - - - - Test_Notice test notice! - - - - - worker-1 Notice::ACTION_LOG 6 3600.000000 F - - - - - - - - -#end 2012-07-20-01-51-27 +#close 2012-07-20-01-51-27 diff --git a/testing/btest/Baseline/scripts.base.frameworks.notice.suppression-cluster/manager-1.notice.log b/testing/btest/Baseline/scripts.base.frameworks.notice.suppression-cluster/manager-1.notice.log index bd77a90c86..88f25b066f 100644 --- a/testing/btest/Baseline/scripts.base.frameworks.notice.suppression-cluster/manager-1.notice.log +++ b/testing/btest/Baseline/scripts.base.frameworks.notice.suppression-cluster/manager-1.notice.log @@ -3,8 +3,8 @@ #empty_field (empty) #unset_field - #path notice -#start 2012-07-20-01-51-36 +#open 2012-07-20-01-51-36 #fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p proto note msg sub src dst p n peer_descr actions policy_items suppress_for dropped remote_location.country_code remote_location.region remote_location.city remote_location.latitude remote_location.longitude metric_index.host metric_index.str metric_index.network #types time string addr port addr port enum enum string string addr addr port count string table[enum] table[count] interval bool string string string double double addr string subnet 1342749096.545663 - - - - - - Test_Notice test notice! - - - - - worker-2 Notice::ACTION_LOG 6 3600.000000 F - - - - - - - - -#end 2012-07-20-01-51-45 +#close 2012-07-20-01-51-45 diff --git a/testing/btest/Baseline/scripts.base.frameworks.notice.suppression/notice.log b/testing/btest/Baseline/scripts.base.frameworks.notice.suppression/notice.log index 5a3cdfa69f..7c7254f87e 100644 --- a/testing/btest/Baseline/scripts.base.frameworks.notice.suppression/notice.log +++ b/testing/btest/Baseline/scripts.base.frameworks.notice.suppression/notice.log @@ -3,8 +3,8 @@ #empty_field (empty) #unset_field - #path notice -#start 2012-07-20-01-49-23 +#open 2012-07-20-01-49-23 #fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p proto note msg sub src dst p n peer_descr actions policy_items suppress_for dropped remote_location.country_code remote_location.region remote_location.city remote_location.latitude remote_location.longitude #types time string addr port addr port enum enum string string addr addr port count string table[enum] table[count] interval bool string string string double double 1342748963.685754 - - - - - - Test_Notice test - - - - - bro Notice::ACTION_LOG 6 3600.000000 F - - - - - -#end 2012-07-20-01-49-23 +#close 2012-07-20-01-49-23 diff --git a/testing/btest/Baseline/scripts.base.frameworks.reporter.disable-stderr/.stderr b/testing/btest/Baseline/scripts.base.frameworks.reporter.disable-stderr/.stderr new file mode 100644 index 0000000000..e69de29bb2 diff --git a/testing/btest/Baseline/scripts.base.frameworks.reporter.disable-stderr/reporter.log b/testing/btest/Baseline/scripts.base.frameworks.reporter.disable-stderr/reporter.log new file mode 100644 index 0000000000..144c094b2f --- /dev/null +++ b/testing/btest/Baseline/scripts.base.frameworks.reporter.disable-stderr/reporter.log @@ -0,0 +1,10 @@ +#separator \x09 +#set_separator , +#empty_field (empty) +#unset_field - +#path reporter +#open 2012-08-10-20-09-16 +#fields ts level message location +#types time enum string string +0.000000 Reporter::ERROR no such index (test[3]) /da/home/robin/bro/master/testing/btest/.tmp/scripts.base.frameworks.reporter.disable-stderr/disable-stderr.bro, line 12 +#close 2012-08-10-20-09-16 diff --git a/testing/btest/Baseline/scripts.base.frameworks.reporter.stderr/.stderr b/testing/btest/Baseline/scripts.base.frameworks.reporter.stderr/.stderr new file mode 100644 index 0000000000..78af1e7a73 --- /dev/null +++ b/testing/btest/Baseline/scripts.base.frameworks.reporter.stderr/.stderr @@ -0,0 +1 @@ +ERROR: no such index (test[3]) (/blah/testing/btest/.tmp/scripts.base.frameworks.reporter.stderr/stderr.bro, line 9) diff --git a/testing/btest/Baseline/scripts.base.frameworks.reporter.stderr/reporter.log b/testing/btest/Baseline/scripts.base.frameworks.reporter.stderr/reporter.log new file mode 100644 index 0000000000..b314bc45c3 --- /dev/null +++ b/testing/btest/Baseline/scripts.base.frameworks.reporter.stderr/reporter.log @@ -0,0 +1,10 @@ +#separator \x09 +#set_separator , +#empty_field (empty) +#unset_field - +#path reporter +#open 2012-08-10-20-09-23 +#fields ts level message location +#types time enum string string +0.000000 Reporter::ERROR no such index (test[3]) /da/home/robin/bro/master/testing/btest/.tmp/scripts.base.frameworks.reporter.stderr/stderr.bro, line 9 +#close 2012-08-10-20-09-23 diff --git a/testing/btest/Baseline/scripts.base.protocols.ftp.ftp-ipv4/conn.log b/testing/btest/Baseline/scripts.base.protocols.ftp.ftp-ipv4/conn.log index 316056fa8c..3520980833 100644 --- a/testing/btest/Baseline/scripts.base.protocols.ftp.ftp-ipv4/conn.log +++ b/testing/btest/Baseline/scripts.base.protocols.ftp.ftp-ipv4/conn.log @@ -3,7 +3,7 @@ #empty_field (empty) #unset_field - #path conn -#start 2012-02-21-16-53-13 +#open 2012-02-21-16-53-13 #fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p proto service duration orig_bytes resp_bytes conn_state local_orig missed_bytes history orig_pkts orig_ip_bytes resp_pkts resp_ip_bytes tunnel_parents #types time string addr port addr port enum string interval count count string bool count string count count count count table[string] 1329843175.736107 arKYeMETxOg 141.142.220.235 37604 199.233.217.249 56666 tcp ftp-data 0.112432 0 342 SF - 0 ShAdfFa 4 216 4 562 (empty) @@ -11,4 +11,4 @@ 1329843194.151526 nQcgTWjvg4c 199.233.217.249 61920 141.142.220.235 33582 tcp ftp-data 0.056211 342 0 SF - 0 ShADaFf 5 614 3 164 (empty) 1329843197.783443 j4u32Pc5bif 199.233.217.249 61918 141.142.220.235 37835 tcp ftp-data 0.056005 77 0 SF - 0 ShADaFf 5 349 3 164 (empty) 1329843161.968492 UWkUyAuUGXf 141.142.220.235 50003 199.233.217.249 21 tcp ftp 38.055625 180 3146 SF - 0 ShAdDfFa 38 2164 25 4458 (empty) -#end 2012-02-21-16-53-20 +#close 2012-02-21-16-53-20 diff --git a/testing/btest/Baseline/scripts.base.protocols.ftp.ftp-ipv4/ftp.log b/testing/btest/Baseline/scripts.base.protocols.ftp.ftp-ipv4/ftp.log index cee57182ed..0d0a8f57f1 100644 --- a/testing/btest/Baseline/scripts.base.protocols.ftp.ftp-ipv4/ftp.log +++ b/testing/btest/Baseline/scripts.base.protocols.ftp.ftp-ipv4/ftp.log @@ -3,9 +3,9 @@ #empty_field (empty) #unset_field - #path ftp -#start 2012-02-21-16-53-13 +#open 2012-02-21-16-53-13 #fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p user password command arg mime_type mime_desc file_size reply_code reply_msg tags extraction_file #types time string addr port addr port string string string string string string count count string table[string] file 1329843179.926563 UWkUyAuUGXf 141.142.220.235 50003 199.233.217.249 21 anonymous test RETR ftp://199.233.217.249/./robots.txt text/plain ASCII text 77 226 Transfer complete. - - 1329843197.727769 UWkUyAuUGXf 141.142.220.235 50003 199.233.217.249 21 anonymous test RETR ftp://199.233.217.249/./robots.txt text/plain ASCII text, with CRLF line terminators 77 226 Transfer complete. - - -#end 2012-02-21-16-53-20 +#close 2012-02-21-16-53-20 diff --git a/testing/btest/Baseline/scripts.base.protocols.ftp.ftp-ipv6/conn.log b/testing/btest/Baseline/scripts.base.protocols.ftp.ftp-ipv6/conn.log index 299bdbc4ba..3d81f45670 100644 --- a/testing/btest/Baseline/scripts.base.protocols.ftp.ftp-ipv6/conn.log +++ b/testing/btest/Baseline/scripts.base.protocols.ftp.ftp-ipv6/conn.log @@ -3,7 +3,7 @@ #empty_field (empty) #unset_field - #path conn -#start 2012-02-15-17-43-15 +#open 2012-02-15-17-43-15 #fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p proto service duration orig_bytes resp_bytes conn_state local_orig missed_bytes history orig_pkts orig_ip_bytes resp_pkts resp_ip_bytes tunnel_parents #types time string addr port addr port enum string interval count count string bool count string count count count count table[string] 1329327783.316897 arKYeMETxOg 2001:470:1f11:81f:c999:d94:aa7c:2e3e 49186 2001:470:4867:99::21 57086 tcp ftp-data 0.219721 0 342 SF - 0 ShAdfFa 5 372 4 642 (empty) @@ -12,4 +12,4 @@ 1329327795.571921 j4u32Pc5bif 2001:470:4867:99::21 55785 2001:470:1f11:81f:c999:d94:aa7c:2e3e 49189 tcp ftp-data 0.109813 77 0 SF - 0 ShADFaf 5 449 4 300 (empty) 1329327777.822004 UWkUyAuUGXf 2001:470:1f11:81f:c999:d94:aa7c:2e3e 49185 2001:470:4867:99::21 21 tcp ftp 26.658219 310 3448 SF - 0 ShAdDfFa 57 4426 34 5908 (empty) 1329327800.017649 TEfuqmmG4bh 2001:470:4867:99::21 55647 2001:470:1f11:81f:c999:d94:aa7c:2e3e 49190 tcp ftp-data 0.109181 342 0 SF - 0 ShADFaf 5 714 4 300 (empty) -#end 2012-02-15-17-43-24 +#close 2012-02-15-17-43-24 diff --git a/testing/btest/Baseline/scripts.base.protocols.ftp.ftp-ipv6/ftp.log b/testing/btest/Baseline/scripts.base.protocols.ftp.ftp-ipv6/ftp.log index 096b91df65..62ea4df18d 100644 --- a/testing/btest/Baseline/scripts.base.protocols.ftp.ftp-ipv6/ftp.log +++ b/testing/btest/Baseline/scripts.base.protocols.ftp.ftp-ipv6/ftp.log @@ -3,9 +3,9 @@ #empty_field (empty) #unset_field - #path ftp -#start 2012-02-15-17-43-07 +#open 2012-02-15-17-43-07 #fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p user password command arg mime_type mime_desc file_size reply_code reply_msg tags extraction_file #types time string addr port addr port string string string string string string count count string table[string] file 1329327787.396984 UWkUyAuUGXf 2001:470:1f11:81f:c999:d94:aa7c:2e3e 49185 2001:470:4867:99::21 21 anonymous test RETR ftp://[2001:470:4867:99::21]/robots.txt - - 77 226 Transfer complete. - - 1329327795.463946 UWkUyAuUGXf 2001:470:1f11:81f:c999:d94:aa7c:2e3e 49185 2001:470:4867:99::21 21 anonymous test RETR ftp://[2001:470:4867:99::21]/robots.txt - - 77 226 Transfer complete. - - -#end 2012-02-15-17-43-24 +#close 2012-02-15-17-43-24 diff --git a/testing/btest/Baseline/scripts.base.protocols.http.100-continue/http.log b/testing/btest/Baseline/scripts.base.protocols.http.100-continue/http.log index c457f9b64b..13c8b12502 100644 --- a/testing/btest/Baseline/scripts.base.protocols.http.100-continue/http.log +++ b/testing/btest/Baseline/scripts.base.protocols.http.100-continue/http.log @@ -3,8 +3,8 @@ #empty_field (empty) #unset_field - #path http -#start 2009-03-19-05-21-36 +#open 2009-03-19-05-21-36 #fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p trans_depth method host uri referrer user_agent request_body_len response_body_len status_code status_msg info_code info_msg filename tags username password proxied mime_type md5 extraction_file #types time string addr port addr port count string string string string string count count count string count string string table[enum] string string table[string] string string file 1237440095.634312 UWkUyAuUGXf 192.168.3.103 54102 128.146.216.51 80 1 POST www.osu.edu / - curl/7.17.1 (i386-apple-darwin8.11.1) libcurl/7.17.1 zlib/1.2.3 2001 60731 200 OK 100 Continue - (empty) - - - text/html - - -#end 2009-03-19-05-21-36 +#close 2009-03-19-05-21-36 diff --git a/testing/btest/Baseline/scripts.base.protocols.http.http-extract-files/http.log b/testing/btest/Baseline/scripts.base.protocols.http.http-extract-files/http.log index 46ae431fc2..0d61a6c8b3 100644 --- a/testing/btest/Baseline/scripts.base.protocols.http.http-extract-files/http.log +++ b/testing/btest/Baseline/scripts.base.protocols.http.http-extract-files/http.log @@ -3,8 +3,8 @@ #empty_field (empty) #unset_field - #path http -#start 2005-10-07-23-23-56 +#open 2005-10-07-23-23-56 #fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p trans_depth method host uri referrer user_agent request_body_len response_body_len status_code status_msg info_code info_msg filename tags username password proxied mime_type md5 extraction_file #types time string addr port addr port count string string string string string count count count string count string string table[enum] string string table[string] string string file 1128727435.634189 arKYeMETxOg 141.42.64.125 56730 125.190.109.199 80 1 GET www.icir.org / - Wget/1.10 0 9130 200 OK - - - (empty) - - - text/html - http-item_141.42.64.125:56730-125.190.109.199:80_resp_1.dat -#end 2005-10-07-23-23-57 +#close 2005-10-07-23-23-57 diff --git a/testing/btest/Baseline/scripts.base.protocols.http.http-mime-and-md5/http.log b/testing/btest/Baseline/scripts.base.protocols.http.http-mime-and-md5/http.log index 69e6613a3c..409d8fc812 100644 --- a/testing/btest/Baseline/scripts.base.protocols.http.http-mime-and-md5/http.log +++ b/testing/btest/Baseline/scripts.base.protocols.http.http-mime-and-md5/http.log @@ -3,7 +3,7 @@ #empty_field (empty) #unset_field - #path http -#start 2009-11-18-20-58-04 +#open 2009-11-18-20-58-04 #fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p trans_depth method host uri referrer user_agent request_body_len response_body_len status_code status_msg info_code info_msg filename tags username password proxied mime_type md5 extraction_file #types time string addr port addr port count string string string string string count count count string count string string table[enum] string string table[string] string string file 1258577884.844956 UWkUyAuUGXf 192.168.1.104 1673 63.245.209.11 80 1 GET www.mozilla.org /style/enhanced.css http://www.mozilla.org/projects/calendar/ Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.5) Gecko/20091102 Firefox/3.5.5 0 2675 200 OK - - - (empty) - - - FAKE_MIME - - @@ -11,4 +11,4 @@ 1258577885.317160 UWkUyAuUGXf 192.168.1.104 1673 63.245.209.11 80 3 GET www.mozilla.org /images/template/screen/bullet_utility.png http://www.mozilla.org/style/screen.css Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.5) Gecko/20091102 Firefox/3.5.5 0 94 200 OK - - - (empty) - - - FAKE_MIME - - 1258577885.349639 UWkUyAuUGXf 192.168.1.104 1673 63.245.209.11 80 4 GET www.mozilla.org /images/template/screen/key-point-top.png http://www.mozilla.org/style/screen.css Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.5) Gecko/20091102 Firefox/3.5.5 0 2349 200 OK - - - (empty) - - - image/png e0029eea80812e9a8e57b8d05d52938a - 1258577885.394612 UWkUyAuUGXf 192.168.1.104 1673 63.245.209.11 80 5 GET www.mozilla.org /projects/calendar/images/header-sunbird.png http://www.mozilla.org/projects/calendar/calendar.css Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.5) Gecko/20091102 Firefox/3.5.5 0 27579 200 OK - - - (empty) - - - image/png 30aa926344f58019d047e85ba049ca1e - -#end 2009-11-18-20-58-32 +#close 2009-11-18-20-58-32 diff --git a/testing/btest/Baseline/scripts.base.protocols.http.http-pipelining/http.log b/testing/btest/Baseline/scripts.base.protocols.http.http-pipelining/http.log index 6e7eb96454..6b5e395902 100644 --- a/testing/btest/Baseline/scripts.base.protocols.http.http-pipelining/http.log +++ b/testing/btest/Baseline/scripts.base.protocols.http.http-pipelining/http.log @@ -3,7 +3,7 @@ #empty_field (empty) #unset_field - #path http -#start 2009-11-18-20-58-04 +#open 2009-11-18-20-58-04 #fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p trans_depth method host uri referrer user_agent request_body_len response_body_len status_code status_msg info_code info_msg filename tags username password proxied md5 extraction_file #types time string addr port addr port count string string string string string count count count string count string string table[enum] string string table[string] string file 1258577884.844956 UWkUyAuUGXf 192.168.1.104 1673 63.245.209.11 80 1 GET www.mozilla.org /style/enhanced.css http://www.mozilla.org/projects/calendar/ Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.5) Gecko/20091102 Firefox/3.5.5 0 2675 200 OK - - - (empty) - - - - - @@ -11,4 +11,4 @@ 1258577885.317160 UWkUyAuUGXf 192.168.1.104 1673 63.245.209.11 80 3 GET www.mozilla.org /images/template/screen/bullet_utility.png http://www.mozilla.org/style/screen.css Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.5) Gecko/20091102 Firefox/3.5.5 0 94 200 OK - - - (empty) - - - - - 1258577885.349639 UWkUyAuUGXf 192.168.1.104 1673 63.245.209.11 80 4 GET www.mozilla.org /images/template/screen/key-point-top.png http://www.mozilla.org/style/screen.css Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.5) Gecko/20091102 Firefox/3.5.5 0 2349 200 OK - - - (empty) - - - - - 1258577885.394612 UWkUyAuUGXf 192.168.1.104 1673 63.245.209.11 80 5 GET www.mozilla.org /projects/calendar/images/header-sunbird.png http://www.mozilla.org/projects/calendar/calendar.css Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.5) Gecko/20091102 Firefox/3.5.5 0 27579 200 OK - - - (empty) - - - - - -#end 2009-11-18-20-58-32 +#close 2009-11-18-20-58-32 diff --git a/testing/btest/Baseline/scripts.base.protocols.irc.basic/irc.log b/testing/btest/Baseline/scripts.base.protocols.irc.basic/irc.log index fe18751420..46adaa4c3e 100644 --- a/testing/btest/Baseline/scripts.base.protocols.irc.basic/irc.log +++ b/testing/btest/Baseline/scripts.base.protocols.irc.basic/irc.log @@ -3,11 +3,11 @@ #empty_field (empty) #unset_field - #path irc -#start 2011-07-20-19-12-44 +#open 2011-07-20-19-12-44 #fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p nick user command value addl dcc_file_name dcc_file_size extraction_file #types time string addr port addr port string string string string string string count file 1311189164.119437 UWkUyAuUGXf 192.168.1.77 57640 66.198.80.67 6667 - - NICK bloed - - - - 1311189164.119437 UWkUyAuUGXf 192.168.1.77 57640 66.198.80.67 6667 bloed - USER sdkfje sdkfje Montreal.QC.CA.Undernet.org dkdkrwq - - - 1311189174.474127 UWkUyAuUGXf 192.168.1.77 57640 66.198.80.67 6667 bloed sdkfje JOIN #easymovies (empty) - - - 1311189316.326025 UWkUyAuUGXf 192.168.1.77 57640 66.198.80.67 6667 bloed sdkfje DCC #easymovies (empty) ladyvampress-default(2011-07-07)-OS.zip 42208 - -#end 2011-07-20-19-15-42 +#close 2011-07-20-19-15-42 diff --git a/testing/btest/Baseline/scripts.base.protocols.irc.dcc-extract/irc.log b/testing/btest/Baseline/scripts.base.protocols.irc.dcc-extract/irc.log index 8bd6bd8394..e204a627b1 100644 --- a/testing/btest/Baseline/scripts.base.protocols.irc.dcc-extract/irc.log +++ b/testing/btest/Baseline/scripts.base.protocols.irc.dcc-extract/irc.log @@ -3,11 +3,11 @@ #empty_field (empty) #unset_field - #path irc -#start 2011-07-20-19-12-44 +#open 2011-07-20-19-12-44 #fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p nick user command value addl dcc_file_name dcc_file_size dcc_mime_type extraction_file #types time string addr port addr port string string string string string string count string file 1311189164.119437 UWkUyAuUGXf 192.168.1.77 57640 66.198.80.67 6667 - - NICK bloed - - - - - 1311189164.119437 UWkUyAuUGXf 192.168.1.77 57640 66.198.80.67 6667 bloed - USER sdkfje sdkfje Montreal.QC.CA.Undernet.org dkdkrwq - - - - 1311189174.474127 UWkUyAuUGXf 192.168.1.77 57640 66.198.80.67 6667 bloed sdkfje JOIN #easymovies (empty) - - - - 1311189316.326025 UWkUyAuUGXf 192.168.1.77 57640 66.198.80.67 6667 bloed sdkfje DCC #easymovies (empty) ladyvampress-default(2011-07-07)-OS.zip 42208 FAKE_MIME irc-dcc-item_192.168.1.77:57655-209.197.168.151:1024_1.dat -#end 2011-07-20-19-15-42 +#close 2011-07-20-19-15-42 diff --git a/testing/btest/Baseline/scripts.base.protocols.smtp.basic/smtp.log b/testing/btest/Baseline/scripts.base.protocols.smtp.basic/smtp.log index eca41f7d09..ba16578dfb 100644 --- a/testing/btest/Baseline/scripts.base.protocols.smtp.basic/smtp.log +++ b/testing/btest/Baseline/scripts.base.protocols.smtp.basic/smtp.log @@ -3,8 +3,8 @@ #empty_field (empty) #unset_field - #path smtp -#start 2009-10-05-06-06-12 +#open 2009-10-05-06-06-12 #fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p trans_depth helo mailfrom rcptto date from to reply_to msg_id in_reply_to subject x_originating_ip first_received second_received last_reply path user_agent #types time string addr port addr port count string string table[string] string string table[string] string string string string addr string string string vector[addr] string 1254722768.219663 arKYeMETxOg 10.10.1.4 1470 74.53.140.153 25 1 GP Mon, 5 Oct 2009 11:36:07 +0530 "Gurpartap Singh" - <000301ca4581$ef9e57f0$cedb07d0$@in> - SMTP - - - 250 OK id=1Mugho-0003Dg-Un 74.53.140.153,10.10.1.4 Microsoft Office Outlook 12.0 -#end 2009-10-05-06-06-16 +#close 2009-10-05-06-06-16 diff --git a/testing/btest/Baseline/scripts.base.protocols.smtp.mime-extract/smtp_entities.log b/testing/btest/Baseline/scripts.base.protocols.smtp.mime-extract/smtp_entities.log index 9bae222897..396a2e058d 100644 --- a/testing/btest/Baseline/scripts.base.protocols.smtp.mime-extract/smtp_entities.log +++ b/testing/btest/Baseline/scripts.base.protocols.smtp.mime-extract/smtp_entities.log @@ -3,10 +3,10 @@ #empty_field (empty) #unset_field - #path smtp_entities -#start 2009-10-05-06-06-10 +#open 2009-10-05-06-06-10 #fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p trans_depth filename content_len mime_type md5 extraction_file excerpt #types time string addr port addr port count string count string string file string 1254722770.692743 arKYeMETxOg 10.10.1.4 1470 74.53.140.153 25 1 - 79 FAKE_MIME - smtp-entity_10.10.1.4:1470-74.53.140.153:25_1.dat (empty) 1254722770.692743 arKYeMETxOg 10.10.1.4 1470 74.53.140.153 25 1 - 1918 FAKE_MIME - - (empty) 1254722770.692804 arKYeMETxOg 10.10.1.4 1470 74.53.140.153 25 1 NEWS.txt 10823 FAKE_MIME - smtp-entity_10.10.1.4:1470-74.53.140.153:25_2.dat (empty) -#end 2009-10-05-06-06-16 +#close 2009-10-05-06-06-16 diff --git a/testing/btest/Baseline/scripts.base.protocols.smtp.mime/smtp_entities.log b/testing/btest/Baseline/scripts.base.protocols.smtp.mime/smtp_entities.log index 5cb4bb15ef..1abe35e90f 100644 --- a/testing/btest/Baseline/scripts.base.protocols.smtp.mime/smtp_entities.log +++ b/testing/btest/Baseline/scripts.base.protocols.smtp.mime/smtp_entities.log @@ -3,10 +3,10 @@ #empty_field (empty) #unset_field - #path smtp_entities -#start 2009-10-05-06-06-10 +#open 2009-10-05-06-06-10 #fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p trans_depth filename content_len mime_type md5 extraction_file excerpt #types time string addr port addr port count string count string string file string 1254722770.692743 arKYeMETxOg 10.10.1.4 1470 74.53.140.153 25 1 - 79 FAKE_MIME 92bca2e6cdcde73647125da7dccbdd07 - (empty) 1254722770.692743 arKYeMETxOg 10.10.1.4 1470 74.53.140.153 25 1 - 1918 FAKE_MIME - - (empty) 1254722770.692804 arKYeMETxOg 10.10.1.4 1470 74.53.140.153 25 1 NEWS.txt 10823 FAKE_MIME a968bb0f9f9d95835b2e74c845877e87 - (empty) -#end 2009-10-05-06-06-16 +#close 2009-10-05-06-06-16 diff --git a/testing/btest/Baseline/scripts.base.protocols.socks.trace1/socks.log b/testing/btest/Baseline/scripts.base.protocols.socks.trace1/socks.log index 960ea71720..b2a8ef7d4c 100644 --- a/testing/btest/Baseline/scripts.base.protocols.socks.trace1/socks.log +++ b/testing/btest/Baseline/scripts.base.protocols.socks.trace1/socks.log @@ -3,8 +3,8 @@ #empty_field (empty) #unset_field - #path socks -#start 2012-06-20-17-23-38 +#open 2012-06-20-17-23-38 #fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p version user status request.host request.name request_p bound.host bound.name bound_p #types time string addr port addr port count string string addr string port addr string port 1340213015.276495 UWkUyAuUGXf 10.0.0.55 53994 60.190.189.214 8124 5 - succeeded - www.osnews.com 80 192.168.0.31 - 2688 -#end 2012-06-20-17-28-10 +#close 2012-06-20-17-28-10 diff --git a/testing/btest/Baseline/scripts.base.protocols.socks.trace1/tunnel.log b/testing/btest/Baseline/scripts.base.protocols.socks.trace1/tunnel.log index d914b3074e..d5aa58652e 100644 --- a/testing/btest/Baseline/scripts.base.protocols.socks.trace1/tunnel.log +++ b/testing/btest/Baseline/scripts.base.protocols.socks.trace1/tunnel.log @@ -3,8 +3,8 @@ #empty_field (empty) #unset_field - #path tunnel -#start 2012-06-20-17-23-35 +#open 2012-06-20-17-23-35 #fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p tunnel_type action #types time string addr port addr port enum enum 1340213015.276495 - 10.0.0.55 0 60.190.189.214 8124 Tunnel::SOCKS Tunnel::DISCOVER -#end 2012-06-20-17-28-10 +#close 2012-06-20-17-28-10 diff --git a/testing/btest/Baseline/scripts.base.protocols.socks.trace2/socks.log b/testing/btest/Baseline/scripts.base.protocols.socks.trace2/socks.log index ef07cc31a5..4053bd7359 100644 --- a/testing/btest/Baseline/scripts.base.protocols.socks.trace2/socks.log +++ b/testing/btest/Baseline/scripts.base.protocols.socks.trace2/socks.log @@ -3,8 +3,8 @@ #empty_field (empty) #unset_field - #path socks -#start 2012-06-19-13-41-02 +#open 2012-06-19-13-41-02 #fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p version user status request.host request.name request_p bound.host bound.name bound_p #types time string addr port addr port count string string addr string port addr string port 1340113261.914619 UWkUyAuUGXf 10.0.0.50 59580 85.194.84.197 1080 5 - succeeded - www.google.com 443 0.0.0.0 - 443 -#end 2012-06-19-13-41-05 +#close 2012-06-19-13-41-05 diff --git a/testing/btest/Baseline/scripts.base.protocols.socks.trace2/tunnel.log b/testing/btest/Baseline/scripts.base.protocols.socks.trace2/tunnel.log index 10f079b888..82df9b76df 100644 --- a/testing/btest/Baseline/scripts.base.protocols.socks.trace2/tunnel.log +++ b/testing/btest/Baseline/scripts.base.protocols.socks.trace2/tunnel.log @@ -3,8 +3,8 @@ #empty_field (empty) #unset_field - #path tunnel -#start 2012-06-19-13-41-01 +#open 2012-06-19-13-41-01 #fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p tunnel_type action #types time string addr port addr port enum enum 1340113261.914619 - 10.0.0.50 0 85.194.84.197 1080 Tunnel::SOCKS Tunnel::DISCOVER -#end 2012-06-19-13-41-05 +#close 2012-06-19-13-41-05 diff --git a/testing/btest/Baseline/scripts.base.protocols.socks.trace3/tunnel.log b/testing/btest/Baseline/scripts.base.protocols.socks.trace3/tunnel.log index 4299e302ce..867f3ed157 100644 --- a/testing/btest/Baseline/scripts.base.protocols.socks.trace3/tunnel.log +++ b/testing/btest/Baseline/scripts.base.protocols.socks.trace3/tunnel.log @@ -3,8 +3,8 @@ #empty_field (empty) #unset_field - #path tunnel -#start 2008-04-15-22-43-49 +#open 2008-04-15-22-43-49 #fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p tunnel_type action #types time string addr port addr port enum enum 1208299429.265774 - 127.0.0.1 0 127.0.0.1 1080 Tunnel::SOCKS Tunnel::DISCOVER -#end 2008-04-15-22-43-49 +#close 2008-04-15-22-43-49 diff --git a/testing/btest/Baseline/scripts.base.protocols.ssl.basic/ssl.log b/testing/btest/Baseline/scripts.base.protocols.ssl.basic/ssl.log index b77925e498..5bf3feddc5 100644 --- a/testing/btest/Baseline/scripts.base.protocols.ssl.basic/ssl.log +++ b/testing/btest/Baseline/scripts.base.protocols.ssl.basic/ssl.log @@ -3,8 +3,8 @@ #empty_field (empty) #unset_field - #path ssl -#start 2012-04-27-14-53-12 +#open 2012-04-27-14-53-12 #fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p version cipher server_name session_id subject issuer_subject not_valid_before not_valid_after last_alert #types time string addr port addr port string string string string string string time time string 1335538392.319381 UWkUyAuUGXf 192.168.1.105 62045 74.125.224.79 443 TLSv10 TLS_ECDHE_RSA_WITH_RC4_128_SHA ssl.gstatic.com - CN=*.gstatic.com,O=Google Inc,L=Mountain View,ST=California,C=US CN=Google Internet Authority,O=Google Inc,C=US 1334102677.000000 1365639277.000000 - -#end 2012-04-27-14-53-16 +#close 2012-04-27-14-53-16 diff --git a/testing/btest/Baseline/scripts.policy.protocols.conn.known-hosts/knownhosts-all.log b/testing/btest/Baseline/scripts.policy.protocols.conn.known-hosts/knownhosts-all.log index 6951e4d51f..d5f665e4bc 100644 --- a/testing/btest/Baseline/scripts.policy.protocols.conn.known-hosts/knownhosts-all.log +++ b/testing/btest/Baseline/scripts.policy.protocols.conn.known-hosts/knownhosts-all.log @@ -3,11 +3,11 @@ #empty_field (empty) #unset_field - #path known_hosts -#start 2011-03-18-19-06-08 +#open 2011-03-18-19-06-08 #fields ts host #types time addr 1300475168.783842 141.142.220.118 1300475168.783842 208.80.152.118 1300475168.915940 208.80.152.3 1300475168.962628 208.80.152.2 -#end 2011-03-18-19-06-13 +#close 2011-03-18-19-06-13 diff --git a/testing/btest/Baseline/scripts.policy.protocols.conn.known-hosts/knownhosts-local.log b/testing/btest/Baseline/scripts.policy.protocols.conn.known-hosts/knownhosts-local.log index b70a701448..a625691aa4 100644 --- a/testing/btest/Baseline/scripts.policy.protocols.conn.known-hosts/knownhosts-local.log +++ b/testing/btest/Baseline/scripts.policy.protocols.conn.known-hosts/knownhosts-local.log @@ -3,8 +3,8 @@ #empty_field (empty) #unset_field - #path known_hosts -#start 2011-03-18-19-06-08 +#open 2011-03-18-19-06-08 #fields ts host #types time addr 1300475168.783842 141.142.220.118 -#end 2011-03-18-19-06-13 +#close 2011-03-18-19-06-13 diff --git a/testing/btest/Baseline/scripts.policy.protocols.conn.known-hosts/knownhosts-remote.log b/testing/btest/Baseline/scripts.policy.protocols.conn.known-hosts/knownhosts-remote.log index 8e9d8c6c79..d05ccf6081 100644 --- a/testing/btest/Baseline/scripts.policy.protocols.conn.known-hosts/knownhosts-remote.log +++ b/testing/btest/Baseline/scripts.policy.protocols.conn.known-hosts/knownhosts-remote.log @@ -3,10 +3,10 @@ #empty_field (empty) #unset_field - #path known_hosts -#start 2011-03-18-19-06-08 +#open 2011-03-18-19-06-08 #fields ts host #types time addr 1300475168.783842 208.80.152.118 1300475168.915940 208.80.152.3 1300475168.962628 208.80.152.2 -#end 2011-03-18-19-06-13 +#close 2011-03-18-19-06-13 diff --git a/testing/btest/Baseline/scripts.policy.protocols.conn.known-services/knownservices-all.log b/testing/btest/Baseline/scripts.policy.protocols.conn.known-services/knownservices-all.log index 25198e92d5..af097e5db3 100644 --- a/testing/btest/Baseline/scripts.policy.protocols.conn.known-services/knownservices-all.log +++ b/testing/btest/Baseline/scripts.policy.protocols.conn.known-services/knownservices-all.log @@ -3,7 +3,7 @@ #empty_field (empty) #unset_field - #path known_services -#start 2011-06-24-15-51-31 +#open 2011-06-24-15-51-31 #fields ts host port_num port_proto service #types time addr port enum table[string] 1308930691.049431 172.16.238.131 22 tcp SSH @@ -11,4 +11,4 @@ 1308930716.462556 74.125.225.81 80 tcp HTTP 1308930718.361665 172.16.238.131 21 tcp FTP 1308930726.872485 141.142.192.39 22 tcp SSH -#end 2011-06-24-15-52-08 +#close 2011-06-24-15-52-08 diff --git a/testing/btest/Baseline/scripts.policy.protocols.conn.known-services/knownservices-local.log b/testing/btest/Baseline/scripts.policy.protocols.conn.known-services/knownservices-local.log index 598f49fa65..7c27e63a24 100644 --- a/testing/btest/Baseline/scripts.policy.protocols.conn.known-services/knownservices-local.log +++ b/testing/btest/Baseline/scripts.policy.protocols.conn.known-services/knownservices-local.log @@ -3,10 +3,10 @@ #empty_field (empty) #unset_field - #path known_services -#start 2011-06-24-15-51-31 +#open 2011-06-24-15-51-31 #fields ts host port_num port_proto service #types time addr port enum table[string] 1308930691.049431 172.16.238.131 22 tcp SSH 1308930694.550308 172.16.238.131 80 tcp HTTP 1308930718.361665 172.16.238.131 21 tcp FTP -#end 2011-06-24-15-52-08 +#close 2011-06-24-15-52-08 diff --git a/testing/btest/Baseline/scripts.policy.protocols.conn.known-services/knownservices-remote.log b/testing/btest/Baseline/scripts.policy.protocols.conn.known-services/knownservices-remote.log index c248b18146..77fbe1ef70 100644 --- a/testing/btest/Baseline/scripts.policy.protocols.conn.known-services/knownservices-remote.log +++ b/testing/btest/Baseline/scripts.policy.protocols.conn.known-services/knownservices-remote.log @@ -3,9 +3,9 @@ #empty_field (empty) #unset_field - #path known_services -#start 2011-06-24-15-51-56 +#open 2011-06-24-15-51-56 #fields ts host port_num port_proto service #types time addr port enum table[string] 1308930716.462556 74.125.225.81 80 tcp HTTP 1308930726.872485 141.142.192.39 22 tcp SSH -#end 2011-06-24-15-52-08 +#close 2011-06-24-15-52-08 diff --git a/testing/btest/Baseline/scripts.policy.protocols.dns.event-priority/dns.log b/testing/btest/Baseline/scripts.policy.protocols.dns.event-priority/dns.log index fb024db6d2..f4b77edde7 100644 --- a/testing/btest/Baseline/scripts.policy.protocols.dns.event-priority/dns.log +++ b/testing/btest/Baseline/scripts.policy.protocols.dns.event-priority/dns.log @@ -3,8 +3,8 @@ #empty_field (empty) #unset_field - #path dns -#start 1999-06-28-23-40-27 +#open 1999-06-28-23-40-27 #fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p proto trans_id query qclass qclass_name qtype qtype_name rcode rcode_name AA TC RD RA Z answers TTLs auth addl #types time string addr port addr port enum count string count string count string count string bool bool bool bool count vector[string] vector[interval] table[string] table[string] 930613226.529070 UWkUyAuUGXf 212.180.42.100 25000 131.243.64.3 53 tcp 34798 - - - - - 0 NOERROR F F F T 0 4.3.2.1 31337.000000 - - -#end 1999-06-28-23-40-27 +#close 1999-06-28-23-40-27 diff --git a/testing/btest/Baseline/signatures.bad-eval-condition/.stderr b/testing/btest/Baseline/signatures.bad-eval-condition/.stderr new file mode 100644 index 0000000000..c4de35ffe9 --- /dev/null +++ b/testing/btest/Baseline/signatures.bad-eval-condition/.stderr @@ -0,0 +1,2 @@ +error: Error in signature (./blah.sig:6): eval function parameters must be a 'signature_state' and a 'string' type (mark_conn) + diff --git a/testing/btest/Baseline/signatures.eval-condition/conn.log b/testing/btest/Baseline/signatures.eval-condition/conn.log new file mode 100644 index 0000000000..a803f74320 --- /dev/null +++ b/testing/btest/Baseline/signatures.eval-condition/conn.log @@ -0,0 +1,14 @@ +#separator \x09 +#set_separator , +#empty_field (empty) +#unset_field - +#path conn +#open 2012-08-23-16-41-23 +#fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p proto service duration orig_bytes resp_bytes conn_state local_orig missed_bytes history orig_pkts orig_ip_bytes resp_pkts resp_ip_bytes tunnel_parents +#types time string addr port addr port enum string interval count count string bool count string count count count count table[string] +1329843175.736107 arKYeMETxOg 141.142.220.235 37604 199.233.217.249 56666 tcp ftp-data 0.112432 0 342 SF - 0 ShAdfFa 4 216 4 562 (empty) +1329843179.871641 k6kgXLOoSKl 141.142.220.235 59378 199.233.217.249 56667 tcp ftp-data 0.111218 0 77 SF - 0 ShAdfFa 4 216 4 297 (empty) +1329843194.151526 nQcgTWjvg4c 199.233.217.249 61920 141.142.220.235 33582 tcp ftp-data 0.056211 342 0 SF - 0 ShADaFf 5 614 3 164 (empty) +1329843197.783443 j4u32Pc5bif 199.233.217.249 61918 141.142.220.235 37835 tcp ftp-data 0.056005 77 0 SF - 0 ShADaFf 5 349 3 164 (empty) +1329843161.968492 UWkUyAuUGXf 141.142.220.235 50003 199.233.217.249 21 tcp ftp,blah 38.055625 180 3146 SF - 0 ShAdDfFa 38 2164 25 4458 (empty) +#close 2012-08-23-16-41-23 diff --git a/testing/btest/Baseline/core.load-sigs/output b/testing/btest/Baseline/signatures.load-sigs/output similarity index 100% rename from testing/btest/Baseline/core.load-sigs/output rename to testing/btest/Baseline/signatures.load-sigs/output diff --git a/testing/btest/Traces/ipv6_zero_len_ah.trace b/testing/btest/Traces/ipv6_zero_len_ah.trace new file mode 100644 index 0000000000..7c3922525c Binary files /dev/null and b/testing/btest/Traces/ipv6_zero_len_ah.trace differ diff --git a/testing/btest/analyzers/conn-size-cc.bro b/testing/btest/analyzers/conn-size-cc.bro deleted file mode 100644 index 0ba7977cf5..0000000000 --- a/testing/btest/analyzers/conn-size-cc.bro +++ /dev/null @@ -1,2 +0,0 @@ -# @TEST-EXEC: bro -C -r ${TRACES}/conn-size.trace tcp udp icmp report_conn_size_analyzer=T -# @TEST-EXEC: btest-diff conn.log diff --git a/testing/btest/analyzers/conn-size.bro b/testing/btest/analyzers/conn-size.bro deleted file mode 100644 index 0ba7977cf5..0000000000 --- a/testing/btest/analyzers/conn-size.bro +++ /dev/null @@ -1,2 +0,0 @@ -# @TEST-EXEC: bro -C -r ${TRACES}/conn-size.trace tcp udp icmp report_conn_size_analyzer=T -# @TEST-EXEC: btest-diff conn.log diff --git a/testing/btest/bifs/analyzer_name.bro b/testing/btest/bifs/analyzer_name.bro new file mode 100644 index 0000000000..034344f5c4 --- /dev/null +++ b/testing/btest/bifs/analyzer_name.bro @@ -0,0 +1,9 @@ +# +# @TEST-EXEC: bro %INPUT >out +# @TEST-EXEC: btest-diff out + +event bro_init() + { + local a = 1; + print analyzer_name(a); + } diff --git a/testing/btest/bifs/bro_version.bro b/testing/btest/bifs/bro_version.bro new file mode 100644 index 0000000000..7465cbc0f5 --- /dev/null +++ b/testing/btest/bifs/bro_version.bro @@ -0,0 +1,9 @@ +# +# @TEST-EXEC: bro %INPUT + +event bro_init() + { + local a = bro_version(); + if ( |a| == 0 ) + exit(1); + } diff --git a/testing/btest/bifs/capture_state_updates.bro b/testing/btest/bifs/capture_state_updates.bro new file mode 100644 index 0000000000..3abfdffdc1 --- /dev/null +++ b/testing/btest/bifs/capture_state_updates.bro @@ -0,0 +1,9 @@ +# +# @TEST-EXEC: bro %INPUT >out +# @TEST-EXEC: btest-diff out +# @TEST-EXEC: test -f testfile + +event bro_init() + { + print capture_state_updates("testfile"); + } diff --git a/testing/btest/bifs/checkpoint_state.bro b/testing/btest/bifs/checkpoint_state.bro new file mode 100644 index 0000000000..2a66bd1729 --- /dev/null +++ b/testing/btest/bifs/checkpoint_state.bro @@ -0,0 +1,10 @@ +# +# @TEST-EXEC: bro %INPUT +# @TEST-EXEC: test -f .state/state.bst + +event bro_init() + { + local a = checkpoint_state(); + if ( a != T ) + exit(1); + } diff --git a/testing/btest/bifs/current_analyzer.bro b/testing/btest/bifs/current_analyzer.bro new file mode 100644 index 0000000000..45b495c046 --- /dev/null +++ b/testing/btest/bifs/current_analyzer.bro @@ -0,0 +1,11 @@ +# +# @TEST-EXEC: bro %INPUT + +event bro_init() + { + local a = current_analyzer(); + if ( a != 0 ) + exit(1); + + # TODO: add a test for non-zero return value + } diff --git a/testing/btest/bifs/current_time.bro b/testing/btest/bifs/current_time.bro new file mode 100644 index 0000000000..5d16df396d --- /dev/null +++ b/testing/btest/bifs/current_time.bro @@ -0,0 +1,9 @@ +# +# @TEST-EXEC: bro %INPUT + +event bro_init() + { + local a = current_time(); + if ( a <= double_to_time(0) ) + exit(1); + } diff --git a/testing/btest/bifs/entropy_test.bro b/testing/btest/bifs/entropy_test.bro new file mode 100644 index 0000000000..ca01c79ed7 --- /dev/null +++ b/testing/btest/bifs/entropy_test.bro @@ -0,0 +1,24 @@ +# +# @TEST-EXEC: bro %INPUT >out +# @TEST-EXEC: btest-diff out + +event bro_init() + { + local a = "dh3Hie02uh^s#Sdf9L3frd243h$d78r2G4cM6*Q05d(7rh46f!0|4-f"; + if ( entropy_test_init(1) != T ) + exit(1); + + if ( entropy_test_add(1, a) != T ) + exit(1); + + print entropy_test_finish(1); + + local b = "0011000aaabbbbcccc000011111000000000aaaabbbbcccc0000000"; + if ( entropy_test_init(2) != T ) + exit(1); + + if ( entropy_test_add(2, b) != T ) + exit(1); + + print entropy_test_finish(2); + } diff --git a/testing/btest/bifs/get_matcher_stats.bro b/testing/btest/bifs/get_matcher_stats.bro new file mode 100644 index 0000000000..baee49fe1e --- /dev/null +++ b/testing/btest/bifs/get_matcher_stats.bro @@ -0,0 +1,9 @@ +# +# @TEST-EXEC: bro %INPUT + +event bro_init() + { + local a = get_matcher_stats(); + if ( a$matchers == 0 ) + exit(1); + } diff --git a/testing/btest/bifs/gethostname.bro b/testing/btest/bifs/gethostname.bro new file mode 100644 index 0000000000..97af719745 --- /dev/null +++ b/testing/btest/bifs/gethostname.bro @@ -0,0 +1,9 @@ +# +# @TEST-EXEC: bro %INPUT + +event bro_init() + { + local a = gethostname(); + if ( |a| == 0 ) + exit(1); + } diff --git a/testing/btest/bifs/getpid.bro b/testing/btest/bifs/getpid.bro new file mode 100644 index 0000000000..98edc19a44 --- /dev/null +++ b/testing/btest/bifs/getpid.bro @@ -0,0 +1,9 @@ +# +# @TEST-EXEC: bro %INPUT + +event bro_init() + { + local a = getpid(); + if ( a == 0 ) + exit(1); + } diff --git a/testing/btest/bifs/global_sizes.bro b/testing/btest/bifs/global_sizes.bro new file mode 100644 index 0000000000..4862db318b --- /dev/null +++ b/testing/btest/bifs/global_sizes.bro @@ -0,0 +1,16 @@ +# +# @TEST-EXEC: bro %INPUT >out +# @TEST-EXEC: btest-diff out + +event bro_init() + { + local a = global_sizes(); + for ( i in a ) + { + # the table is quite large, so just look for one item we expect + if ( i == "bro_init" ) + print "found bro_init"; + + } + + } diff --git a/testing/btest/bifs/identify_data.bro b/testing/btest/bifs/identify_data.bro new file mode 100644 index 0000000000..39f289d40b --- /dev/null +++ b/testing/btest/bifs/identify_data.bro @@ -0,0 +1,16 @@ +# +# @TEST-EXEC: bro %INPUT | sed 's/PNG image data/PNG image/g' >out +# @TEST-EXEC: btest-diff out + +event bro_init() + { + # plain text + local a = "This is a test"; + print identify_data(a, F); + print identify_data(a, T); + + # PNG image + local b = "\x89\x50\x4e\x47\x0d\x0a\x1a\x0a"; + print identify_data(b, F); + print identify_data(b, T); + } diff --git a/testing/btest/bifs/is_local_interface.bro b/testing/btest/bifs/is_local_interface.bro new file mode 100644 index 0000000000..8befdca385 --- /dev/null +++ b/testing/btest/bifs/is_local_interface.bro @@ -0,0 +1,11 @@ +# +# @TEST-EXEC: bro %INPUT >out +# @TEST-EXEC: btest-diff out + +event bro_init() + { + print is_local_interface(127.0.0.1); + print is_local_interface(1.2.3.4); + print is_local_interface([2607::a:b:c:d]); + print is_local_interface([::1]); + } diff --git a/testing/btest/bifs/reading_traces.bro b/testing/btest/bifs/reading_traces.bro new file mode 100644 index 0000000000..fc83c50ccb --- /dev/null +++ b/testing/btest/bifs/reading_traces.bro @@ -0,0 +1,10 @@ + +# @TEST-EXEC: bro %INPUT >out1 +# @TEST-EXEC: btest-diff out1 +# @TEST-EXEC: bro -r $TRACES/web.trace %INPUT >out2 +# @TEST-EXEC: btest-diff out2 + +event bro_init() + { + print reading_traces(); + } diff --git a/testing/btest/bifs/resource_usage.bro b/testing/btest/bifs/resource_usage.bro new file mode 100644 index 0000000000..35f5b020d6 --- /dev/null +++ b/testing/btest/bifs/resource_usage.bro @@ -0,0 +1,9 @@ +# +# @TEST-EXEC: bro %INPUT + +event bro_init() + { + local a = resource_usage(); + if ( a$version != bro_version() ) + exit(1); + } diff --git a/testing/btest/bifs/strftime.bro b/testing/btest/bifs/strftime.bro new file mode 100644 index 0000000000..31f9538632 --- /dev/null +++ b/testing/btest/bifs/strftime.bro @@ -0,0 +1,17 @@ +# +# @TEST-EXEC: bro %INPUT >out +# @TEST-EXEC: btest-diff out + +event bro_init() + { + local f1 = "%Y-%m-%d %H:%M:%S"; + local f2 = "%H%M%S %Y%m%d"; + + local a = double_to_time(0); + print strftime(f1, a); + print strftime(f2, a); + + a = double_to_time(123456789); + print strftime(f1, a); + print strftime(f2, a); + } diff --git a/testing/btest/btest.cfg b/testing/btest/btest.cfg index 4c4074ee24..d86b45d8a9 100644 --- a/testing/btest/btest.cfg +++ b/testing/btest/btest.cfg @@ -1,5 +1,5 @@ [btest] -TestDirs = doc bifs language core scripts istate coverage +TestDirs = doc bifs language core scripts istate coverage signatures TmpDir = %(testbase)s/.tmp BaselineDir = %(testbase)s/Baseline IgnoreDirs = .svn CVS .tmp diff --git a/testing/btest/core/ipv6_zero_len_ah.test b/testing/btest/core/ipv6_zero_len_ah.test new file mode 100644 index 0000000000..dc3acf8443 --- /dev/null +++ b/testing/btest/core/ipv6_zero_len_ah.test @@ -0,0 +1,11 @@ +# @TEST-EXEC: bro -r $TRACES/ipv6_zero_len_ah.trace %INPUT >output +# @TEST-EXEC: btest-diff output + +# Shouldn't crash, but we also won't have seq and data fields set of the ip6_ah +# record. + +event ipv6_ext_headers(c: connection, p: pkt_hdr) + { + print c$id; + print p; + } diff --git a/testing/btest/core/leaks/basic-cluster.bro b/testing/btest/core/leaks/basic-cluster.bro index f5b40c1104..319368bc6e 100644 --- a/testing/btest/core/leaks/basic-cluster.bro +++ b/testing/btest/core/leaks/basic-cluster.bro @@ -1,21 +1,22 @@ # Needs perftools support. # +# @TEST-SERIALIZE: comm # @TEST-GROUP: leaks - +# # @TEST-REQUIRES: bro --help 2>&1 | grep -q mem-leaks - +# # @TEST-EXEC: btest-bg-run manager-1 HEAP_CHECK_DUMP_DIRECTORY=. HEAPCHECK=local BROPATH=$BROPATH:.. CLUSTER_NODE=manager-1 bro -m %INPUT # @TEST-EXEC: btest-bg-run proxy-1 HEAP_CHECK_DUMP_DIRECTORY=. HEAPCHECK=local BROPATH=$BROPATH:.. CLUSTER_NODE=proxy-1 bro -m %INPUT # @TEST-EXEC: sleep 1 # @TEST-EXEC: btest-bg-run worker-1 HEAP_CHECK_DUMP_DIRECTORY=. HEAPCHECK=local BROPATH=$BROPATH:.. CLUSTER_NODE=worker-1 bro -m -r $TRACES/web.trace --pseudo-realtime %INPUT # @TEST-EXEC: btest-bg-run worker-2 HEAP_CHECK_DUMP_DIRECTORY=. HEAPCHECK=local BROPATH=$BROPATH:.. CLUSTER_NODE=worker-2 bro -m -r $TRACES/web.trace --pseudo-realtime %INPUT -# @TEST-EXEC: btest-bg-wait -k 30 +# @TEST-EXEC: btest-bg-wait 60 # @TEST-EXEC: btest-diff manager-1/metrics.log @TEST-START-FILE cluster-layout.bro redef Cluster::nodes = { - ["manager-1"] = [$node_type=Cluster::MANAGER, $ip=127.0.0.1, $p=37757/tcp, $workers=set("worker-1")], - ["proxy-1"] = [$node_type=Cluster::PROXY, $ip=127.0.0.1, $p=37758/tcp, $manager="manager-1", $workers=set("worker-1")], + ["manager-1"] = [$node_type=Cluster::MANAGER, $ip=127.0.0.1, $p=37757/tcp, $workers=set("worker-1", "worker-2")], + ["proxy-1"] = [$node_type=Cluster::PROXY, $ip=127.0.0.1, $p=37758/tcp, $manager="manager-1", $workers=set("worker-1", "worker-2")], ["worker-1"] = [$node_type=Cluster::WORKER, $ip=127.0.0.1, $p=37760/tcp, $manager="manager-1", $proxy="proxy-1", $interface="eth0"], ["worker-2"] = [$node_type=Cluster::WORKER, $ip=127.0.0.1, $p=37761/tcp, $manager="manager-1", $proxy="proxy-1", $interface="eth1"], }; @@ -32,11 +33,51 @@ event bro_init() &priority=5 Metrics::add_filter(TEST_METRIC, [$name="foo-bar", $break_interval=3secs]); - - if ( Cluster::local_node_type() == Cluster::WORKER ) + } + +event remote_connection_closed(p: event_peer) + { + terminate(); + } + +global ready_for_data: event(); + +redef Cluster::manager2worker_events += /ready_for_data/; + +@if ( Cluster::local_node_type() == Cluster::WORKER ) + +event ready_for_data() + { + Metrics::add_data(TEST_METRIC, [$host=1.2.3.4], 3); + Metrics::add_data(TEST_METRIC, [$host=6.5.4.3], 2); + Metrics::add_data(TEST_METRIC, [$host=7.2.1.5], 1); + } + +@endif + +@if ( Cluster::local_node_type() == Cluster::MANAGER ) + +global n = 0; +global peer_count = 0; + +event Metrics::log_metrics(rec: Metrics::Info) + { + n = n + 1; + if ( n == 3 ) { - Metrics::add_data(TEST_METRIC, [$host=1.2.3.4], 3); - Metrics::add_data(TEST_METRIC, [$host=6.5.4.3], 2); - Metrics::add_data(TEST_METRIC, [$host=7.2.1.5], 1); + terminate_communication(); + terminate(); } } + +event remote_connection_handshake_done(p: event_peer) + { + print p; + peer_count = peer_count + 1; + if ( peer_count == 3 ) + { + event ready_for_data(); + } + } + +@endif diff --git a/testing/btest/core/leaks/remote.bro b/testing/btest/core/leaks/remote.bro index f888d8f6ee..41bbaec076 100644 --- a/testing/btest/core/leaks/remote.bro +++ b/testing/btest/core/leaks/remote.bro @@ -1,20 +1,23 @@ # Needs perftools support. # +# @TEST-SERIALIZE: comm # @TEST-GROUP: leaks # # @TEST-REQUIRES: bro --help 2>&1 | grep -q mem-leaks # -# @TEST-EXEC: btest-bg-run sender HEAP_CHECK_DUMP_DIRECTORY=. HEAPCHECK=local bro -m --pseudo-realtime %INPUT ../sender.bro +# @TEST-EXEC: btest-bg-run sender HEAP_CHECK_DUMP_DIRECTORY=. HEAPCHECK=local bro -b -m --pseudo-realtime %INPUT ../sender.bro # @TEST-EXEC: sleep 1 -# @TEST-EXEC: btest-bg-run receiver HEAP_CHECK_DUMP_DIRECTORY=. HEAPCHECK=local bro -m --pseudo-realtime %INPUT ../receiver.bro +# @TEST-EXEC: btest-bg-run receiver HEAP_CHECK_DUMP_DIRECTORY=. HEAPCHECK=local bro -b -m --pseudo-realtime %INPUT ../receiver.bro # @TEST-EXEC: sleep 1 -# @TEST-EXEC: btest-bg-wait -k 10 +# @TEST-EXEC: btest-bg-wait 30 # @TEST-EXEC: btest-diff sender/test.log # @TEST-EXEC: btest-diff sender/test.failure.log # @TEST-EXEC: btest-diff sender/test.success.log -# @TEST-EXEC: cmp receiver/test.log sender/test.log -# @TEST-EXEC: cmp receiver/test.failure.log sender/test.failure.log -# @TEST-EXEC: cmp receiver/test.success.log sender/test.success.log +# @TEST-EXEC: ( cd sender && for i in *.log; do cat $i | $SCRIPTS/diff-remove-timestamps >c.$i; done ) +# @TEST-EXEC: ( cd receiver && for i in *.log; do cat $i | $SCRIPTS/diff-remove-timestamps >c.$i; done ) +# @TEST-EXEC: cmp receiver/c.test.log sender/c.test.log +# @TEST-EXEC: cmp receiver/c.test.failure.log sender/c.test.failure.log +# @TEST-EXEC: cmp receiver/c.test.success.log sender/c.test.success.log # This is the common part loaded by both sender and receiver. module Test; @@ -43,10 +46,10 @@ event bro_init() @TEST-START-FILE sender.bro -module Test; - @load frameworks/communication/listen +module Test; + function fail(rec: Log): bool { return rec$status != "success"; @@ -68,14 +71,27 @@ event remote_connection_handshake_done(p: event_peer) Log::write(Test::LOG, [$t=network_time(), $id=cid, $status="failure", $country="MX"]); disconnect(p); } + +event remote_connection_closed(p: event_peer) + { + terminate(); + } + @TEST-END-FILE @TEST-START-FILE receiver.bro ##### +@load base/frameworks/communication + redef Communication::nodes += { ["foo"] = [$host = 127.0.0.1, $connect=T, $request_logs=T] }; +event remote_connection_closed(p: event_peer) + { + terminate(); + } + @TEST-END-FILE diff --git a/testing/btest/core/tunnels/false-teredo.bro b/testing/btest/core/tunnels/false-teredo.bro index 37088e9535..381478bd54 100644 --- a/testing/btest/core/tunnels/false-teredo.bro +++ b/testing/btest/core/tunnels/false-teredo.bro @@ -1,8 +1,23 @@ # @TEST-EXEC: bro -r $TRACES/tunnels/false-teredo.pcap %INPUT >output # @TEST-EXEC: test ! -e weird.log +# @TEST-EXEC: test ! -e dpd.log # @TEST-EXEC: bro -r $TRACES/tunnels/false-teredo.pcap %INPUT Tunnel::yielding_teredo_decapsulation=F >output # @TEST-EXEC: btest-diff weird.log -# @TEST-EXEC: btest-diff dpd.log +# @TEST-EXEC: test ! -e dpd.log + +# In the first case, there isn't any weird or protocol violation logged +# since the teredo analyzer recognizes that the DNS analyzer has confirmed +# the protocol and yields. + +# In the second case, there are weirds since the teredo analyzer decapsulates +# despite the presence of the confirmed DNS analyzer and the resulting +# inner packets are malformed (no surprise there). There's also no dpd.log +# since the teredo analyzer doesn't confirm until it's seen a valid teredo +# encapsulation in both directions and protocol violations aren't logged +# until there's been a confirmation. + +# In either case, the analyzer doesn't, by default, get disabled as a result +# of the protocol violations. function print_teredo(name: string, outer: connection, inner: teredo_hdr) { diff --git a/testing/btest/core/tunnels/teredo-known-services.test b/testing/btest/core/tunnels/teredo-known-services.test new file mode 100644 index 0000000000..862930758f --- /dev/null +++ b/testing/btest/core/tunnels/teredo-known-services.test @@ -0,0 +1,11 @@ +# @TEST-EXEC: bro -b -r $TRACES/tunnels/false-teredo.pcap base/frameworks/dpd protocols/conn/known-services Tunnel::delay_teredo_confirmation=T "Site::local_nets+={192.168.1.0/24}" +# @TEST-EXEC: test ! -e known_services.log +# @TEST-EXEC: bro -b -r $TRACES/tunnels/false-teredo.pcap base/frameworks/dpd protocols/conn/known-services Tunnel::delay_teredo_confirmation=F "Site::local_nets+={192.168.1.0/24}" +# @TEST-EXEC: btest-diff known_services.log + +# The first case using Tunnel::delay_teredo_confirmation=T doesn't produce +# a known services.log since valid Teredo encapsulations from both endpoints +# of a connection is never witnessed and a protocol_confirmation never issued. + +# The second case issues protocol_confirmations more hastily and so bogus +# entries in known-services.log are more likely to appear. diff --git a/testing/btest/coverage/bare-mode-errors.test b/testing/btest/coverage/bare-mode-errors.test index 21e7d4f4a9..635726841b 100644 --- a/testing/btest/coverage/bare-mode-errors.test +++ b/testing/btest/coverage/bare-mode-errors.test @@ -8,6 +8,7 @@ # @TEST-SERIALIZE: comm # # @TEST-EXEC: test -d $DIST/scripts -# @TEST-EXEC: for script in `find $DIST/scripts -name \*\.bro -not -path '*/site/*'`; do echo $script; if echo "$script" | egrep -q 'communication/listen|controllee'; then rm -rf load_attempt .bgprocs; btest-bg-run load_attempt bro -b $script; btest-bg-wait -k 2; cat load_attempt/.stderr >>allerrors; else bro -b $script 2>>allerrors; fi done || exit 0 +# @TEST-EXEC: for script in `find $DIST/scripts/ -name \*\.bro -not -path '*/site/*'`; do echo $script; if echo "$script" | egrep -q 'communication/listen|controllee'; then rm -rf load_attempt .bgprocs; btest-bg-run load_attempt bro -b $script; btest-bg-wait -k 2; cat load_attempt/.stderr >>allerrors; else bro -b $script 2>>allerrors; fi done || exit 0 # @TEST-EXEC: cat allerrors | grep -v "received termination signal" | sort | uniq > unique_errors -# @TEST-EXEC: btest-diff unique_errors +# @TEST-EXEC: if [ $(grep -c CURL_INCLUDE_DIR-NOTFOUND $BUILD/CMakeCache.txt) -ne 0 ]; then cp unique_errors unique_errors_no_elasticsearch; fi +# @TEST-EXEC: if [ $(grep -c CURL_INCLUDE_DIR-NOTFOUND $BUILD/CMakeCache.txt) -ne 0 ]; then btest-diff unique_errors_no_elasticsearch; else btest-diff unique_errors; fi diff --git a/testing/btest/istate/bro-ipv6-socket.bro b/testing/btest/istate/bro-ipv6-socket.bro index b339bf4487..305f32caab 100644 --- a/testing/btest/istate/bro-ipv6-socket.bro +++ b/testing/btest/istate/bro-ipv6-socket.bro @@ -4,7 +4,7 @@ # # @TEST-EXEC: btest-bg-run recv bro -b ../recv.bro # @TEST-EXEC: btest-bg-run send bro -b ../send.bro -# @TEST-EXEC: btest-bg-wait -k 20 +# @TEST-EXEC: btest-bg-wait 20 # # @TEST-EXEC: btest-diff recv/.stdout # @TEST-EXEC: btest-diff send/.stdout @@ -14,7 +14,7 @@ @load base/frameworks/communication redef Communication::nodes += { - ["foo"] = [$host=[::1], $connect=T, $events=/my_event/] + ["foo"] = [$host=[::1], $connect=T, $retry=1sec, $events=/my_event/] }; global my_event: event(s: string); diff --git a/testing/btest/istate/broccoli-ipv6-socket.bro b/testing/btest/istate/broccoli-ipv6-socket.bro index 21067c1b23..be6266fdec 100644 --- a/testing/btest/istate/broccoli-ipv6-socket.bro +++ b/testing/btest/istate/broccoli-ipv6-socket.bro @@ -4,7 +4,8 @@ # @TEST-REQUIRES: ifconfig | grep -q -E "inet6 ::1|inet6 addr: ::1" # # @TEST-EXEC: btest-bg-run bro bro $DIST/aux/broccoli/test/broccoli-v6addrs.bro "Communication::listen_ipv6=T" +# @TEST-EXEC: sleep 1 # @TEST-EXEC: btest-bg-run broccoli $BUILD/aux/broccoli/test/broccoli-v6addrs -6 ::1 -# @TEST-EXEC: btest-bg-wait -k 20 +# @TEST-EXEC: btest-bg-wait 20 # @TEST-EXEC: btest-diff bro/.stdout # @TEST-EXEC: btest-diff broccoli/.stdout diff --git a/testing/btest/istate/broccoli-ipv6.bro b/testing/btest/istate/broccoli-ipv6.bro index ba181d4987..b4fdfb5fcf 100644 --- a/testing/btest/istate/broccoli-ipv6.bro +++ b/testing/btest/istate/broccoli-ipv6.bro @@ -3,7 +3,8 @@ # @TEST-REQUIRES: test -e $BUILD/aux/broccoli/src/libbroccoli.so || test -e $BUILD/aux/broccoli/src/libbroccoli.dylib # # @TEST-EXEC: btest-bg-run bro bro $DIST/aux/broccoli/test/broccoli-v6addrs.bro +# @TEST-EXEC: sleep 1 # @TEST-EXEC: btest-bg-run broccoli $BUILD/aux/broccoli/test/broccoli-v6addrs -# @TEST-EXEC: btest-bg-wait -k 20 +# @TEST-EXEC: btest-bg-wait 20 # @TEST-EXEC: btest-diff bro/.stdout # @TEST-EXEC: btest-diff broccoli/.stdout diff --git a/testing/btest/istate/broccoli-ssl.bro b/testing/btest/istate/broccoli-ssl.bro index 4465cd1bb3..dcbea93150 100644 --- a/testing/btest/istate/broccoli-ssl.bro +++ b/testing/btest/istate/broccoli-ssl.bro @@ -4,8 +4,9 @@ # # @TEST-EXEC: chmod 600 broccoli.conf # @TEST-EXEC: btest-bg-run bro bro $DIST/aux/broccoli/test/broccoli-v6addrs.bro "Communication::listen_ssl=T" "ssl_ca_certificate=../ca_cert.pem" "ssl_private_key=../bro.pem" +# @TEST-EXEC: sleep 1 # @TEST-EXEC: btest-bg-run broccoli BROCCOLI_CONFIG_FILE=../broccoli.conf $BUILD/aux/broccoli/test/broccoli-v6addrs -# @TEST-EXEC: btest-bg-wait -k 20 +# @TEST-EXEC: btest-bg-wait 20 # @TEST-EXEC: btest-diff bro/.stdout # @TEST-EXEC: btest-diff broccoli/.stdout diff --git a/testing/btest/istate/broccoli.bro b/testing/btest/istate/broccoli.bro index 2bae5dc080..2fdd4cbda4 100644 --- a/testing/btest/istate/broccoli.bro +++ b/testing/btest/istate/broccoli.bro @@ -3,8 +3,9 @@ # @TEST-REQUIRES: test -e $BUILD/aux/broccoli/src/libbroccoli.so || test -e $BUILD/aux/broccoli/src/libbroccoli.dylib # # @TEST-EXEC: btest-bg-run bro bro %INPUT $DIST/aux/broccoli/test/broping-record.bro +# @TEST-EXEC: sleep 1 # @TEST-EXEC: btest-bg-run broccoli $BUILD/aux/broccoli/test/broping -r -c 3 127.0.0.1 -# @TEST-EXEC: btest-bg-wait -k 20 +# @TEST-EXEC: btest-bg-wait 20 # @TEST-EXEC: cat bro/ping.log | sed 's/one-way.*//g' >bro.log # @TEST-EXEC: cat broccoli/.stdout | sed 's/time=.*//g' >broccoli.log # @TEST-EXEC: btest-diff bro.log diff --git a/testing/btest/istate/events-ssl.bro b/testing/btest/istate/events-ssl.bro index e09bf112fd..1d285869b4 100644 --- a/testing/btest/istate/events-ssl.bro +++ b/testing/btest/istate/events-ssl.bro @@ -2,7 +2,7 @@ # # @TEST-EXEC: btest-bg-run sender bro -C -r $TRACES/web.trace --pseudo-realtime ../sender.bro # @TEST-EXEC: btest-bg-run receiver bro ../receiver.bro -# @TEST-EXEC: btest-bg-wait -k 20 +# @TEST-EXEC: btest-bg-wait 20 # # @TEST-EXEC: btest-diff sender/http.log # @TEST-EXEC: btest-diff receiver/http.log @@ -55,7 +55,7 @@ event bro_init() redef peer_description = "events-rcv"; redef Communication::nodes += { - ["foo"] = [$host = 127.0.0.1, $events = /http_.*|signature_match/, $connect=T, $ssl=T] + ["foo"] = [$host = 127.0.0.1, $events = /http_.*|signature_match/, $connect=T, $ssl=T, $retry=1sec] }; redef ssl_ca_certificate = "../ca_cert.pem"; diff --git a/testing/btest/istate/events.bro b/testing/btest/istate/events.bro index 70726a9f20..590aabcd23 100644 --- a/testing/btest/istate/events.bro +++ b/testing/btest/istate/events.bro @@ -2,7 +2,7 @@ # # @TEST-EXEC: btest-bg-run sender bro -Bthreading,logging,comm -C -r $TRACES/web.trace --pseudo-realtime ../sender.bro # @TEST-EXEC: btest-bg-run receiver bro -Bthreading,logging,comm ../receiver.bro -# @TEST-EXEC: btest-bg-wait -k 20 +# @TEST-EXEC: btest-bg-wait 20 # # @TEST-EXEC: btest-diff sender/http.log # @TEST-EXEC: btest-diff receiver/http.log @@ -50,7 +50,7 @@ event bro_init() redef peer_description = "events-rcv"; redef Communication::nodes += { - ["foo"] = [$host = 127.0.0.1, $events = /http_.*|signature_match/, $connect=T] + ["foo"] = [$host = 127.0.0.1, $events = /http_.*|signature_match/, $connect=T, $retry=1sec] }; event remote_connection_closed(p: event_peer) diff --git a/testing/btest/istate/sync.bro b/testing/btest/istate/sync.bro index 776ddfd2fa..e1364a9553 100644 --- a/testing/btest/istate/sync.bro +++ b/testing/btest/istate/sync.bro @@ -154,7 +154,8 @@ event bro_init() } redef Communication::nodes += { - ["foo"] = [$host = 127.0.0.1, $events = /.*/, $connect=T, $sync=T] + ["foo"] = [$host = 127.0.0.1, $events = /.*/, $connect=T, $sync=T, + $retry=1sec] }; event remote_connection_closed(p: event_peer) diff --git a/testing/btest/language/addr.bro b/testing/btest/language/addr.bro new file mode 100644 index 0000000000..1cd93bad03 --- /dev/null +++ b/testing/btest/language/addr.bro @@ -0,0 +1,47 @@ +# @TEST-EXEC: bro %INPUT >out +# @TEST-EXEC: btest-diff out + +function test_case(msg: string, expect: bool) + { + print fmt("%s (%s)", msg, expect ? "PASS" : "FAIL"); + } + + +event bro_init() +{ + # IPv4 addresses + local a1: addr = 0.0.0.0; + local a2: addr = 10.0.0.11; + local a3: addr = 255.255.255.255; + local a4 = 192.1.2.3; + + test_case( "IPv4 address inequality", a1 != a2 ); + test_case( "IPv4 address equality", a1 == 0.0.0.0 ); + test_case( "IPv4 address comparison", a1 < a2 ); + test_case( "IPv4 address comparison", a3 > a2 ); + test_case( "size of IPv4 address", |a1| == 32 ); + test_case( "IPv4 address type inference", type_name(a4) == "addr" ); + + # IPv6 addresses + local b1: addr = [::]; + local b2: addr = [::255.255.255.255]; + local b3: addr = [::ffff:ffff]; + local b4: addr = [ffff::ffff]; + local b5: addr = [0000:0000:0000:0000:0000:0000:0000:0000]; + local b6: addr = [aaaa:bbbb:cccc:dddd:eeee:ffff:1111:2222]; + local b7: addr = [AAAA:BBBB:CCCC:DDDD:EEEE:FFFF:1111:2222]; + local b8 = [a::b]; + + test_case( "IPv6 address inequality", b1 != b2 ); + test_case( "IPv6 address equality", b1 == b5 ); + test_case( "IPv6 address equality", b2 == b3 ); + test_case( "IPv6 address comparison", b1 < b2 ); + test_case( "IPv6 address comparison", b4 > b2 ); + test_case( "IPv6 address not case-sensitive", b6 == b7 ); + test_case( "size of IPv6 address", |b1| == 128 ); + test_case( "IPv6 address type inference", type_name(b8) == "addr" ); + + test_case( "IPv4 and IPv6 address inequality", a1 != b1 ); + +} + diff --git a/testing/btest/language/any.bro b/testing/btest/language/any.bro new file mode 100644 index 0000000000..7437ee9851 --- /dev/null +++ b/testing/btest/language/any.bro @@ -0,0 +1,40 @@ +# @TEST-EXEC: bro %INPUT >out +# @TEST-EXEC: btest-diff out + +function test_case(msg: string, expect: bool) + { + print fmt("%s (%s)", msg, expect ? "PASS" : "FAIL"); + } + +function anyarg(arg1: any, arg1type: string) + { + test_case( arg1type, type_name(arg1) == arg1type ); + } + +event bro_init() +{ + local any1: any = 5; + local any2: any = "bar"; + local any3: any = /bar/; + + # Test using variable of type "any" + + anyarg( any1, "count" ); + anyarg( any2, "string" ); + anyarg( any3, "pattern" ); + + # Test of other types + + anyarg( T, "bool" ); + anyarg( "foo", "string" ); + anyarg( 15, "count" ); + anyarg( +15, "int" ); + anyarg( 15.0, "double" ); + anyarg( /foo/, "pattern" ); + anyarg( 127.0.0.1, "addr" ); + anyarg( [::1], "addr" ); + anyarg( 127.0.0.1/16, "subnet" ); + anyarg( [ffff::1]/64, "subnet" ); + anyarg( 123/tcp, "port" ); +} + diff --git a/testing/btest/language/at-if.bro b/testing/btest/language/at-if.bro new file mode 100644 index 0000000000..979ed0bb9a --- /dev/null +++ b/testing/btest/language/at-if.bro @@ -0,0 +1,49 @@ +# @TEST-EXEC: bro %INPUT >out +# @TEST-EXEC: btest-diff out + +function test_case(msg: string, expect: bool) + { + print fmt("%s (%s)", msg, expect ? "PASS" : "FAIL"); + } + + +event bro_init() +{ + local xyz = 0; + + # Test "if" without "else" + + @if ( F ) + xyz += 1; + @endif + + @if ( T ) + xyz += 2; + @endif + + test_case( "@if", xyz == 2 ); + + # Test "if" with an "else" + + xyz = 0; + + @if ( F ) + xyz += 1; + @else + xyz += 2; + @endif + + test_case( "@if...@else", xyz == 2 ); + + xyz = 0; + + @if ( T ) + xyz += 1; + @else + xyz += 2; + @endif + + test_case( "@if...@else", xyz == 1 ); + +} + diff --git a/testing/btest/language/at-ifdef.bro b/testing/btest/language/at-ifdef.bro new file mode 100644 index 0000000000..c30236f204 --- /dev/null +++ b/testing/btest/language/at-ifdef.bro @@ -0,0 +1,50 @@ +# @TEST-EXEC: bro %INPUT >out +# @TEST-EXEC: btest-diff out + +function test_case(msg: string, expect: bool) + { + print fmt("%s (%s)", msg, expect ? "PASS" : "FAIL"); + } + +global thisisdefined = 123; + +event bro_init() +{ + local xyz = 0; + + # Test "ifdef" without "else" + + @ifdef ( notdefined ) + xyz += 1; + @endif + + @ifdef ( thisisdefined ) + xyz += 2; + @endif + + test_case( "@ifdef", xyz == 2 ); + + # Test "ifdef" with an "else" + + xyz = 0; + + @ifdef ( doesnotexist ) + xyz += 1; + @else + xyz += 2; + @endif + + test_case( "@ifdef...@else", xyz == 2 ); + + xyz = 0; + + @ifdef ( thisisdefined ) + xyz += 1; + @else + xyz += 2; + @endif + + test_case( "@ifdef...@else", xyz == 1 ); + +} + diff --git a/testing/btest/language/at-ifndef.bro b/testing/btest/language/at-ifndef.bro new file mode 100644 index 0000000000..c98287590f --- /dev/null +++ b/testing/btest/language/at-ifndef.bro @@ -0,0 +1,50 @@ +# @TEST-EXEC: bro %INPUT >out +# @TEST-EXEC: btest-diff out + +function test_case(msg: string, expect: bool) + { + print fmt("%s (%s)", msg, expect ? "PASS" : "FAIL"); + } + +global thisisdefined = 123; + +event bro_init() +{ + local xyz = 0; + + # Test "ifndef" without "else" + + @ifndef ( notdefined ) + xyz += 1; + @endif + + @ifndef ( thisisdefined ) + xyz += 2; + @endif + + test_case( "@ifndef", xyz == 1 ); + + # Test "ifndef" with an "else" + + xyz = 0; + + @ifndef ( doesnotexist ) + xyz += 1; + @else + xyz += 2; + @endif + + test_case( "@ifndef...@else", xyz == 1 ); + + xyz = 0; + + @ifndef ( thisisdefined ) + xyz += 1; + @else + xyz += 2; + @endif + + test_case( "@ifndef...@else", xyz == 2 ); + +} + diff --git a/testing/btest/language/at-load.bro b/testing/btest/language/at-load.bro new file mode 100644 index 0000000000..b51594be16 --- /dev/null +++ b/testing/btest/language/at-load.bro @@ -0,0 +1,43 @@ +# @TEST-EXEC: bro %INPUT >out +# @TEST-EXEC: btest-diff out + +# In this script, we try to access each object defined in a "@load"ed script + +@load secondtestfile + +event bro_init() +{ + test_case( "function", T ); + test_case( "global variable", num == 123 ); + test_case( "const", daysperyear == 365 ); + event testevent( "foo" ); +} + + +# @TEST-START-FILE secondtestfile + +# In this script, we define some objects to be used in another script + +# Note: this script is not listed on the bro command-line (instead, it +# is "@load"ed from the other script) + +global test_case: function(msg: string, expect: bool); + +global testevent: event(msg: string); + +global num: count = 123; + +const daysperyear: count = 365; + +function test_case(msg: string, expect: bool) + { + print fmt("%s (%s)", msg, expect ? "PASS" : "FAIL"); + } + +event testevent(msg: string) + { + test_case( "event", T ); + } + +# @TEST-END-FILE + diff --git a/testing/btest/language/bool.bro b/testing/btest/language/bool.bro new file mode 100644 index 0000000000..b75343025f --- /dev/null +++ b/testing/btest/language/bool.bro @@ -0,0 +1,29 @@ +# @TEST-EXEC: bro %INPUT >out +# @TEST-EXEC: btest-diff out + +function test_case(msg: string, expect: bool) + { + print fmt("%s (%s)", msg, expect ? "PASS" : "FAIL"); + } + + +event bro_init() +{ + local b1: bool = T; + local b2: bool = F; + local b3: bool = T; + local b4 = T; + local b5 = F; + + test_case( "equality operator", b1 == b3 ); + test_case( "inequality operator", b1 != b2 ); + test_case( "logical or operator", b1 || b2 ); + test_case( "logical and operator", b1 && b3 ); + test_case( "negation operator", !b2 ); + test_case( "absolute value", |b1| == 1 ); + test_case( "absolute value", |b2| == 0 ); + test_case( "type inference", type_name(b4) == "bool" ); + test_case( "type inference", type_name(b5) == "bool" ); + +} + diff --git a/testing/btest/language/conditional-expression.bro b/testing/btest/language/conditional-expression.bro new file mode 100644 index 0000000000..74648b6ce8 --- /dev/null +++ b/testing/btest/language/conditional-expression.bro @@ -0,0 +1,66 @@ +# @TEST-EXEC: bro %INPUT >out +# @TEST-EXEC: btest-diff out + +function test_case(msg: string, expect: bool) + { + print fmt("%s (%s)", msg, expect ? "PASS" : "FAIL"); + } + +global ct: count; + +function f1(): bool + { + ct += 1; + return T; + } + +function f2(): bool + { + ct += 4; + return F; + } + + +event bro_init() +{ + local a: count; + local b: count; + local res: count; + local res2: bool; + + # Test that the correct operand is evaluated + + a = b = 0; + res = T ? ++a : ++b; + test_case( "true condition", a == 1 && b == 0 && res == 1); + + a = b = 0; + res = F ? ++a : ++b; + test_case( "false condition", a == 0 && b == 1 && res == 1); + + # Test again using function calls as operands + + ct = 0; + res2 = ct == 0 ? f1() : f2(); + test_case( "true condition", ct == 1 && res2 == T); + + ct = 0; + res2 = ct != 0 ? f1() : f2(); + test_case( "false condition", ct == 4 && res2 == F); + + # Test that the conditional operator is right-associative + + ct = 0; + T ? f1() : T ? f1() : f2(); + test_case( "associativity", ct == 1 ); + + ct = 0; + T ? f1() : (T ? f1() : f2()); + test_case( "associativity", ct == 1 ); + + ct = 0; + (T ? f1() : T) ? f1() : f2(); + test_case( "associativity", ct == 2 ); + +} + diff --git a/testing/btest/language/copy.bro b/testing/btest/language/copy.bro new file mode 100644 index 0000000000..6740a080c7 --- /dev/null +++ b/testing/btest/language/copy.bro @@ -0,0 +1,30 @@ +# @TEST-EXEC: bro %INPUT >out +# @TEST-EXEC: btest-diff out + +function test_case(msg: string, expect: bool) + { + print fmt("%s (%s)", msg, expect ? "PASS" : "FAIL"); + } + + + +event bro_init() +{ + # "b" is not a copy of "a" + local a: set[string] = set("this", "test"); + local b: set[string] = a; + + delete a["this"]; + + test_case( "direct assignment", |b| == 1 && "this" !in b ); + + # "d" is a copy of "c" + local c: set[string] = set("this", "test"); + local d: set[string] = copy(c); + + delete c["this"]; + + test_case( "using copy", |d| == 2 && "this" in d); + +} + diff --git a/testing/btest/language/count.bro b/testing/btest/language/count.bro new file mode 100644 index 0000000000..d6dcf5a97e --- /dev/null +++ b/testing/btest/language/count.bro @@ -0,0 +1,59 @@ +# @TEST-EXEC: bro %INPUT >out +# @TEST-EXEC: btest-diff out + +function test_case(msg: string, expect: bool) + { + print fmt("%s (%s)", msg, expect ? "PASS" : "FAIL"); + } + + +event bro_init() +{ + local c1: count = 0; + local c2: count = 5; + local c3: count = 0xFF; + local c4: count = 255; + local c5: count = 18446744073709551615; # maximum allowed value + local c6: count = 0xffffffffffffffff; # maximum allowed value + local c7: counter = 5; + local c8 = 1; + + # Type inference test + + test_case( "type inference", type_name(c8) == "count" ); + + # Counter alias test + + test_case( "counter alias", c2 == c7 ); + + # Test various constant representations + + test_case( "hexadecimal", c3 == c4 ); + + # Operator tests + + test_case( "inequality operator", c1 != c2 ); + test_case( "relational operator", c1 < c2 ); + test_case( "relational operator", c1 <= c2 ); + test_case( "relational operator", c2 > c1 ); + test_case( "relational operator", c2 >= c1 ); + test_case( "absolute value", |c1| == 0 ); + test_case( "absolute value", |c2| == 5 ); + test_case( "pre-increment operator", ++c2 == 6 ); + test_case( "pre-decrement operator", --c2 == 5 ); + test_case( "modulus operator", c2%2 == 1 ); + test_case( "division operator", c2/2 == 2 ); + c2 += 3; + test_case( "assignment operator", c2 == 8 ); + c2 -= 2; + test_case( "assignment operator", c2 == 6 ); + + # Max. value tests + + local str1 = fmt("max count value = %d", c5); + test_case( str1, str1 == "max count value = 18446744073709551615" ); + local str2 = fmt("max count value = %d", c6); + test_case( str2, str2 == "max count value = 18446744073709551615" ); + +} + diff --git a/testing/btest/language/double.bro b/testing/btest/language/double.bro new file mode 100644 index 0000000000..62ca768e22 --- /dev/null +++ b/testing/btest/language/double.bro @@ -0,0 +1,79 @@ +# @TEST-EXEC: bro %INPUT >out +# @TEST-EXEC: btest-diff out + +function test_case(msg: string, expect: bool) + { + print fmt("%s (%s)", msg, expect ? "PASS" : "FAIL"); + } + + +event bro_init() +{ + local d1: double = 3; + local d2: double = +3; + local d3: double = 3.; + local d4: double = 3.0; + local d5: double = +3.0; + local d6: double = 3e0; + local d7: double = 3E0; + local d8: double = 3e+0; + local d9: double = 3e-0; + local d10: double = 3.0e0; + local d11: double = +3.0e0; + local d12: double = +3.0e+0; + local d13: double = +3.0E+0; + local d14: double = +3.0E-0; + local d15: double = .03E+2; + local d16: double = .03E2; + local d17: double = 3.0001; + local d18: double = -3.0001; + local d19: double = 1.7976931348623157e308; # maximum allowed value + local d20 = 7.0; + local d21 = 7e0; + local d22 = 7e+1; + + # Type inference tests + + test_case( "type inference", type_name(d20) == "double" ); + test_case( "type inference", type_name(d21) == "double" ); + test_case( "type inference", type_name(d22) == "double" ); + + # Test various constant representations + + test_case( "double representations", d1 == d2 ); + test_case( "double representations", d1 == d3 ); + test_case( "double representations", d1 == d4 ); + test_case( "double representations", d1 == d5 ); + test_case( "double representations", d1 == d6 ); + test_case( "double representations", d1 == d7 ); + test_case( "double representations", d1 == d8 ); + test_case( "double representations", d1 == d9 ); + test_case( "double representations", d1 == d10 ); + test_case( "double representations", d1 == d11 ); + test_case( "double representations", d1 == d12 ); + test_case( "double representations", d1 == d13 ); + test_case( "double representations", d1 == d14 ); + test_case( "double representations", d1 == d15 ); + test_case( "double representations", d1 == d16 ); + + # Operator tests + + test_case( "inequality operator", d18 != d17 ); + test_case( "absolute value", |d18| == d17 ); + d4 += 2; + test_case( "assignment operator", d4 == 5.0 ); + d4 -= 3; + test_case( "assignment operator", d4 == 2.0 ); + test_case( "relational operator", d4 <= d3 ); + test_case( "relational operator", d4 < d3 ); + test_case( "relational operator", d17 >= d3 ); + test_case( "relational operator", d17 > d3 ); + test_case( "division operator", d3/2 == 1.5 ); + + # Max. value test + + local str1 = fmt("max double value = %.16e", d19); + test_case( str1, str1 == "max double value = 1.7976931348623157e+308" ); + +} + diff --git a/testing/btest/language/enum.bro b/testing/btest/language/enum.bro new file mode 100644 index 0000000000..5cafb323a6 --- /dev/null +++ b/testing/btest/language/enum.bro @@ -0,0 +1,32 @@ +# @TEST-EXEC: bro %INPUT >out +# @TEST-EXEC: btest-diff out + +function test_case(msg: string, expect: bool) + { + print fmt("%s (%s)", msg, expect ? "PASS" : "FAIL"); + } + + +# enum with optional comma at end of definition +type color: enum { Red, White, Blue, }; + +# enum without optional comma +type city: enum { Rome, Paris }; + + +event bro_init() +{ + local e1: color = Blue; + local e2: color = White; + local e3: color = Blue; + local e4: city = Rome; + + test_case( "enum equality comparison", e1 != e2 ); + test_case( "enum equality comparison", e1 == e3 ); + test_case( "enum equality comparison", e1 != e4 ); + + # type inference + local x = Blue; + test_case( "type inference", x == e1 ); +} + diff --git a/testing/btest/language/event.bro b/testing/btest/language/event.bro new file mode 100644 index 0000000000..1ea5c7b6d8 --- /dev/null +++ b/testing/btest/language/event.bro @@ -0,0 +1,49 @@ +# @TEST-EXEC: bro %INPUT >out +# @TEST-EXEC: btest-diff out + + +event e1() + { + print "event statement"; + return; + print "Error: this should not happen"; + } + +event e2() + { + print "schedule statement"; + } + +event e3(test: string) + { + print "event part1"; + } + +event e4(num: count) + { + print "assign event variable"; + } + +# Note: the name of this event is intentionally the same as one above +event e3(test: string) + { + print "event part2"; + } + +event bro_init() +{ + # Test calling an event with "event" statement + event e1(); + + # Test calling an event with "schedule" statement + schedule 1 sec { e2() }; + + # Test calling an event that has two separate definitions + event e3("foo"); + + # Test assigning an event variable to an event + local e5: event(num: count); + e5 = e4; + event e5(6); # TODO: this does not do anything +} + diff --git a/testing/btest/language/file.bro b/testing/btest/language/file.bro new file mode 100644 index 0000000000..1f631eb4fe --- /dev/null +++ b/testing/btest/language/file.bro @@ -0,0 +1,19 @@ +# @TEST-EXEC: bro %INPUT +# @TEST-EXEC: btest-diff out1 +# @TEST-EXEC: btest-diff out2 + + +event bro_init() +{ + local f1: file = open( "out1" ); + print f1, 20; + print f1, 12; + close(f1); + + # Type inference test + + local f2 = open( "out2" ); + print f2, "test", 123, 456; + close(f2); +} + diff --git a/testing/btest/language/for.bro b/testing/btest/language/for.bro new file mode 100644 index 0000000000..f10ef0eb1b --- /dev/null +++ b/testing/btest/language/for.bro @@ -0,0 +1,44 @@ +# @TEST-EXEC: bro %INPUT >out +# @TEST-EXEC: btest-diff out + +function test_case(msg: string, expect: bool) + { + print fmt("%s (%s)", msg, expect ? "PASS" : "FAIL"); + } + + + +event bro_init() +{ + local vv: vector of string = vector( "a", "b", "c" ); + local ct: count = 0; + + # Test a "for" loop without "break" or "next" + + ct = 0; + for ( i in vv ) ++ct; + test_case("for loop", ct == 3 ); + + # Test the "break" statement + + ct = 0; + for ( i in vv ) + { + ++ct; + break; + test_case("Error: this should not happen", F); + } + test_case("for loop with break", ct == 1 ); + + # Test the "next" statement + + ct = 0; + for ( i in vv ) + { + ++ct; + next; + test_case("Error: this should not happen", F); + } + test_case("for loop with next", ct == 3 ); +} + diff --git a/testing/btest/language/function.bro b/testing/btest/language/function.bro new file mode 100644 index 0000000000..13efbb91f8 --- /dev/null +++ b/testing/btest/language/function.bro @@ -0,0 +1,73 @@ +# @TEST-EXEC: bro %INPUT >out +# @TEST-EXEC: btest-diff out + +function test_case(msg: string, expect: bool) + { + print fmt("%s (%s)", msg, expect ? "PASS" : "FAIL"); + } + + +function f1() + { + test_case("no args without return value", T ); + } + +function f2() + { + test_case("no args no return value, empty return", T ); + return; + } + +function f3(): bool + { + return T; + } + +function f4(test: string) + { + test_case("args without return value", T ); + } + +function f5(test: string): bool + { + return T; + } + +function f6(test: string, num: count): bool + { + local val: int = -num; + if ( test == "bar" && num == 3 && val < 0 ) return T; + return F; + } + +function f7(test: string): bool + { + return F; + } + +event bro_init() +{ + f1(); + f2(); + test_case("no args with return value", f3() ); + f4("foo"); + test_case("args with return value", f5("foo") ); + test_case("multiple args with return value", f6("bar", 3) ); + + local f10 = function() { test_case("anonymous function without args or return value", T ); }; + f10(); + + local f11 = function(): bool { return T; }; + test_case("anonymous function with return value", f11() ); + + local f12 = function(val: int): bool { if (val > 0) return T; else return F; }; + test_case("anonymous function with args and return value", f12(2) ); + + # Test that a function variable can later be assigned to a function + local f13: function(test: string): bool; + f13 = f5; + test_case("assign function variable", f13("foo") ); + f13 = f7; + test_case("reassign function variable", !f13("bar") ); +} + diff --git a/testing/btest/language/if.bro b/testing/btest/language/if.bro new file mode 100644 index 0000000000..e9acea865f --- /dev/null +++ b/testing/btest/language/if.bro @@ -0,0 +1,71 @@ +# @TEST-EXEC: bro %INPUT >out +# @TEST-EXEC: btest-diff out + +function test_case(msg: string, expect: bool) + { + print fmt("%s (%s)", msg, expect ? "PASS" : "FAIL"); + } + + + +event bro_init() +{ + # Test "if" without "else" + + if ( T ) test_case( "if T", T); + + if ( F ) test_case( "Error: this should not happen", F); + + # Test "if" with only an "else" + + if ( T ) test_case( "if T else", T); + else test_case( "Error: this should not happen", F); + + if ( F ) test_case( "Error: this should not happen", F); + else test_case( "if F else", T); + + # Test "if" with only an "else if" + + if ( T ) test_case( "if T else if F", T); + else if ( F ) test_case( "Error: this should not happen", F); + + if ( F ) test_case( "Error: this should not happen", F); + else if ( T ) test_case( "if F else if T", T); + + if ( T ) test_case( "if T else if T", T); + else if ( T ) test_case( "Error: this should not happen", F); + + if ( F ) test_case( "Error: this should not happen", F); + else if ( F ) test_case( "Error: this should not happen", F); + + # Test "if" with both "else if" and "else" + + if ( T ) test_case( "if T else if F else", T); + else if ( F ) test_case( "Error: this should not happen", F); + else test_case( "Error: this should not happen", F); + + if ( F ) test_case( "Error: this should not happen", F); + else if ( T ) test_case( "if F else if T else", T); + else test_case( "Error: this should not happen", F); + + if ( T ) test_case( "if T else if T else", T); + else if ( T ) test_case( "Error: this should not happen", F); + else test_case( "Error: this should not happen", F); + + if ( F ) test_case( "Error: this should not happen", F); + else if ( F ) test_case( "Error: this should not happen", F); + else test_case( "if F else if F else", T); + + # Test "if" with multiple "else if" and an "else" + + if ( F ) test_case( "Error: this should not happen", F); + else if ( F ) test_case( "Error: this should not happen", F); + else if ( T ) test_case( "if F else if F else if T else", T); + else test_case( "Error: this should not happen", F); + + if ( F ) test_case( "Error: this should not happen", F); + else if ( F ) test_case( "Error: this should not happen", F); + else if ( F ) test_case( "Error: this should not happen", F); + else test_case( "if F else if F else if F else", T); +} + diff --git a/testing/btest/language/int.bro b/testing/btest/language/int.bro new file mode 100644 index 0000000000..5cfa1620bd --- /dev/null +++ b/testing/btest/language/int.bro @@ -0,0 +1,70 @@ +# @TEST-EXEC: bro %INPUT >out +# @TEST-EXEC: btest-diff out + +function test_case(msg: string, expect: bool) + { + print fmt("%s (%s)", msg, expect ? "PASS" : "FAIL"); + } + + +event bro_init() +{ + local i1: int = 3; + local i2: int = +3; + local i3: int = -3; + local i4: int = +0; + local i5: int = -0; + local i6: int = 12; + local i7: int = +0xc; + local i8: int = 0xC; + local i9: int = -0xC; + local i10: int = -12; + local i11: int = 9223372036854775807; # max. allowed value + local i12: int = -9223372036854775808; # min. allowed value + local i13: int = 0x7fffffffffffffff; # max. allowed value + local i14: int = -0x8000000000000000; # min. allowed value + local i15 = +3; + + # Type inference test + + test_case( "type inference", type_name(i15) == "int" ); + + # Test various constant representations + + test_case( "optional '+' sign", i1 == i2 ); + test_case( "negative vs. positive", i1 != i3 ); + test_case( "negative vs. positive", i4 == i5 ); + test_case( "hexadecimal", i6 == i7 ); + test_case( "hexadecimal", i6 == i8 ); + test_case( "hexadecimal", i9 == i10 ); + + # Operator tests + + test_case( "relational operator", i2 > i3 ); + test_case( "relational operator", i2 >= i3 ); + test_case( "relational operator", i3 < i2 ); + test_case( "relational operator", i3 <= i2 ); + test_case( "absolute value", |i4| == 0 ); + test_case( "absolute value", |i3| == 3 ); + test_case( "pre-increment operator", ++i2 == 4 ); + test_case( "pre-decrement operator", --i2 == 3 ); + test_case( "modulus operator", i2%2 == 1 ); + test_case( "division operator", i2/2 == 1 ); + i2 += 4; + test_case( "assignment operator", i2 == 7 ); + i2 -= 2; + test_case( "assignment operator", i2 == 5 ); + + # Max/min value tests + + local str1 = fmt("max int value = %d", i11); + test_case( str1, str1 == "max int value = 9223372036854775807" ); + local str2 = fmt("min int value = %d", i12); + test_case( str2, str2 == "min int value = -9223372036854775808" ); + local str3 = fmt("max int value = %d", i13); + test_case( str3, str3 == "max int value = 9223372036854775807" ); + local str4 = fmt("min int value = %d", i14); + test_case( str4, str4 == "min int value = -9223372036854775808" ); + +} + diff --git a/testing/btest/language/interval.bro b/testing/btest/language/interval.bro new file mode 100644 index 0000000000..66d44206d3 --- /dev/null +++ b/testing/btest/language/interval.bro @@ -0,0 +1,92 @@ +# @TEST-EXEC: bro %INPUT >out +# @TEST-EXEC: btest-diff out + +function test_case(msg: string, expect: bool) + { + print fmt("%s (%s)", msg, expect ? "PASS" : "FAIL"); + } + +function approx_equal(x: double, y: double): bool + { + # return T if x and y are approximately equal, and F otherwise + return |(x - y)/x| < 1e-6 ? T : F; + } + +event bro_init() +{ + # Constants without space and no letter "s" + + local in11: interval = 2usec; + local in12: interval = 2msec; + local in13: interval = 120sec; + local in14: interval = 2min; + local in15: interval = -2hr; + local in16: interval = 2.5day; + + # Constants with space and no letter "s" + + local in21: interval = 2 usec; + local in22: interval = 2 msec; + local in23: interval = 120 sec; + local in24: interval = 2 min; + local in25: interval = -2 hr; + local in26: interval = 2.5 day; + + # Constants with space and letter "s" + + local in31: interval = 2 usecs; + local in32: interval = 2 msecs; + local in33: interval = 1.2e2 secs; + local in34: interval = 2 mins; + local in35: interval = -2 hrs; + local in36: interval = 2.5 days; + + # Type inference + + local in41 = 2 usec; + local in42 = 2.1usec; + local in43 = 3usecs; + + # Type inference tests + + test_case( "type inference", type_name(in41) == "interval" ); + test_case( "type inference", type_name(in42) == "interval" ); + test_case( "type inference", type_name(in43) == "interval" ); + + # Test various constant representations + + test_case( "optional space", in11 == in21 ); + test_case( "plural/singular interval are same", in11 == in31 ); + + # Operator tests + + test_case( "different units with same numeric value", in11 != in12 ); + test_case( "compare different time units", in13 == in34 ); + test_case( "compare different time units", in13 <= in34 ); + test_case( "compare different time units", in13 >= in34 ); + test_case( "compare different time units", in13 < in36 ); + test_case( "compare different time units", in13 <= in36 ); + test_case( "compare different time units", in13 > in35 ); + test_case( "compare different time units", in13 >= in35 ); + test_case( "add different time units", in13 + in14 == 4min ); + test_case( "subtract different time units", in24 - in23 == 0sec ); + test_case( "absolute value", |in25| == 2.0*3600 ); + test_case( "absolute value", |in36| == 2.5*86400 ); + in34 += 2hr; + test_case( "assignment operator", in34 == 122min ); + in34 -= 2hr; + test_case( "assignment operator", in34 == 2min ); + test_case( "multiplication operator", in33*2 == 4min ); + test_case( "division operator", in35/2 == -1hr ); + test_case( "division operator", approx_equal(in32/in31, 1e3) ); + + # Test relative size of each interval unit + + test_case( "relative size of units", approx_equal(1msec/1usec, 1000) ); + test_case( "relative size of units", approx_equal(1sec/1msec, 1000) ); + test_case( "relative size of units", approx_equal(1min/1sec, 60) ); + test_case( "relative size of units", approx_equal(1hr/1min, 60) ); + test_case( "relative size of units", approx_equal(1day/1hr, 24) ); + +} + diff --git a/testing/btest/language/module.bro b/testing/btest/language/module.bro new file mode 100644 index 0000000000..4c70546406 --- /dev/null +++ b/testing/btest/language/module.bro @@ -0,0 +1,41 @@ +# @TEST-EXEC: bro %INPUT secondtestfile >out +# @TEST-EXEC: btest-diff out + +# In this source file, we define a module and export some objects + +module thisisatest; + +export { + global test_case: function(msg: string, expect: bool); + + global testevent: event(msg: string); + + global num: count = 123; + + const daysperyear: count = 365; +} + +function test_case(msg: string, expect: bool) + { + print fmt("%s (%s)", msg, expect ? "PASS" : "FAIL"); + } + +event testevent(msg: string) + { + test_case( "event", T ); + } + + +# @TEST-START-FILE secondtestfile + +# In this source file, we try to access each exported object from the module + +event bro_init() +{ + thisisatest::test_case( "function", T ); + thisisatest::test_case( "global variable", thisisatest::num == 123 ); + thisisatest::test_case( "const", thisisatest::daysperyear == 365 ); + event thisisatest::testevent( "foo" ); +} + +# @TEST-END-FILE diff --git a/testing/btest/language/no-module.bro b/testing/btest/language/no-module.bro new file mode 100644 index 0000000000..eadce66c18 --- /dev/null +++ b/testing/btest/language/no-module.bro @@ -0,0 +1,34 @@ +# @TEST-EXEC: bro %INPUT secondtestfile >out +# @TEST-EXEC: btest-diff out + +# This is the same test as "module.bro", but here we omit the module definition + + +global num: count = 123; + +const daysperyear: count = 365; + +function test_case(msg: string, expect: bool) + { + print fmt("%s (%s)", msg, expect ? "PASS" : "FAIL"); + } + +event testevent(msg: string) + { + test_case( "event", T ); + } + + +# @TEST-START-FILE secondtestfile + +# In this script, we try to access each object defined in the other script + +event bro_init() +{ + test_case( "function", T ); + test_case( "global variable", num == 123 ); + test_case( "const", daysperyear == 365 ); + event testevent( "foo" ); +} + +# @TEST-END-FILE diff --git a/testing/btest/language/null-statement.bro b/testing/btest/language/null-statement.bro new file mode 100644 index 0000000000..420ebd8a6c --- /dev/null +++ b/testing/btest/language/null-statement.bro @@ -0,0 +1,34 @@ +# @TEST-EXEC: bro %INPUT >out +# @TEST-EXEC: btest-diff out + + +function f1(test: string) + { + ; # null statement in function + } + +event bro_init() +{ + local s1: set[string] = set( "this", "test" ); + + ; # null statement in event + + for ( i in s1 ) + ; # null statement in for loop + + if ( |s1| > 0 ) ; # null statement in if statement + + f1("foo"); + + { ; } # null compound statement + + if ( |s1| == 0 ) + { + print "Error: this should not happen"; + } + else + ; # null statement in else + + print "done"; +} + diff --git a/testing/btest/language/pattern.bro b/testing/btest/language/pattern.bro new file mode 100644 index 0000000000..ec50dc66fe --- /dev/null +++ b/testing/btest/language/pattern.bro @@ -0,0 +1,32 @@ +# @TEST-EXEC: bro %INPUT >out +# @TEST-EXEC: btest-diff out + +function test_case(msg: string, expect: bool) + { + print fmt("%s (%s)", msg, expect ? "PASS" : "FAIL"); + } + + +event bro_init() +{ + local p1: pattern = /foo|bar/; + local p2: pattern = /oob/; + local p3: pattern = /^oob/; + local p4 = /foo/; + + # Type inference tests + + test_case( "type inference", type_name(p4) == "pattern" ); + + # Operator tests + + test_case( "equality operator", "foo" == p1 ); + test_case( "equality operator (order of operands)", p1 == "foo" ); + test_case( "inequality operator", "foobar" != p1 ); + test_case( "inequality operator (order of operands)", p1 != "foobar" ); + test_case( "in operator", p1 in "foobar" ); + test_case( "in operator", p2 in "foobar" ); + test_case( "!in operator", p3 !in "foobar" ); + +} + diff --git a/testing/btest/language/port.bro b/testing/btest/language/port.bro new file mode 100644 index 0000000000..1874e1dca3 --- /dev/null +++ b/testing/btest/language/port.bro @@ -0,0 +1,40 @@ +# @TEST-EXEC: bro %INPUT >out +# @TEST-EXEC: btest-diff out + +function test_case(msg: string, expect: bool) + { + print fmt("%s (%s)", msg, expect ? "PASS" : "FAIL"); + } + + +event bro_init() +{ + local p1: port = 1/icmp; + local p2: port = 2/udp; + local p3: port = 3/tcp; + local p4: port = 4/unknown; + local p5 = 123/tcp; + + # maximum allowed values for each port type + local p6: port = 255/icmp; + local p7: port = 65535/udp; + local p8: port = 65535/tcp; + local p9: port = 255/unknown; + + # Type inference test + + test_case( "type inference", type_name(p5) == "port" ); + + # Operator tests + + test_case( "protocol ordering", p1 > p2 ); + test_case( "protocol ordering", p2 > p3 ); + test_case( "protocol ordering", p3 > p4 ); + test_case( "protocol ordering", p8 < p7 ); + test_case( "protocol ordering", p9 < p6 ); + test_case( "different protocol but same numeric value", p7 != p8 ); + test_case( "different protocol but same numeric value", p6 != p9 ); + test_case( "equality operator", 65535/tcp == p8 ); + +} + diff --git a/testing/btest/language/precedence.bro b/testing/btest/language/precedence.bro new file mode 100644 index 0000000000..da8fef311c --- /dev/null +++ b/testing/btest/language/precedence.bro @@ -0,0 +1,110 @@ +# @TEST-EXEC: bro %INPUT >out +# @TEST-EXEC: btest-diff out + +function test_case(msg: string, expect: bool) + { + print fmt("%s (%s)", msg, expect ? "PASS" : "FAIL"); + } + +# This is an incomplete set of tests to demonstrate the order of precedence +# of bro script operators + +event bro_init() +{ + local n1: int; + local n2: int; + local n3: int; + + # Tests that show "++" has higher precedence than "*" + + n1 = n2 = 5; + n1 = ++n1 * 3; + n2 = (++n2) * 3; + test_case( "++ and *", n1 == 18 ); + test_case( "++ and *", n2 == 18 ); + + n1 = 5; + n1 = 3 * ++n1; + test_case( "* and ++", n1 == 18 ); + + # Tests that show "*" has same precedence as "%" + + n1 = 3 * 5 % 2; + n2 = (3 * 5) % 2; + n3 = 3 * (5 % 2); + test_case( "* and %", n1 == 1 ); + test_case( "* and %", n2 == 1 ); + test_case( "* and %", n3 == 3 ); + + n1 = 7 % 3 * 2; + n2 = (7 % 3) * 2; + n3 = 7 % (3 * 2); + test_case( "% and *", n1 == 2 ); + test_case( "% and *", n2 == 2 ); + test_case( "% and *", n3 == 1 ); + + # Tests that show "*" has higher precedence than "+" + + n1 = 1 + 2 * 3; + n2 = 1 + (2 * 3); + n3 = (1 + 2) * 3; + test_case( "+ and *", n1 == 7 ); + test_case( "+ and *", n2 == 7 ); + test_case( "+ and *", n3 == 9 ); + + # Tests that show "+" has higher precedence than "<" + + test_case( "< and +", 5 < 3 + 7 ); + test_case( "< and +", 5 < (3 + 7) ); + + test_case( "+ and <", 7 + 3 > 5 ); + test_case( "+ and <", (7 + 3) > 5 ); + + # Tests that show "+" has higher precedence than "+=" + + n1 = n2 = n3 = 0; + n1 += 1 + 2; + n2 += (1 + 2); + (n3 += 1) + 2; + test_case( "+= and +", n1 == 3 ); + test_case( "+= and +", n2 == 3 ); + test_case( "+= and +", n3 == 1 ); + + local r1: bool; + local r2: bool; + local r3: bool; + + # Tests that show "&&" has higher precedence than "||" + + r1 = F && F || T; + r2 = (F && F) || T; + r3 = F && (F || T); + test_case( "&& and ||", r1 ); + test_case( "&& and ||", r2 ); + test_case( "&& and ||", !r3 ); + + r1 = T || F && F; + r2 = T || (F && F); + r3 = (T || F) && F; + test_case( "|| and &&", r1 ); + test_case( "|| and &&", r2 ); + test_case( "|| and &&", !r3 ); + + # Tests that show "||" has higher precedence than conditional operator + + r1 = T || T ? F : F; + r2 = (T || T) ? F : F; + r3 = T || (T ? F : F); + test_case( "|| and conditional operator", !r1 ); + test_case( "|| and conditional operator", !r2 ); + test_case( "|| and conditional operator", r3 ); + + r1 = T ? F : F || T; + r2 = T ? F : (F || T); + r3 = (T ? F : F) || T; + test_case( "conditional operator and ||", !r1 ); + test_case( "conditional operator and ||", !r2 ); + test_case( "conditional operator and ||", r3 ); + +} + diff --git a/testing/btest/language/set.bro b/testing/btest/language/set.bro new file mode 100644 index 0000000000..5e56e3b9b8 --- /dev/null +++ b/testing/btest/language/set.bro @@ -0,0 +1,140 @@ +# @TEST-EXEC: bro %INPUT >out +# @TEST-EXEC: btest-diff out + +function test_case(msg: string, expect: bool) + { + print fmt("%s (%s)", msg, expect ? "PASS" : "FAIL"); + } + + +# Note: only global sets can be initialized with curly braces +global sg1: set[string] = { "curly", "braces" }; +global sg2: set[port, string, bool] = { [10/udp, "curly", F], + [11/udp, "braces", T] }; +global sg3 = { "more", "curly", "braces" }; + +event bro_init() +{ + local s1: set[string] = set( "test", "example" ); + local s2: set[string] = set(); + local s3: set[string]; + local s4 = set( "type inference" ); + local s5: set[port, string, bool] = set( [1/tcp, "test", T], + [2/tcp, "example", F] ); + local s6: set[port, string, bool] = set(); + local s7: set[port, string, bool]; + local s8 = set( [8/tcp, "type inference", T] ); + + # Type inference tests + + test_case( "type inference", type_name(s4) == "set[string]" ); + test_case( "type inference", type_name(s8) == "set[port,string,bool]" ); + test_case( "type inference", type_name(sg3) == "set[string]" ); + + # Test the size of each set + + test_case( "cardinality", |s1| == 2 ); + test_case( "cardinality", |s2| == 0 ); + test_case( "cardinality", |s3| == 0 ); + test_case( "cardinality", |s4| == 1 ); + test_case( "cardinality", |s5| == 2 ); + test_case( "cardinality", |s6| == 0 ); + test_case( "cardinality", |s7| == 0 ); + test_case( "cardinality", |s8| == 1 ); + test_case( "cardinality", |sg1| == 2 ); + test_case( "cardinality", |sg2| == 2 ); + test_case( "cardinality", |sg3| == 3 ); + + # Test iterating over each set + + local ct: count; + ct = 0; + for ( c in s1 ) + { + if ( type_name(c) != "string" ) + print "Error: wrong set element type"; + ++ct; + } + test_case( "iterate over set", ct == 2 ); + + ct = 0; + for ( c in s2 ) + { + ++ct; + } + test_case( "iterate over set", ct == 0 ); + + ct = 0; + for ( [c1,c2,c3] in s5 ) + { + ++ct; + } + test_case( "iterate over set", ct == 2 ); + + ct = 0; + for ( [c1,c2,c3] in sg2 ) + { + ++ct; + } + test_case( "iterate over set", ct == 2 ); + + # Test adding elements to each set (Note: cannot add elements to sets + # of multiple types) + + add s1["added"]; + add s1["added"]; # element already exists (nothing happens) + test_case( "add element", |s1| == 3 ); + test_case( "in operator", "added" in s1 ); + + add s2["another"]; + test_case( "add element", |s2| == 1 ); + add s2["test"]; + test_case( "add element", |s2| == 2 ); + test_case( "in operator", "another" in s2 ); + test_case( "in operator", "test" in s2 ); + + add s3["foo"]; + test_case( "add element", |s3| == 1 ); + test_case( "in operator", "foo" in s3 ); + + add s4["local"]; + test_case( "add element", |s4| == 2 ); + test_case( "in operator", "local" in s4 ); + + add sg1["global"]; + test_case( "add element", |sg1| == 3 ); + test_case( "in operator", "global" in sg1 ); + + add sg3["more global"]; + test_case( "add element", |sg3| == 4 ); + test_case( "in operator", "more global" in sg3 ); + + # Test removing elements from each set (Note: cannot remove elements + # from sets of multiple types) + + delete s1["test"]; + delete s1["foobar"]; # element does not exist (nothing happens) + test_case( "remove element", |s1| == 2 ); + test_case( "!in operator", "test" !in s1 ); + + delete s2["test"]; + test_case( "remove element", |s2| == 1 ); + test_case( "!in operator", "test" !in s2 ); + + delete s3["foo"]; + test_case( "remove element", |s3| == 0 ); + test_case( "!in operator", "foo" !in s3 ); + + delete s4["type inference"]; + test_case( "remove element", |s4| == 1 ); + test_case( "!in operator", "type inference" !in s4 ); + + delete sg1["braces"]; + test_case( "remove element", |sg1| == 2 ); + test_case( "!in operator", "braces" !in sg1 ); + + delete sg3["curly"]; + test_case( "remove element", |sg3| == 3 ); + test_case( "!in operator", "curly" !in sg3 ); +} + diff --git a/testing/btest/language/short-circuit.bro b/testing/btest/language/short-circuit.bro new file mode 100644 index 0000000000..f0ba585cea --- /dev/null +++ b/testing/btest/language/short-circuit.bro @@ -0,0 +1,48 @@ +# @TEST-EXEC: bro %INPUT >out +# @TEST-EXEC: btest-diff out + +function test_case(msg: string, expect: bool) + { + print fmt("%s (%s)", msg, expect ? "PASS" : "FAIL"); + } + +global ct: count; + +function t_func(): bool + { + ct += 1; + return T; + } + +function f_func(): bool + { + ct += 2; + return F; + } + + +event bro_init() +{ + local res: bool; + + # both functions should be called + ct = 0; + res = t_func() && f_func(); + test_case("&& operator (eval. both operands)", res == F && ct == 3 ); + + # only first function should be called + ct = 0; + res = f_func() && t_func(); + test_case("&& operator (eval. 1st operand)", res == F && ct == 2 ); + + # only first function should be called + ct = 0; + res = t_func() || f_func(); + test_case("|| operator (eval. 1st operand)", res == T && ct == 1 ); + + # both functions should be called + ct = 0; + res = f_func() || t_func(); + test_case("|| operator (eval. both operands)", res == T && ct == 3 ); +} + diff --git a/testing/btest/language/string.bro b/testing/btest/language/string.bro new file mode 100644 index 0000000000..3b9137cda5 --- /dev/null +++ b/testing/btest/language/string.bro @@ -0,0 +1,74 @@ +# @TEST-EXEC: bro %INPUT >out +# @TEST-EXEC: btest-diff out + +function test_case(msg: string, expect: bool) + { + print fmt("%s (%s)", msg, expect ? "PASS" : "FAIL"); + } + + +event bro_init() +{ + local s1: string = "a\ty"; # tab + local s2: string = "a\nb"; # newline + local s3: string = "a\"b"; # double quote + local s4: string = "a\\b"; # backslash + local s5: string = "a\x9y"; # 1-digit hex value (tab character) + local s6: string = "a\x0ab"; # 2-digit hex value (newline character) + local s7: string = "a\x22b"; # 2-digit hex value (double quote) + local s8: string = "a\x00b"; # 2-digit hex value (null character) + local s9: string = "a\011y"; # 3-digit octal value (tab character) + local s10: string = "a\12b"; # 2-digit octal value (newline character) + local s11: string = "a\0b"; # 1-digit octal value (null character) + + local s20: string = ""; + local s21: string = "x"; + local s22: string = s21 + s11; + local s23: string = "test"; + local s24: string = "this is a very long string" + + "which continues on the next line" + + "the end"; + local s25: string = "on"; + local s26 = "x"; + + # Type inference test + + test_case( "type inference", type_name(s26) == "string" ); + + # Escape sequence tests + + test_case( "tab escape sequence", |s1| == 3 ); + test_case( "newline escape sequence", |s2| == 3 ); + test_case( "double quote escape sequence", |s3| == 3 ); + test_case( "backslash escape sequence", |s4| == 3 ); + test_case( "1-digit hex escape sequence", |s5| == 3 ); + test_case( "2-digit hex escape sequence", |s6| == 3 ); + test_case( "2-digit hex escape sequence", |s7| == 3 ); + test_case( "2-digit hex escape sequence", |s8| == 3 ); + test_case( "3-digit octal escape sequence", |s9| == 3 ); + test_case( "2-digit octal escape sequence", |s10| == 3 ); + test_case( "1-digit octal escape sequence", |s11| == 3 ); + test_case( "tab escape sequence", s1 == s5 ); + test_case( "tab escape sequence", s5 == s9 ); + test_case( "newline escape sequence", s2 == s6 ); + test_case( "newline escape sequence", s6 == s10 ); + test_case( "double quote escape sequence", s3 == s7 ); + test_case( "null escape sequence", s8 == s11 ); + + # Operator tests + + test_case( "empty string", |s20| == 0 ); + test_case( "nonempty string", |s21| == 1 ); + test_case( "string comparison", s21 > s11 ); + test_case( "string comparison", s21 >= s11 ); + test_case( "string comparison", s11 < s21 ); + test_case( "string comparison", s11 <= s21 ); + test_case( "string concatenation", |s22| == 4 ); + s23 += s21; + test_case( "string concatenation", s23 == "testx" ); + test_case( "multi-line string initialization", |s24| == 65 ); + test_case( "in operator", s25 in s24 ); + test_case( "!in operator", s25 !in s23 ); + +} + diff --git a/testing/btest/language/subnet.bro b/testing/btest/language/subnet.bro new file mode 100644 index 0000000000..ea641f6983 --- /dev/null +++ b/testing/btest/language/subnet.bro @@ -0,0 +1,47 @@ +# @TEST-EXEC: bro %INPUT >out +# @TEST-EXEC: btest-diff out + +function test_case(msg: string, expect: bool) + { + print fmt("%s (%s)", msg, expect ? "PASS" : "FAIL"); + } + + +event bro_init() +{ + # IPv4 addr + local a1: addr = 192.1.2.3; + + # IPv4 subnets + local s1: subnet = 0.0.0.0/0; + local s2: subnet = 192.0.0.0/8; + local s3: subnet = 255.255.255.255/32; + local s4 = 10.0.0.0/16; + + test_case( "IPv4 subnet equality", a1/8 == s2 ); + test_case( "IPv4 subnet inequality", a1/4 != s2 ); + test_case( "IPv4 subnet in operator", a1 in s2 ); + test_case( "IPv4 subnet !in operator", a1 !in s3 ); + test_case( "IPv4 subnet type inference", type_name(s4) == "subnet" ); + + # IPv6 addrs + local b1: addr = [ffff::]; + local b2: addr = [ffff::1]; + local b3: addr = [ffff:1::1]; + + # IPv6 subnets + local t1: subnet = [::]/0; + local t2: subnet = [ffff::]/64; + local t3 = [a::]/32; + + test_case( "IPv6 subnet equality", b1/64 == t2 ); + test_case( "IPv6 subnet inequality", b3/64 != t2 ); + test_case( "IPv6 subnet in operator", b2 in t2 ); + test_case( "IPv6 subnet !in operator", b3 !in t2 ); + test_case( "IPv6 subnet type inference", type_name(t3) == "subnet" ); + + test_case( "IPv4 and IPv6 subnet inequality", s1 != t1 ); + test_case( "IPv4 address and IPv6 subnet", a1 !in t2 ); + +} + diff --git a/testing/btest/language/table.bro b/testing/btest/language/table.bro new file mode 100644 index 0000000000..d1b0751970 --- /dev/null +++ b/testing/btest/language/table.bro @@ -0,0 +1,149 @@ +# @TEST-EXEC: bro %INPUT >out +# @TEST-EXEC: btest-diff out + +function test_case(msg: string, expect: bool) + { + print fmt("%s (%s)", msg, expect ? "PASS" : "FAIL"); + } + +# Note: only global tables can be initialized with curly braces when the table +# type is not explicitly specified +global tg1 = { [1] = "type", [2] = "inference", [3] = "test" }; + +event bro_init() +{ + local t1: table[count] of string = table( [5] = "test", [0] = "example" ); + local t2: table[count] of string = table(); + local t3: table[count] of string; + local t4 = table( [1] = "type inference" ); + local t5: table[count] of string = { [1] = "curly", [3] = "braces" }; + local t6: table[port, string, bool] of string = table( + [1/tcp, "test", T] = "test1", + [2/tcp, "example", F] = "test2" ); + local t7: table[port, string, bool] of string = table(); + local t8: table[port, string, bool] of string; + local t9 = table( [8/tcp, "type inference", T] = "this" ); + local t10: table[port, string, bool] of string = { + [10/udp, "curly", F] = "first", + [11/udp, "braces", T] = "second" }; + + # Type inference tests + + test_case( "type inference", type_name(t4) == "table[count] of string" ); + test_case( "type inference", type_name(t9) == "table[port,string,bool] of string" ); + test_case( "type inference", type_name(tg1) == "table[count] of string" ); + + # Test the size of each table + + test_case( "cardinality", |t1| == 2 ); + test_case( "cardinality", |t2| == 0 ); + test_case( "cardinality", |t3| == 0 ); + test_case( "cardinality", |t4| == 1 ); + test_case( "cardinality", |t5| == 2 ); + test_case( "cardinality", |t6| == 2 ); + test_case( "cardinality", |t7| == 0 ); + test_case( "cardinality", |t8| == 0 ); + test_case( "cardinality", |t9| == 1 ); + test_case( "cardinality", |t10| == 2 ); + test_case( "cardinality", |tg1| == 3 ); + + # Test iterating over each table + + local ct: count; + ct = 0; + for ( c in t1 ) + { + if ( type_name(c) != "count" ) + print "Error: wrong index type"; + if ( type_name(t1[c]) != "string" ) + print "Error: wrong table type"; + ++ct; + } + test_case( "iterate over table", ct == 2 ); + + ct = 0; + for ( c in t2 ) + { + ++ct; + } + test_case( "iterate over table", ct == 0 ); + + ct = 0; + for ( c in t3 ) + { + ++ct; + } + test_case( "iterate over table", ct == 0 ); + + ct = 0; + for ( [c1, c2, c3] in t6 ) + { + ++ct; + } + test_case( "iterate over table", ct == 2 ); + + ct = 0; + for ( [c1, c2, c3] in t7 ) + { + ++ct; + } + test_case( "iterate over table", ct == 0 ); + + # Test overwriting elements in each table (Note: cannot overwrite + # elements in tables of multiple types) + + t1[5] = "overwrite"; + test_case( "overwrite element", |t1| == 2 && t1[5] == "overwrite" ); + + # Test adding elements to each table (Note: cannot add elements to + # tables of multiple types) + + t1[1] = "added"; + test_case( "add element", |t1| == 3 ); + test_case( "in operator", 1 in t1 ); + + t2[11] = "another"; + test_case( "add element", |t2| == 1 ); + t2[0] = "test"; + test_case( "add element", |t2| == 2 ); + test_case( "in operator", 11 in t2 ); + test_case( "in operator", 0 in t2 ); + + t3[3] = "foo"; + test_case( "add element", |t3| == 1 ); + test_case( "in operator", 3 in t3 ); + + t4[4] = "local"; + test_case( "add element", |t4| == 2 ); + test_case( "in operator", 4 in t4 ); + + t5[10] = "local2"; + test_case( "add element", |t5| == 3 ); + test_case( "in operator", 10 in t5 ); + + # Test removing elements from each table (Note: cannot remove elements + # from tables of multiple types) + + delete t1[0]; + delete t1[17]; # element does not exist (nothing happens) + test_case( "remove element", |t1| == 2 ); + test_case( "!in operator", 0 !in t1 ); + + delete t2[0]; + test_case( "remove element", |t2| == 1 ); + test_case( "!in operator", 0 !in t2 ); + + delete t3[3]; + test_case( "remove element", |t3| == 0 ); + test_case( "!in operator", 3 !in t3 ); + + delete t4[1]; + test_case( "remove element", |t4| == 1 ); + test_case( "!in operator", 1 !in t4 ); + + delete t5[1]; + test_case( "remove element", |t5| == 2 ); + test_case( "!in operator", 1 !in t5 ); + +} + diff --git a/testing/btest/language/time.bro b/testing/btest/language/time.bro new file mode 100644 index 0000000000..43b6694101 --- /dev/null +++ b/testing/btest/language/time.bro @@ -0,0 +1,33 @@ +# @TEST-EXEC: bro %INPUT >out +# @TEST-EXEC: btest-diff out + +function test_case(msg: string, expect: bool) + { + print fmt("%s (%s)", msg, expect ? "PASS" : "FAIL"); + } + + +event bro_init() +{ + local t1: time = current_time(); + local t2: time = t1 + 3 sec; + local t3: time = t2 - 10 sec; + local t4: time = t1; + local t5: time = double_to_time(1234567890); + local t6 = current_time(); + + # Type inference test + + test_case( "type inference", type_name(t6) == "time" ); + + # Operator tests + + test_case( "add interval", t1 < t2 ); + test_case( "subtract interval", t1 > t3 ); + test_case( "inequality", t1 != t3 ); + test_case( "equality", t1 == t4 ); + test_case( "subtract time", t2 - t1 == 3sec); + test_case( "size operator", |t5| == 1234567890.0 ); + +} + diff --git a/testing/btest/language/timeout.bro b/testing/btest/language/timeout.bro new file mode 100644 index 0000000000..6bc0419b2f --- /dev/null +++ b/testing/btest/language/timeout.bro @@ -0,0 +1,19 @@ +# @TEST-EXEC: bro %INPUT >out +# @TEST-EXEC: btest-diff out + + +event bro_init() +{ + local h1: addr = 1.2.3.4; + + when ( local h1name = lookup_addr(h1) ) + { + print "lookup successful"; + } + timeout 3 secs + { + print "timeout"; + } + +} + diff --git a/testing/btest/language/vector.bro b/testing/btest/language/vector.bro new file mode 100644 index 0000000000..928ddcb645 --- /dev/null +++ b/testing/btest/language/vector.bro @@ -0,0 +1,167 @@ +# @TEST-EXEC: bro %INPUT >out +# @TEST-EXEC: btest-diff out + +function test_case(msg: string, expect: bool) + { + print fmt("%s (%s)", msg, expect ? "PASS" : "FAIL"); + } + + +# Note: only global vectors can be initialized with curly braces +global vg1: vector of string = { "curly", "braces" }; + +event bro_init() +{ + local v1: vector of string = vector( "test", "example" ); + local v2: vector of string = vector(); + local v3: vector of string; + local v4 = vector( "type inference" ); + local v5 = vector( 1, 2, 3 ); + local v6 = vector( 10, 20, 30 ); + local v7 = v5 + v6; + local v8 = v6 - v5; + local v9 = v5 * v6; + local v10 = v6 / v5; + local v11 = v6 % v5; + local v12 = vector( T, F, T ); + local v13 = vector( F, F, T ); + local v14 = v12 && v13; + local v15 = v12 || v13; + + # Type inference tests + + test_case( "type inference", type_name(v4) == "vector of string" ); + test_case( "type inference", type_name(v5) == "vector of count" ); + test_case( "type inference", type_name(v12) == "vector of bool" ); + + # Test the size of each vector + + test_case( "cardinality", |v1| == 2 ); + test_case( "cardinality", |v2| == 0 ); + test_case( "cardinality", |v3| == 0 ); + test_case( "cardinality", |v4| == 1 ); + test_case( "cardinality", |v5| == 3 ); + test_case( "cardinality", |v6| == 3 ); + test_case( "cardinality", |v7| == 3 ); + test_case( "cardinality", |v8| == 3 ); + test_case( "cardinality", |v9| == 3 ); + test_case( "cardinality", |v10| == 3 ); + test_case( "cardinality", |v11| == 3 ); + test_case( "cardinality", |v12| == 3 ); + test_case( "cardinality", |v13| == 3 ); + test_case( "cardinality", |v14| == 3 ); + test_case( "cardinality", |v15| == 3 ); + test_case( "cardinality", |vg1| == 2 ); + + # Test that vectors use zero-based indexing + + test_case( "zero-based indexing", v1[0] == "test" && v5[0] == 1 ); + + # Test iterating over each vector + + local ct: count; + ct = 0; + for ( c in v1 ) + { + if ( type_name(c) != "int" ) + print "Error: wrong index type"; + if ( type_name(v1[c]) != "string" ) + print "Error: wrong vector type"; + ++ct; + } + test_case( "iterate over vector", ct == 2 ); + + ct = 0; + for ( c in v2 ) + { + ++ct; + } + test_case( "iterate over vector", ct == 0 ); + + ct = 0; + for ( c in vg1 ) + { + ++ct; + } + test_case( "iterate over vector", ct == 2 ); + + # Test adding elements to each vector + + v1[2] = "added"; + test_case( "add element", |v1| == 3 ); + test_case( "access element", v1[2] == "added" ); + + v2[0] = "another"; + test_case( "add element", |v2| == 1 ); + v2[1] = "test"; + test_case( "add element", |v2| == 2 ); + test_case( "access element", v2[0] == "another" ); + test_case( "access element", v2[1] == "test" ); + + v3[0] = "foo"; + test_case( "add element", |v3| == 1 ); + test_case( "access element", v3[0] == "foo" ); + + v4[1] = "local"; + test_case( "add element", |v4| == 2 ); + test_case( "access element", v4[1] == "local" ); + + v5[3] = 77; + test_case( "add element", |v5| == 4 ); + test_case( "access element", v5[3] == 77 ); + + vg1[2] = "global"; + test_case( "add element", |vg1| == 3 ); + test_case( "access element", vg1[2] == "global" ); + + # Test overwriting elements of each vector + + v1[0] = "new1"; + test_case( "overwrite element", |v1| == 3 ); + test_case( "access element", v1[0] == "new1" ); + + v2[1] = "new2"; + test_case( "overwrite element", |v2| == 2 ); + test_case( "access element", v2[0] == "another" ); + test_case( "access element", v2[1] == "new2" ); + + v3[0] = "new3"; + test_case( "overwrite element", |v3| == 1 ); + test_case( "access element", v3[0] == "new3" ); + + v4[0] = "new4"; + test_case( "overwrite element", |v4| == 2 ); + test_case( "access element", v4[0] == "new4" ); + + v5[0] = 0; + test_case( "overwrite element", |v5| == 4 ); + test_case( "access element", v5[0] == 0 ); + + vg1[1] = "new5"; + test_case( "overwrite element", |vg1| == 3 ); + test_case( "access element", vg1[1] == "new5" ); + + # Test increment/decrement operators + + ++v5; + test_case( "++ operator", |v5| == 4 && v5[0] == 1 && v5[1] == 3 + && v5[2] == 4 && v5[3] == 78 ); + --v5; + test_case( "-- operator", |v5| == 4 && v5[0] == 0 && v5[1] == 2 + && v5[2] == 3 && v5[3] == 77 ); + + # Test +,-,*,/,% of two vectors + + test_case( "+ operator", v7[0] == 11 && v7[1] == 22 && v7[2] == 33 ); + test_case( "- operator", v8[0] == 9 && v8[1] == 18 && v8[2] == 27 ); + test_case( "* operator", v9[0] == 10 && v9[1] == 40 && v9[2] == 90 ); + test_case( "/ operator", v10[0] == 10 && v10[1] == 10 && v10[2] == 10 ); + test_case( "% operator", v11[0] == 0 && v11[1] == 0 && v11[2] == 0 ); + + # Test &&,|| of two vectors + + test_case( "&& operator", v14[0] == F && v14[1] == F && v14[2] == T ); + test_case( "|| operator", v15[0] == T && v15[1] == F && v15[2] == T ); + +} + diff --git a/testing/btest/language/when.bro b/testing/btest/language/when.bro new file mode 100644 index 0000000000..84c1f06cef --- /dev/null +++ b/testing/btest/language/when.bro @@ -0,0 +1,20 @@ +# @TEST-SERIALIZE: comm +# @TEST-EXEC: btest-bg-run test1 bro %INPUT +# @TEST-EXEC: btest-bg-wait 10 +# @TEST-EXEC: mv test1/.stdout out +# @TEST-EXEC: btest-diff out + +@load frameworks/communication/listen + +event bro_init() +{ + local h1: addr = 127.0.0.1; + + when ( local h1name = lookup_addr(h1) ) + { + print "lookup successful"; + terminate(); + } + print "done"; +} + diff --git a/testing/btest/scripts/base/frameworks/cluster/start-it-up.bro b/testing/btest/scripts/base/frameworks/cluster/start-it-up.bro index a1069d1bd0..acb9c3676a 100644 --- a/testing/btest/scripts/base/frameworks/cluster/start-it-up.bro +++ b/testing/btest/scripts/base/frameworks/cluster/start-it-up.bro @@ -1,11 +1,13 @@ # @TEST-SERIALIZE: comm # # @TEST-EXEC: btest-bg-run manager-1 BROPATH=$BROPATH:.. CLUSTER_NODE=manager-1 bro %INPUT +# @TEST-EXEC: sleep 1 # @TEST-EXEC: btest-bg-run proxy-1 BROPATH=$BROPATH:.. CLUSTER_NODE=proxy-1 bro %INPUT # @TEST-EXEC: btest-bg-run proxy-2 BROPATH=$BROPATH:.. CLUSTER_NODE=proxy-2 bro %INPUT +# @TEST-EXEC: sleep 1 # @TEST-EXEC: btest-bg-run worker-1 BROPATH=$BROPATH:.. CLUSTER_NODE=worker-1 bro %INPUT # @TEST-EXEC: btest-bg-run worker-2 BROPATH=$BROPATH:.. CLUSTER_NODE=worker-2 bro %INPUT -# @TEST-EXEC: btest-bg-wait -k 10 +# @TEST-EXEC: btest-bg-wait 30 # @TEST-EXEC: btest-diff manager-1/.stdout # @TEST-EXEC: btest-diff proxy-1/.stdout # @TEST-EXEC: btest-diff proxy-2/.stdout @@ -22,7 +24,42 @@ redef Cluster::nodes = { }; @TEST-END-FILE +global fully_connected: event(); + +global peer_count = 0; + +global fully_connected_nodes = 0; + +event fully_connected() + { + fully_connected_nodes = fully_connected_nodes + 1; + if ( Cluster::node == "manager-1" ) + { + if ( peer_count == 4 && fully_connected_nodes == 4 ) + terminate_communication(); + } + } + +redef Cluster::worker2manager_events += /fully_connected/; +redef Cluster::proxy2manager_events += /fully_connected/; + event remote_connection_handshake_done(p: event_peer) { print "Connected to a peer"; + peer_count = peer_count + 1; + if ( Cluster::node == "manager-1" ) + { + if ( peer_count == 4 && fully_connected_nodes == 4 ) + terminate_communication(); + } + else + { + if ( peer_count == 2 ) + event fully_connected(); + } + } + +event remote_connection_closed(p: event_peer) + { + terminate(); } diff --git a/testing/btest/scripts/base/frameworks/control/configuration_update.bro b/testing/btest/scripts/base/frameworks/control/configuration_update.bro index 920a162503..d9e62efe08 100644 --- a/testing/btest/scripts/base/frameworks/control/configuration_update.bro +++ b/testing/btest/scripts/base/frameworks/control/configuration_update.bro @@ -1,7 +1,9 @@ # @TEST-SERIALIZE: comm # # @TEST-EXEC: btest-bg-run controllee BROPATH=$BROPATH:.. bro %INPUT frameworks/control/controllee Communication::listen_port=65531/tcp +# @TEST-EXEC: sleep 5 # @TEST-EXEC: btest-bg-run controller BROPATH=$BROPATH:.. bro %INPUT test-redef frameworks/control/controller Control::host=127.0.0.1 Control::host_port=65531/tcp Control::cmd=configuration_update +# @TEST-EXEC: sleep 5 # @TEST-EXEC: btest-bg-run controller2 BROPATH=$BROPATH:.. bro %INPUT frameworks/control/controller Control::host=127.0.0.1 Control::host_port=65531/tcp Control::cmd=shutdown # @TEST-EXEC: btest-bg-wait 10 # @TEST-EXEC: btest-diff controllee/.stdout diff --git a/testing/btest/scripts/base/frameworks/input/basic.bro b/testing/btest/scripts/base/frameworks/input/basic.bro index df2ab676b8..faab303534 100644 --- a/testing/btest/scripts/base/frameworks/input/basic.bro +++ b/testing/btest/scripts/base/frameworks/input/basic.bro @@ -8,9 +8,9 @@ @TEST-START-FILE input.log #separator \x09 #path ssh -#fields b i e c p sn a d t iv s sc ss se vc ve f -#types bool int enum count port subnet addr double time interval string table table table vector vector func -T -42 SSH::LOG 21 123 10.0.0.0/24 1.2.3.4 3.14 1315801931.273616 100.000000 hurz 2,4,1,3 CC,AA,BB EMPTY 10,20,30 EMPTY SSH::foo\x0a{ \x0aif (0 < SSH::i) \x0a\x09return (Foo);\x0aelse\x0a\x09return (Bar);\x0a\x0a} +#fields b i e c p sn a d t iv s sc ss se vc ve ns +#types bool int enum count port subnet addr double time interval string table table table vector vector string +T -42 SSH::LOG 21 123 10.0.0.0/24 1.2.3.4 3.14 1315801931.273616 100.000000 hurz 2,4,1,3 CC,AA,BB EMPTY 10,20,30 EMPTY 4242 @TEST-END-FILE @load base/protocols/ssh @@ -37,6 +37,7 @@ type Val: record { t: time; iv: interval; s: string; + ns: string; sc: set[count]; ss: set[string]; se: set[string]; @@ -57,6 +58,7 @@ event bro_init() event Input::update_finished(name: string, source:string) { print outfile, servers; + print outfile, to_count(servers[-42]$ns); # try to actually use a string. If null-termination is wrong this will fail. close(outfile); terminate(); } diff --git a/testing/btest/scripts/base/frameworks/input/bignumber.bro b/testing/btest/scripts/base/frameworks/input/bignumber.bro new file mode 100644 index 0000000000..250f84bbb2 --- /dev/null +++ b/testing/btest/scripts/base/frameworks/input/bignumber.bro @@ -0,0 +1,45 @@ +# (uses listen.bro just to ensure input sources are more reliably fully-read). +# @TEST-SERIALIZE: comm +# +# @TEST-EXEC: btest-bg-run bro bro -b %INPUT +# @TEST-EXEC: btest-bg-wait -k 5 +# @TEST-EXEC: btest-diff out + +@TEST-START-FILE input.log +#separator \x09 +#fields i c +#types int count +9223372036854775800 18446744073709551612 +-9223372036854775800 18446744073709551612 +@TEST-END-FILE + +@load frameworks/communication/listen + +global outfile: file; + +module A; + +type Idx: record { + i: int; +}; + +type Val: record { + c: count; +}; + +global servers: table[int] of Val = table(); + +event bro_init() + { + outfile = open("../out"); + # first read in the old stuff into the table... + Input::add_table([$source="../input.log", $name="ssh", $idx=Idx, $val=Val, $destination=servers]); + Input::remove("ssh"); + } + +event Input::update_finished(name: string, source:string) + { + print outfile, servers; + close(outfile); + terminate(); + } diff --git a/testing/btest/scripts/base/frameworks/input/binary.bro b/testing/btest/scripts/base/frameworks/input/binary.bro new file mode 100644 index 0000000000..8d75abc5a9 --- /dev/null +++ b/testing/btest/scripts/base/frameworks/input/binary.bro @@ -0,0 +1,56 @@ +# (uses listen.bro just to ensure input sources are more reliably fully-read). +# @TEST-SERIALIZE: comm +# +# @TEST-EXEC: btest-bg-run bro bro -b %INPUT +# @TEST-EXEC: btest-bg-wait -k 5 +# @TEST-EXEC: btest-diff out + +redef InputAscii::separator = "|"; +redef InputAscii::set_separator = ","; +redef InputAscii::empty_field = "(empty)"; +redef InputAscii::unset_field = "-"; + +@TEST-START-FILE input.log +#separator | +#set_separator|, +#empty_field|(empty) +#unset_field|- +#path|ssh +#open|2012-07-20-01-49-19 +#fields|data|data2 +#types|string|string +abc\x0a\xffdef|DATA2 +abc\x7c\xffdef|DATA2 +abc\xff\x7cdef|DATA2 +#end|2012-07-20-01-49-19 +@TEST-END-FILE + +@load frameworks/communication/listen + +global outfile: file; +global try: count; + +type Val: record { + data: string; + data2: string; +}; + +event line(description: Input::EventDescription, tpe: Input::Event, a: string, b: string) + { + print outfile, a; + print outfile, b; + try = try + 1; + if ( try == 3 ) + { + close(outfile); + terminate(); + } + } + +event bro_init() + { + try = 0; + outfile = open("../out"); + Input::add_event([$source="../input.log", $name="input", $fields=Val, $ev=line, $want_record=F]); + Input::remove("input"); + } diff --git a/testing/btest/scripts/base/frameworks/input/empty-values-hashing.bro b/testing/btest/scripts/base/frameworks/input/empty-values-hashing.bro new file mode 100644 index 0000000000..b66febba82 --- /dev/null +++ b/testing/btest/scripts/base/frameworks/input/empty-values-hashing.bro @@ -0,0 +1,89 @@ +# (uses listen.bro just to ensure input sources are more reliably fully-read). +# @TEST-SERIALIZE: comm +# +# @TEST-EXEC: cp input1.log input.log +# @TEST-EXEC: btest-bg-run bro bro -b %INPUT +# @TEST-EXEC: sleep 2 +# @TEST-EXEC: cp input2.log input.log +# @TEST-EXEC: btest-bg-wait -k 5 +# @TEST-EXEC: btest-diff out + +@TEST-START-FILE input1.log +#separator \x09 +#fields i s ss +#types int sting string +1 - TEST +2 - - +@TEST-END-FILE +@TEST-START-FILE input2.log +#separator \x09 +#fields i s ss +#types int sting string +1 TEST - +2 TEST TEST +@TEST-END-FILE + +@load frameworks/communication/listen + + +module A; + +type Idx: record { + i: int; +}; + +type Val: record { + s: string; + ss: string; +}; + +global servers: table[int] of Val = table(); + +global outfile: file; + +global try: count; + +event line(description: Input::TableDescription, tpe: Input::Event, left: Idx, right: Val) + { + print outfile, "============EVENT============"; + print outfile, "Description"; + print outfile, description; + print outfile, "Type"; + print outfile, tpe; + print outfile, "Left"; + print outfile, left; + print outfile, "Right"; + print outfile, right; + } + +event bro_init() + { + outfile = open("../out"); + try = 0; + # first read in the old stuff into the table... + Input::add_table([$source="../input.log", $mode=Input::REREAD, $name="ssh", $idx=Idx, $val=Val, $destination=servers, $ev=line, + $pred(typ: Input::Event, left: Idx, right: Val) = { + print outfile, "============PREDICATE============"; + print outfile, typ; + print outfile, left; + print outfile, right; + return T; + } + ]); + } + + +event Input::update_finished(name: string, source: string) + { + print outfile, "==========SERVERS============"; + print outfile, servers; + + try = try + 1; + if ( try == 2 ) + { + print outfile, "done"; + close(outfile); + Input::remove("input"); + terminate(); + } + } diff --git a/testing/btest/scripts/base/frameworks/input/event.bro b/testing/btest/scripts/base/frameworks/input/event.bro index f07ca0c43e..d0088472e7 100644 --- a/testing/btest/scripts/base/frameworks/input/event.bro +++ b/testing/btest/scripts/base/frameworks/input/event.bro @@ -49,6 +49,6 @@ event bro_init() { try = 0; outfile = open("../out"); - Input::add_event([$source="../input.log", $name="input", $fields=Val, $ev=line]); + Input::add_event([$source="../input.log", $name="input", $fields=Val, $ev=line, $want_record=F]); Input::remove("input"); } diff --git a/testing/btest/scripts/base/frameworks/input/executeraw.bro b/testing/btest/scripts/base/frameworks/input/executeraw.bro index 222b4256d1..626b9cdfd2 100644 --- a/testing/btest/scripts/base/frameworks/input/executeraw.bro +++ b/testing/btest/scripts/base/frameworks/input/executeraw.bro @@ -37,6 +37,6 @@ event line(description: Input::EventDescription, tpe: Input::Event, s: string) event bro_init() { outfile = open("../out.tmp"); - Input::add_event([$source="wc -l ../input.log |", $reader=Input::READER_RAW, $name="input", $fields=Val, $ev=line]); + Input::add_event([$source="wc -l ../input.log |", $reader=Input::READER_RAW, $name="input", $fields=Val, $ev=line, $want_record=F]); Input::remove("input"); } diff --git a/testing/btest/scripts/base/frameworks/input/invalidnumbers.bro b/testing/btest/scripts/base/frameworks/input/invalidnumbers.bro new file mode 100644 index 0000000000..3c755f1d08 --- /dev/null +++ b/testing/btest/scripts/base/frameworks/input/invalidnumbers.bro @@ -0,0 +1,48 @@ +# (uses listen.bro just to ensure input sources are more reliably fully-read). +# @TEST-SERIALIZE: comm +# +# @TEST-EXEC: btest-bg-run bro bro -b %INPUT +# @TEST-EXEC: btest-bg-wait -k 5 +# @TEST-EXEC: btest-diff out +# @TEST-EXEC: sed 1d .stderr > .stderrwithoutfirstline +# @TEST-EXEC: TEST_DIFF_CANONIFIER=$SCRIPTS/diff-remove-abspath btest-diff .stderrwithoutfirstline + +@TEST-START-FILE input.log +#separator \x09 +#fields i c +#types int count +12129223372036854775800 121218446744073709551612 +9223372036854775801TEXTHERE 1Justtext +Justtext 1 +9223372036854775800 -18446744073709551612 +@TEST-END-FILE + +@load frameworks/communication/listen + +global outfile: file; + +module A; + +type Idx: record { + i: int; +}; + +type Val: record { + c: count; +}; + +global servers: table[int] of Val = table(); + +event bro_init() + { + outfile = open("../out"); + # first read in the old stuff into the table... + Input::add_table([$source="../input.log", $name="ssh", $idx=Idx, $val=Val, $destination=servers]); + Input::remove("ssh"); + } + +event Input::update_finished(name: string, source:string) + { + print outfile, servers; + terminate(); + } diff --git a/testing/btest/scripts/base/frameworks/input/missing-file.bro b/testing/btest/scripts/base/frameworks/input/missing-file.bro new file mode 100644 index 0000000000..aa5acf619e --- /dev/null +++ b/testing/btest/scripts/base/frameworks/input/missing-file.bro @@ -0,0 +1,30 @@ +# (uses listen.bro just to ensure input sources are more reliably fully-read). +# @TEST-SERIALIZE: comm +# +# @TEST-EXEC: btest-bg-run bro bro -b %INPUT +# @TEST-EXEC: btest-bg-wait -k 5 +# @TEST-EXEC: btest-diff bro/.stderr + +@load frameworks/communication/listen + +global outfile: file; +global try: count; + +module A; + +type Val: record { + i: int; + b: bool; +}; + +event line(description: Input::EventDescription, tpe: Input::Event, i: int, b: bool) + { + } + +event bro_init() + { + try = 0; + outfile = open("../out"); + Input::add_event([$source="does-not-exist.dat", $name="input", $fields=Val, $ev=line, $want_record=F]); + Input::remove("input"); + } diff --git a/testing/btest/scripts/base/frameworks/input/predicate.bro b/testing/btest/scripts/base/frameworks/input/predicate.bro index 2cda6f5fb9..fcd986c9a6 100644 --- a/testing/btest/scripts/base/frameworks/input/predicate.bro +++ b/testing/btest/scripts/base/frameworks/input/predicate.bro @@ -35,7 +35,7 @@ type Val: record { b: bool; }; -global servers: table[int] of Val = table(); +global servers: table[int] of bool = table(); event bro_init() { diff --git a/testing/btest/scripts/base/frameworks/input/predicaterefusesecondsamerecord.bro b/testing/btest/scripts/base/frameworks/input/predicaterefusesecondsamerecord.bro new file mode 100644 index 0000000000..d572b30090 --- /dev/null +++ b/testing/btest/scripts/base/frameworks/input/predicaterefusesecondsamerecord.bro @@ -0,0 +1,56 @@ +# (uses listen.bro just to ensure input sources are more reliably fully-read). +# @TEST-SERIALIZE: comm +# +# @TEST-EXEC: btest-bg-run bro bro -b %INPUT +# @TEST-EXEC: btest-bg-wait -k 5 +# @TEST-EXEC: btest-diff out + +# Ok, this one tests a fun case. +# Input file contains two lines mapping to the same index, but with different values, +# where the predicate accepts the first one and refuses the second one. +# Desired result -> first entry stays. + +@TEST-START-FILE input.log +#fields restriction guid severity confidence detecttime address protocol portlist asn prefix rir cc impact description alternativeid_restriction alternativeid +need-to-know 8c864306-d21a-37b1-8705-746a786719bf medium 65 1342656000 1.0.17.227 - - 2519 VECTANT VECTANT Ltd. 1.0.16.0/23 apnic JP spam infrastructure spamming public http://reputation.alienvault.com/reputation.generic +need-to-know 8c864306-d21a-37b1-8705-746a786719bf medium 95 1342569600 1.228.83.33 6 25 9318 HANARO-AS Hanaro Telecom Inc. 1.224.0.0/13 apnic KR spam infrastructure direct ube sources, spam operations & spam services public http://www.spamhaus.org/query/bl?ip=1.228.83.33 +need-to-know 8c864306-d21a-37b1-8705-746a786719bf medium 65 1342656000 1.228.83.33 - - 9318 HANARO-AS Hanaro Telecom Inc. 1.224.0.0/13 apnic KR spam infrastructure spamming;malware domain public http://reputation.alienvault.com/reputation.generic +@TEST-END-FILE + +@load frameworks/communication/listen + +global outfile: file; + +redef InputAscii::empty_field = "EMPTY"; + +module A; + +type Idx: record { + address: addr; +}; + +type Val: record { + asn: string; + severity: string; + confidence: count; + detecttime: time; +}; + +global servers: table[addr] of Val = table(); + +event bro_init() + { + outfile = open("../out"); + # first read in the old stuff into the table... + Input::add_table([$source="../input.log", $name="input", $idx=Idx, $val=Val, $destination=servers, + $pred(typ: Input::Event, left: Idx, right: Val) = { if ( right$confidence > 90 ) { return T; } return F; } + ]); + Input::remove("input"); + } + +event Input::update_finished(name: string, source: string) + { + print outfile, servers; + close(outfile); + terminate(); + } diff --git a/testing/btest/scripts/base/frameworks/input/raw.bro b/testing/btest/scripts/base/frameworks/input/raw.bro index cb19213173..d15aec22bb 100644 --- a/testing/btest/scripts/base/frameworks/input/raw.bro +++ b/testing/btest/scripts/base/frameworks/input/raw.bro @@ -44,6 +44,6 @@ event bro_init() { try = 0; outfile = open("../out"); - Input::add_event([$source="../input.log", $reader=Input::READER_RAW, $mode=Input::STREAM, $name="input", $fields=Val, $ev=line]); + Input::add_event([$source="../input.log", $reader=Input::READER_RAW, $mode=Input::STREAM, $name="input", $fields=Val, $ev=line, $want_record=F]); Input::remove("input"); } diff --git a/testing/btest/scripts/base/frameworks/input/rereadraw.bro b/testing/btest/scripts/base/frameworks/input/rereadraw.bro index 1051351c2b..2fdcdc8f9e 100644 --- a/testing/btest/scripts/base/frameworks/input/rereadraw.bro +++ b/testing/btest/scripts/base/frameworks/input/rereadraw.bro @@ -44,7 +44,7 @@ event bro_init() { try = 0; outfile = open("../out"); - Input::add_event([$source="../input.log", $reader=Input::READER_RAW, $mode=Input::REREAD, $name="input", $fields=Val, $ev=line]); + Input::add_event([$source="../input.log", $reader=Input::READER_RAW, $mode=Input::REREAD, $name="input", $fields=Val, $ev=line, $want_record=F]); Input::force_update("input"); Input::remove("input"); } diff --git a/testing/btest/scripts/base/frameworks/input/set.bro b/testing/btest/scripts/base/frameworks/input/set.bro new file mode 100644 index 0000000000..5215523ee3 --- /dev/null +++ b/testing/btest/scripts/base/frameworks/input/set.bro @@ -0,0 +1,46 @@ +# (uses listen.bro just to ensure input sources are more reliably fully-read). +# @TEST-SERIALIZE: comm +# +# @TEST-EXEC: btest-bg-run bro bro -b %INPUT +# @TEST-EXEC: btest-bg-wait -k 5 +# @TEST-EXEC: TEST_DIFF_CANONIFIER=$SCRIPTS/diff-sort btest-diff out + +@TEST-START-FILE input.log +#separator \x09 +#fields ip +#types addr +192.168.17.1 +192.168.17.2 +192.168.17.7 +192.168.17.14 +192.168.17.42 +@TEST-END-FILE + +@load frameworks/communication/listen + +global outfile: file; + +redef InputAscii::empty_field = "EMPTY"; + +module A; + +type Idx: record { + ip: addr; +}; + +global servers: set[addr] = set(); + +event bro_init() + { + outfile = open("../out"); + # first read in the old stuff into the table... + Input::add_table([$source="../input.log", $name="ssh", $idx=Idx, $destination=servers]); + Input::remove("ssh"); + } + +event Input::update_finished(name: string, source:string) + { + print outfile, servers; + close(outfile); + terminate(); + } diff --git a/testing/btest/scripts/base/frameworks/input/setseparator.bro b/testing/btest/scripts/base/frameworks/input/setseparator.bro new file mode 100644 index 0000000000..44b9d08d54 --- /dev/null +++ b/testing/btest/scripts/base/frameworks/input/setseparator.bro @@ -0,0 +1,46 @@ +# (uses listen.bro just to ensure input sources are more reliably fully-read). +# @TEST-SERIALIZE: comm +# +# @TEST-EXEC: btest-bg-run bro bro -b %INPUT +# @TEST-EXEC: btest-bg-wait -k 5 +# @TEST-EXEC: TEST_DIFF_CANONIFIER=$SCRIPTS/diff-sort btest-diff out + +@TEST-START-FILE input.log +#separator \x09 +#fields i s ss +1 a|b|c|d|e|f 1|2|3|4|5|6 +@TEST-END-FILE + +redef InputAscii::set_separator = "|"; + +@load frameworks/communication/listen + +global outfile: file; + +module A; + +type Idx: record { + i: int; +}; + +type Val: record { + s: set[string]; + ss:vector of count; +}; + +global servers: table[int] of Val = table(); + +event bro_init() + { + outfile = open("../out"); + # first read in the old stuff into the table... + Input::add_table([$source="../input.log", $name="ssh", $idx=Idx, $val=Val, $destination=servers]); + Input::remove("ssh"); + } + +event Input::update_finished(name: string, source:string) + { + print outfile, servers; + close(outfile); + terminate(); + } diff --git a/testing/btest/scripts/base/frameworks/input/setspecialcases.bro b/testing/btest/scripts/base/frameworks/input/setspecialcases.bro new file mode 100644 index 0000000000..239bdfe7e7 --- /dev/null +++ b/testing/btest/scripts/base/frameworks/input/setspecialcases.bro @@ -0,0 +1,50 @@ +# (uses listen.bro just to ensure input sources are more reliably fully-read). +# @TEST-SERIALIZE: comm +# +# @TEST-EXEC: btest-bg-run bro bro -b %INPUT +# @TEST-EXEC: btest-bg-wait -k 5 +# @TEST-EXEC: TEST_DIFF_CANONIFIER=$SCRIPTS/diff-sort btest-diff out + +@TEST-START-FILE input.log +#separator \x09 +#fields i s ss +1 testing\x2ctesting\x2ctesting\x2c testing\x2ctesting\x2ctesting\x2c +2 testing,,testing testing,,testing +3 ,testing ,testing +4 testing, testing, +5 ,,, ,,, +6 +@TEST-END-FILE + + +@load frameworks/communication/listen + +global outfile: file; + +module A; + +type Idx: record { + i: int; +}; + +type Val: record { + s: set[string]; + s: vector of string; +}; + +global servers: table[int] of Val = table(); + +event bro_init() + { + outfile = open("../out"); + # first read in the old stuff into the table... + Input::add_table([$source="../input.log", $name="ssh", $idx=Idx, $val=Val, $destination=servers]); + Input::remove("ssh"); + } + +event Input::update_finished(name: string, source:string) + { + print outfile, servers; + close(outfile); + terminate(); + } diff --git a/testing/btest/scripts/base/frameworks/input/streamraw.bro b/testing/btest/scripts/base/frameworks/input/streamraw.bro index a6aba88c5f..3bc06f7dea 100644 --- a/testing/btest/scripts/base/frameworks/input/streamraw.bro +++ b/testing/btest/scripts/base/frameworks/input/streamraw.bro @@ -58,5 +58,5 @@ event bro_init() { outfile = open("../out"); try = 0; - Input::add_event([$source="../input.log", $reader=Input::READER_RAW, $mode=Input::STREAM, $name="input", $fields=Val, $ev=line]); + Input::add_event([$source="../input.log", $reader=Input::READER_RAW, $mode=Input::STREAM, $name="input", $fields=Val, $ev=line, $want_record=F]); } diff --git a/testing/btest/scripts/base/frameworks/input/subrecord-event.bro b/testing/btest/scripts/base/frameworks/input/subrecord-event.bro new file mode 100644 index 0000000000..4e7dc1690a --- /dev/null +++ b/testing/btest/scripts/base/frameworks/input/subrecord-event.bro @@ -0,0 +1,75 @@ +# (uses listen.bro just to ensure input sources are more reliably fully-read). +# @TEST-SERIALIZE: comm +# +# @TEST-EXEC: btest-bg-run bro bro -b %INPUT +# @TEST-EXEC: btest-bg-wait -k 5 +# @TEST-EXEC: btest-diff out + +@TEST-START-FILE input.log +#separator \x09 +#path ssh +#fields sub.b i sub.e sub.c sub.p sub.sn sub.two.a sub.two.d t iv s sc ss se vc ve f +#types bool int enum count port subnet addr double time interval string table table table vector vector func +T -42 SSH::LOG 21 123 10.0.0.0/24 1.2.3.4 3.14 1315801931.273616 100.000000 hurz 2,4,1,3 CC,AA,BB EMPTY 10,20,30 EMPTY SSH::foo\x0a{ \x0aif (0 < SSH::i) \x0a\x09return (Foo);\x0aelse\x0a\x09return (Bar);\x0a\x0a} +@TEST-END-FILE + +@load base/protocols/ssh +@load frameworks/communication/listen + +global outfile: file; +global try: count; + +redef InputAscii::empty_field = "EMPTY"; + +module A; + +type Idx: record { + i: int; +}; + +type SubVal2: record { + a: addr; + d: double; +}; + +type SubVal: record { + b: bool; + e: Log::ID; + c: count; + p: port; + sn: subnet; + two: SubVal2; +}; + +type Val: record { + sub: SubVal; + t: time; + iv: interval; + s: string; + sc: set[count]; + ss: set[string]; + se: set[string]; + vc: vector of int; + ve: vector of int; +}; + + + +event line(description: Input::EventDescription, tpe: Input::Event, value: Val) + { + print outfile, value; + try = try + 1; + if ( try == 1 ) + { + close(outfile); + terminate(); + } + } + +event bro_init() + { + try = 0; + outfile = open("../out"); + Input::add_event([$source="../input.log", $name="ssh", $fields=Val, $ev=line, $want_record=T]); + Input::remove("ssh"); + } diff --git a/testing/btest/scripts/base/frameworks/input/subrecord.bro b/testing/btest/scripts/base/frameworks/input/subrecord.bro new file mode 100644 index 0000000000..8c845a1842 --- /dev/null +++ b/testing/btest/scripts/base/frameworks/input/subrecord.bro @@ -0,0 +1,70 @@ +# (uses listen.bro just to ensure input sources are more reliably fully-read). +# @TEST-SERIALIZE: comm +# +# @TEST-EXEC: btest-bg-run bro bro -b %INPUT +# @TEST-EXEC: btest-bg-wait -k 5 +# @TEST-EXEC: btest-diff out + +@TEST-START-FILE input.log +#separator \x09 +#path ssh +#fields sub.b i sub.e sub.c sub.p sub.sn sub.two.a sub.two.d t iv s sc ss se vc ve f +#types bool int enum count port subnet addr double time interval string table table table vector vector func +T -42 SSH::LOG 21 123 10.0.0.0/24 1.2.3.4 3.14 1315801931.273616 100.000000 hurz 2,4,1,3 CC,AA,BB EMPTY 10,20,30 EMPTY SSH::foo\x0a{ \x0aif (0 < SSH::i) \x0a\x09return (Foo);\x0aelse\x0a\x09return (Bar);\x0a\x0a} +@TEST-END-FILE + +@load base/protocols/ssh +@load frameworks/communication/listen + +global outfile: file; + +redef InputAscii::empty_field = "EMPTY"; + +module A; + +type Idx: record { + i: int; +}; + +type SubVal2: record { + a: addr; + d: double; +}; + +type SubVal: record { + b: bool; + e: Log::ID; + c: count; + p: port; + sn: subnet; + two: SubVal2; +}; + +type Val: record { + sub: SubVal; + t: time; + iv: interval; + s: string; + sc: set[count]; + ss: set[string]; + se: set[string]; + vc: vector of int; + ve: vector of int; +}; + +global servers: table[int] of Val = table(); + +event bro_init() + { + outfile = open("../out"); + # first read in the old stuff into the table... + Input::add_table([$source="../input.log", $name="ssh", $idx=Idx, $val=Val, $destination=servers]); + Input::remove("ssh"); + } + +event Input::update_finished(name: string, source:string) + { + print outfile, servers; + close(outfile); + terminate(); + } diff --git a/testing/btest/scripts/base/frameworks/input/unsupported_types.bro b/testing/btest/scripts/base/frameworks/input/unsupported_types.bro new file mode 100644 index 0000000000..7affa4065d --- /dev/null +++ b/testing/btest/scripts/base/frameworks/input/unsupported_types.bro @@ -0,0 +1,64 @@ +# (uses listen.bro just to ensure input sources are more reliably fully-read). +# @TEST-SERIALIZE: comm +# +# @TEST-EXEC: btest-bg-run bro bro -b %INPUT +# @TEST-EXEC: btest-bg-wait -k 5 +# @TEST-EXEC: btest-diff out + +@TEST-START-FILE input.log +#separator \x09 +#path ssh +#fields fi b i e c p sn a d t iv s sc ss se vc ve f +#types file bool int enum count port subnet addr double time interval string table table table vector vector func +whatever T -42 SSH::LOG 21 123 10.0.0.0/24 1.2.3.4 3.14 1315801931.273616 100.000000 hurz 2,4,1,3 CC,AA,BB EMPTY 10,20,30 EMPTY SSH::foo\x0a{ \x0aif (0 < SSH::i) \x0a\x09return (Foo);\x0aelse\x0a\x09return (Bar);\x0a\x0a} +@TEST-END-FILE + +@load base/protocols/ssh +@load frameworks/communication/listen + +global outfile: file; + +redef InputAscii::empty_field = "EMPTY"; +redef Input::accept_unsupported_types = T; + +module A; + +type Idx: record { + i: int; +}; + +type Val: record { + fi: file &optional; + b: bool; + e: Log::ID; + c: count; + p: port; + sn: subnet; + a: addr; + d: double; + t: time; + iv: interval; + s: string; + sc: set[count]; + ss: set[string]; + se: set[string]; + vc: vector of int; + ve: vector of int; +}; + +global servers: table[int] of Val = table(); + +event bro_init() + { + outfile = open("../out"); + # first read in the old stuff into the table... + Input::add_table([$source="../input.log", $name="ssh", $idx=Idx, $val=Val, $destination=servers]); + Input::remove("ssh"); + } + +event Input::update_finished(name: string, source:string) + { + print outfile, servers; + close(outfile); + terminate(); + } diff --git a/testing/btest/scripts/base/frameworks/logging/ascii-escape.bro b/testing/btest/scripts/base/frameworks/logging/ascii-escape.bro index 1d0742216d..d73464777a 100644 --- a/testing/btest/scripts/base/frameworks/logging/ascii-escape.bro +++ b/testing/btest/scripts/base/frameworks/logging/ascii-escape.bro @@ -1,6 +1,6 @@ # # @TEST-EXEC: bro -b %INPUT -# @TEST-EXEC: cat ssh.log | egrep -v '#start|#end' >ssh.log.tmp && mv ssh.log.tmp ssh.log +# @TEST-EXEC: cat ssh.log | egrep -v '#open|#close' >ssh.log.tmp && mv ssh.log.tmp ssh.log # @TEST-EXEC: btest-diff ssh.log redef LogAscii::separator = "||"; diff --git a/testing/btest/scripts/base/frameworks/logging/remote-types.bro b/testing/btest/scripts/base/frameworks/logging/remote-types.bro index 3f102e6319..b8425428d3 100644 --- a/testing/btest/scripts/base/frameworks/logging/remote-types.bro +++ b/testing/btest/scripts/base/frameworks/logging/remote-types.bro @@ -4,8 +4,8 @@ # @TEST-EXEC: btest-bg-run receiver bro -B threading,logging --pseudo-realtime %INPUT ../receiver.bro # @TEST-EXEC: btest-bg-wait -k 10 # @TEST-EXEC: btest-diff receiver/test.log -# @TEST-EXEC: cat receiver/test.log | egrep -v '#start|#end' >r.log -# @TEST-EXEC: cat sender/test.log | egrep -v '#start|#end' >s.log +# @TEST-EXEC: cat receiver/test.log | egrep -v '#open|#close' >r.log +# @TEST-EXEC: cat sender/test.log | egrep -v '#open|#close' >s.log # @TEST-EXEC: cmp r.log s.log # Remote version testing all types. diff --git a/testing/btest/scripts/base/frameworks/logging/remote.bro b/testing/btest/scripts/base/frameworks/logging/remote.bro index 48683148f5..ba577cc92b 100644 --- a/testing/btest/scripts/base/frameworks/logging/remote.bro +++ b/testing/btest/scripts/base/frameworks/logging/remote.bro @@ -1,10 +1,10 @@ # @TEST-SERIALIZE: comm # -# @TEST-EXEC: btest-bg-run sender bro --pseudo-realtime %INPUT ../sender.bro +# @TEST-EXEC: btest-bg-run sender bro -b --pseudo-realtime %INPUT ../sender.bro # @TEST-EXEC: sleep 1 -# @TEST-EXEC: btest-bg-run receiver bro --pseudo-realtime %INPUT ../receiver.bro +# @TEST-EXEC: btest-bg-run receiver bro -b --pseudo-realtime %INPUT ../receiver.bro # @TEST-EXEC: sleep 1 -# @TEST-EXEC: btest-bg-wait -k 10 +# @TEST-EXEC: btest-bg-wait 15 # @TEST-EXEC: btest-diff sender/test.log # @TEST-EXEC: btest-diff sender/test.failure.log # @TEST-EXEC: btest-diff sender/test.success.log @@ -41,10 +41,10 @@ event bro_init() @TEST-START-FILE sender.bro -module Test; - @load frameworks/communication/listen +module Test; + function fail(rec: Log): bool { return rec$status != "success"; @@ -66,14 +66,27 @@ event remote_connection_handshake_done(p: event_peer) Log::write(Test::LOG, [$t=network_time(), $id=cid, $status="failure", $country="MX"]); disconnect(p); } + +event remote_connection_closed(p: event_peer) + { + terminate(); + } + @TEST-END-FILE @TEST-START-FILE receiver.bro ##### +@load base/frameworks/communication + redef Communication::nodes += { ["foo"] = [$host = 127.0.0.1, $connect=T, $request_logs=T] }; +event remote_connection_closed(p: event_peer) + { + terminate(); + } + @TEST-END-FILE diff --git a/testing/btest/scripts/base/frameworks/logging/rotate-custom.bro b/testing/btest/scripts/base/frameworks/logging/rotate-custom.bro index 8a7f16d182..c0f0ef8643 100644 --- a/testing/btest/scripts/base/frameworks/logging/rotate-custom.bro +++ b/testing/btest/scripts/base/frameworks/logging/rotate-custom.bro @@ -1,6 +1,7 @@ # -#@TEST-EXEC: bro -b -r ${TRACES}/rotation.trace %INPUT | egrep "test|test2" | sort >out -# @TEST-EXEC: for i in `ls test*.log | sort`; do printf '> %s\n' $i; cat $i; done | sort | uniq >>out +# @TEST-EXEC: bro -b -r ${TRACES}/rotation.trace %INPUT | egrep "test|test2" | sort >out.tmp +# @TEST-EXEC: cat out.tmp pp.log | sort >out +# @TEST-EXEC: for i in `ls test*.log | sort`; do printf '> %s\n' $i; cat $i; done | sort | $SCRIPTS/diff-remove-timestamps | uniq >>out # @TEST-EXEC: btest-diff out # @TEST-EXEC: TEST_DIFF_CANONIFIER=$SCRIPTS/diff-sort btest-diff .stderr @@ -19,7 +20,7 @@ export { } redef Log::default_rotation_interval = 1hr; -redef Log::default_rotation_postprocessor_cmd = "echo 1st"; +redef Log::default_rotation_postprocessor_cmd = "echo 1st >>pp.log"; function custom_rotate(info: Log::RotationInfo) : bool { diff --git a/testing/btest/scripts/base/frameworks/logging/writer-path-conflict.bro b/testing/btest/scripts/base/frameworks/logging/writer-path-conflict.bro index be6c0e9e9e..908fb43c72 100644 --- a/testing/btest/scripts/base/frameworks/logging/writer-path-conflict.bro +++ b/testing/btest/scripts/base/frameworks/logging/writer-path-conflict.bro @@ -1,6 +1,9 @@ # @TEST-EXEC: bro -C -r $TRACES/wikipedia.trace %INPUT # @TEST-EXEC: btest-diff reporter.log # @TEST-EXEC: btest-diff http.log +# @TEST-EXEC: btest-diff http-2.log +# @TEST-EXEC: btest-diff http-3.log +# @TEST-EXEC: btest-diff http-2-2.log @load base/protocols/http @@ -8,7 +11,14 @@ event bro_init() { # Both the default filter for the http stream and this new one will # attempt to have the same writer write to path "http", which will - # be reported as a warning and the write skipped. + # be reported as a warning and the path auto-corrected to "http-2" local filter: Log::Filter = [$name="host-only", $include=set("host")]; + # Same deal here, but should be auto-corrected to "http-3". + local filter2: Log::Filter = [$name="uri-only", $include=set("uri")]; + # Conflict between auto-correct paths needs to be corrected, too, this + # time it will be "http-2-2". + local filter3: Log::Filter = [$path="http-2", $name="status-only", $include=set("status_code")]; Log::add_filter(HTTP::LOG, filter); + Log::add_filter(HTTP::LOG, filter2); + Log::add_filter(HTTP::LOG, filter3); } diff --git a/testing/btest/scripts/base/frameworks/metrics/basic-cluster.bro b/testing/btest/scripts/base/frameworks/metrics/basic-cluster.bro index 09479b7a2f..89ae5bf79f 100644 --- a/testing/btest/scripts/base/frameworks/metrics/basic-cluster.bro +++ b/testing/btest/scripts/base/frameworks/metrics/basic-cluster.bro @@ -5,13 +5,13 @@ # @TEST-EXEC: sleep 1 # @TEST-EXEC: btest-bg-run worker-1 BROPATH=$BROPATH:.. CLUSTER_NODE=worker-1 bro %INPUT # @TEST-EXEC: btest-bg-run worker-2 BROPATH=$BROPATH:.. CLUSTER_NODE=worker-2 bro %INPUT -# @TEST-EXEC: btest-bg-wait -k 10 +# @TEST-EXEC: btest-bg-wait 30 # @TEST-EXEC: btest-diff manager-1/metrics.log @TEST-START-FILE cluster-layout.bro redef Cluster::nodes = { - ["manager-1"] = [$node_type=Cluster::MANAGER, $ip=127.0.0.1, $p=37757/tcp, $workers=set("worker-1")], - ["proxy-1"] = [$node_type=Cluster::PROXY, $ip=127.0.0.1, $p=37758/tcp, $manager="manager-1", $workers=set("worker-1")], + ["manager-1"] = [$node_type=Cluster::MANAGER, $ip=127.0.0.1, $p=37757/tcp, $workers=set("worker-1", "worker-2")], + ["proxy-1"] = [$node_type=Cluster::PROXY, $ip=127.0.0.1, $p=37758/tcp, $manager="manager-1", $workers=set("worker-1", "worker-2")], ["worker-1"] = [$node_type=Cluster::WORKER, $ip=127.0.0.1, $p=37760/tcp, $manager="manager-1", $proxy="proxy-1", $interface="eth0"], ["worker-2"] = [$node_type=Cluster::WORKER, $ip=127.0.0.1, $p=37761/tcp, $manager="manager-1", $proxy="proxy-1", $interface="eth1"], }; @@ -28,11 +28,51 @@ event bro_init() &priority=5 Metrics::add_filter(TEST_METRIC, [$name="foo-bar", $break_interval=3secs]); - - if ( Cluster::local_node_type() == Cluster::WORKER ) + } + +event remote_connection_closed(p: event_peer) + { + terminate(); + } + +global ready_for_data: event(); + +redef Cluster::manager2worker_events += /ready_for_data/; + +@if ( Cluster::local_node_type() == Cluster::WORKER ) + +event ready_for_data() + { + Metrics::add_data(TEST_METRIC, [$host=1.2.3.4], 3); + Metrics::add_data(TEST_METRIC, [$host=6.5.4.3], 2); + Metrics::add_data(TEST_METRIC, [$host=7.2.1.5], 1); + } + +@endif + +@if ( Cluster::local_node_type() == Cluster::MANAGER ) + +global n = 0; +global peer_count = 0; + +event Metrics::log_metrics(rec: Metrics::Info) + { + n = n + 1; + if ( n == 3 ) { - Metrics::add_data(TEST_METRIC, [$host=1.2.3.4], 3); - Metrics::add_data(TEST_METRIC, [$host=6.5.4.3], 2); - Metrics::add_data(TEST_METRIC, [$host=7.2.1.5], 1); + terminate_communication(); + terminate(); } } + +event remote_connection_handshake_done(p: event_peer) + { + print p; + peer_count = peer_count + 1; + if ( peer_count == 3 ) + { + event ready_for_data(); + } + } + +@endif diff --git a/testing/btest/scripts/base/frameworks/metrics/cluster-intermediate-update.bro b/testing/btest/scripts/base/frameworks/metrics/cluster-intermediate-update.bro index 654e42976a..db2c7e9f5d 100644 --- a/testing/btest/scripts/base/frameworks/metrics/cluster-intermediate-update.bro +++ b/testing/btest/scripts/base/frameworks/metrics/cluster-intermediate-update.bro @@ -5,7 +5,7 @@ # @TEST-EXEC: sleep 1 # @TEST-EXEC: btest-bg-run worker-1 BROPATH=$BROPATH:.. CLUSTER_NODE=worker-1 bro %INPUT # @TEST-EXEC: btest-bg-run worker-2 BROPATH=$BROPATH:.. CLUSTER_NODE=worker-2 bro %INPUT -# @TEST-EXEC: btest-bg-wait -k 10 +# @TEST-EXEC: btest-bg-wait 20 # @TEST-EXEC: btest-diff manager-1/notice.log @TEST-START-FILE cluster-layout.bro @@ -37,6 +37,21 @@ event bro_init() &priority=5 $log=T]); } +event remote_connection_closed(p: event_peer) + { + terminate(); + } + +@if ( Cluster::local_node_type() == Cluster::MANAGER ) + +event Notice::log_notice(rec: Notice::Info) + { + terminate_communication(); + terminate(); + } + +@endif + @if ( Cluster::local_node_type() == Cluster::WORKER ) event do_metrics(i: count) diff --git a/testing/btest/scripts/base/frameworks/notice/cluster.bro b/testing/btest/scripts/base/frameworks/notice/cluster.bro index 8d54a27eaf..47932edb8e 100644 --- a/testing/btest/scripts/base/frameworks/notice/cluster.bro +++ b/testing/btest/scripts/base/frameworks/notice/cluster.bro @@ -2,9 +2,9 @@ # # @TEST-EXEC: btest-bg-run manager-1 BROPATH=$BROPATH:.. CLUSTER_NODE=manager-1 bro %INPUT # @TEST-EXEC: btest-bg-run proxy-1 BROPATH=$BROPATH:.. CLUSTER_NODE=proxy-1 bro %INPUT -# @TEST-EXEC: sleep 1 +# @TEST-EXEC: sleep 2 # @TEST-EXEC: btest-bg-run worker-1 BROPATH=$BROPATH:.. CLUSTER_NODE=worker-1 bro %INPUT -# @TEST-EXEC: btest-bg-wait -k 10 +# @TEST-EXEC: btest-bg-wait 20 # @TEST-EXEC: btest-diff manager-1/notice.log @TEST-START-FILE cluster-layout.bro @@ -21,13 +21,44 @@ redef enum Notice::Type += { Test_Notice, }; +event remote_connection_closed(p: event_peer) + { + terminate(); + } + +global ready: event(); + +redef Cluster::manager2worker_events += /ready/; + event delayed_notice() { if ( Cluster::node == "worker-1" ) NOTICE([$note=Test_Notice, $msg="test notice!"]); } -event bro_init() +@if ( Cluster::local_node_type() == Cluster::WORKER ) + +event ready() { schedule 1secs { delayed_notice() }; } + +@endif + +@if ( Cluster::local_node_type() == Cluster::MANAGER ) + +global peer_count = 0; + +event remote_connection_handshake_done(p: event_peer) + { + peer_count = peer_count + 1; + if ( peer_count == 2 ) + event ready(); + } + +event Notice::log_notice(rec: Notice::Info) + { + terminate_communication(); + } + +@endif diff --git a/testing/btest/scripts/base/frameworks/notice/suppression-cluster.bro b/testing/btest/scripts/base/frameworks/notice/suppression-cluster.bro index b812c6451d..5010da82cc 100644 --- a/testing/btest/scripts/base/frameworks/notice/suppression-cluster.bro +++ b/testing/btest/scripts/base/frameworks/notice/suppression-cluster.bro @@ -2,10 +2,10 @@ # # @TEST-EXEC: btest-bg-run manager-1 BROPATH=$BROPATH:.. CLUSTER_NODE=manager-1 bro %INPUT # @TEST-EXEC: btest-bg-run proxy-1 BROPATH=$BROPATH:.. CLUSTER_NODE=proxy-1 bro %INPUT -# @TEST-EXEC: sleep 1 +# @TEST-EXEC: sleep 2 # @TEST-EXEC: btest-bg-run worker-1 BROPATH=$BROPATH:.. CLUSTER_NODE=worker-1 bro %INPUT # @TEST-EXEC: btest-bg-run worker-2 BROPATH=$BROPATH:.. CLUSTER_NODE=worker-2 bro %INPUT -# @TEST-EXEC: btest-bg-wait -k 10 +# @TEST-EXEC: btest-bg-wait 20 # @TEST-EXEC: btest-diff manager-1/notice.log @TEST-START-FILE cluster-layout.bro @@ -23,6 +23,15 @@ redef enum Notice::Type += { Test_Notice, }; +event remote_connection_closed(p: event_peer) + { + terminate(); + } + +global ready: event(); + +redef Cluster::manager2worker_events += /ready/; + event delayed_notice() { NOTICE([$note=Test_Notice, @@ -30,10 +39,33 @@ event delayed_notice() $identifier="this identifier is static"]); } -event bro_init() &priority=5 - { +@if ( Cluster::local_node_type() == Cluster::WORKER ) + +event ready() + { if ( Cluster::node == "worker-1" ) schedule 4secs { delayed_notice() }; if ( Cluster::node == "worker-2" ) schedule 1secs { delayed_notice() }; + } + +event Notice::suppressed(n: Notice::Info) + { + if ( Cluster::node == "worker-1" ) + terminate_communication(); } + +@endif + +@if ( Cluster::local_node_type() == Cluster::MANAGER ) + +global peer_count = 0; + +event remote_connection_handshake_done(p: event_peer) + { + peer_count = peer_count + 1; + if ( peer_count == 3 ) + event ready(); + } + +@endif diff --git a/testing/btest/scripts/base/frameworks/reporter/disable-stderr.bro b/testing/btest/scripts/base/frameworks/reporter/disable-stderr.bro new file mode 100644 index 0000000000..b1afb99b5c --- /dev/null +++ b/testing/btest/scripts/base/frameworks/reporter/disable-stderr.bro @@ -0,0 +1,13 @@ +# @TEST-EXEC: bro %INPUT +# @TEST-EXEC: TEST_DIFF_CANONIFIER=$SCRIPTS/diff-remove-abspath btest-diff .stderr +# @TEST-EXEC: TEST_DIFF_CANONIFIER="$SCRIPTS/diff-remove-abspath | $SCRIPTS/diff-remove-timestamps" btest-diff reporter.log + +redef Reporter::warnings_to_stderr = F; +redef Reporter::errors_to_stderr = F; + +global test: table[count] of string = {}; + +event bro_init() + { + print test[3]; + } diff --git a/testing/btest/scripts/base/frameworks/reporter/stderr.bro b/testing/btest/scripts/base/frameworks/reporter/stderr.bro new file mode 100644 index 0000000000..ef01c9fdf9 --- /dev/null +++ b/testing/btest/scripts/base/frameworks/reporter/stderr.bro @@ -0,0 +1,10 @@ +# @TEST-EXEC: bro %INPUT +# @TEST-EXEC: TEST_DIFF_CANONIFIER=$SCRIPTS/diff-remove-abspath btest-diff .stderr +# @TEST-EXEC: TEST_DIFF_CANONIFIER="$SCRIPTS/diff-remove-abspath | $SCRIPTS/diff-remove-timestamps" btest-diff reporter.log + +global test: table[count] of string = {}; + +event bro_init() + { + print test[3]; + } diff --git a/testing/btest/signatures/bad-eval-condition.bro b/testing/btest/signatures/bad-eval-condition.bro new file mode 100644 index 0000000000..34997b1124 --- /dev/null +++ b/testing/btest/signatures/bad-eval-condition.bro @@ -0,0 +1,22 @@ +# @TEST-EXEC-FAIL: bro -r $TRACES/ftp-ipv4.trace %INPUT +# @TEST-EXEC: btest-diff .stderr + +@load-sigs blah.sig + +@TEST-START-FILE blah.sig +signature blah + { + ip-proto == tcp + src-port == 21 + payload /.*/ + eval mark_conn + } +@TEST-END-FILE + +# wrong function signature for use with signature 'eval' conditions +# needs to be reported +function mark_conn(state: signature_state): bool + { + add state$conn$service["blah"]; + return T; + } diff --git a/testing/btest/signatures/eval-condition.bro b/testing/btest/signatures/eval-condition.bro new file mode 100644 index 0000000000..f3f1171da6 --- /dev/null +++ b/testing/btest/signatures/eval-condition.bro @@ -0,0 +1,20 @@ +# @TEST-EXEC: bro -r $TRACES/ftp-ipv4.trace %INPUT +# @TEST-EXEC: btest-diff conn.log + +@load-sigs blah.sig + +@TEST-START-FILE blah.sig +signature blah + { + ip-proto == tcp + src-port == 21 + payload /.*/ + eval mark_conn + } +@TEST-END-FILE + +function mark_conn(state: signature_state, data: string): bool + { + add state$conn$service["blah"]; + return T; + } diff --git a/testing/btest/core/load-sigs.bro b/testing/btest/signatures/load-sigs.bro similarity index 100% rename from testing/btest/core/load-sigs.bro rename to testing/btest/signatures/load-sigs.bro diff --git a/testing/external/scripts/testing-setup.bro b/testing/external/scripts/testing-setup.bro index fa5664a877..4b4d110864 100644 --- a/testing/external/scripts/testing-setup.bro +++ b/testing/external/scripts/testing-setup.bro @@ -1,6 +1,12 @@ # Sets some testing specific options. @ifdef ( SMTP::never_calc_md5 ) - # MDD5s can depend on libmagic output. + # MDD5s can depend on libmagic output. redef SMTP::never_calc_md5 = T; @endif + +@ifdef ( LogElasticSearch::server_host ) + # Set to empty so that logs-to-elasticsearch.bro doesn't try to setup + #log forwarding to ES. + redef LogElasticSearch::server_host = ""; +@endif diff --git a/testing/scripts/diff-remove-mime-types b/testing/scripts/diff-remove-mime-types index fb447a9989..b8cc3d1e6d 100755 --- a/testing/scripts/diff-remove-mime-types +++ b/testing/scripts/diff-remove-mime-types @@ -3,20 +3,27 @@ # A diff canonifier that removes all MIME types because libmagic output # can differ between installations. -BEGIN { FS="\t"; OFS="\t"; column = -1; } +BEGIN { FS="\t"; OFS="\t"; type_col = -1; desc_col = -1 } /^#fields/ { for ( i = 2; i < NF; ++i ) + { if ( $i == "mime_type" ) - column = i-1; + type_col = i-1; + if ( $i == "mime_desc" ) + desc_col = i-1; + } } -column >= 0 { - if ( $column != "-" ) +function remove_mime (n) { + if ( n >= 0 && $n != "-" ) # Mark that it's set, but ignore content. - $column = "+"; + $n = "+" } +remove_mime(type_col) +remove_mime(desc_col) + { print; } diff --git a/testing/scripts/diff-remove-timestamps b/testing/scripts/diff-remove-timestamps index 84bd21aa60..138b901743 100755 --- a/testing/scripts/diff-remove-timestamps +++ b/testing/scripts/diff-remove-timestamps @@ -11,4 +11,4 @@ fi # The first sed uses a "basic" regexp, the 2nd a "modern:. sed 's/[0-9]\{10\}\.[0-9]\{2,8\}/XXXXXXXXXX.XXXXXX/g' | \ -$sed 's/^#(start|end).(19|20)..-..-..-..-..-..$/#\1 XXXX-XX-XX-XX-XX-XX/g' +$sed 's/^#(open|close).(19|20)..-..-..-..-..-..$/#\1 XXXX-XX-XX-XX-XX-XX/g'