diff --git a/CHANGES b/CHANGES index 51147f5306..159e7d9a9a 100644 --- a/CHANGES +++ b/CHANGES @@ -1,14 +1,389 @@ + +2.0-306 | 2012-04-24 14:37:00 -0700 + + * Add further TLS extension values "extended_random" and + "heartbeat". (Seth Hall) + + * Fix problem with extracting FTP passwords and add "ftpuser" as + another anonymous username. (Seth Hall, discovered by Patrik + Lundin). + +2.0-303 | 2012-04-19 10:01:06 -0700 + + * Changes related to ICMPv6 Neighbor Discovery messages. (Jon Siwek) + + - The 'icmp_conn' record now contains an 'hlim' field since hop limit + in the IP header is an interesting field for at least these ND + messages. + + - Fixed and extended 'icmp_router_advertisement' event parameters. + + - Changed 'icmp_neighbor_advertisement' event parameters to add + more of the known boolean flags. + +2.0-301 | 2012-04-17 17:58:55 -0700 + + * Bro now support ICMPv6. (Matti Mantere, Jon Siwek, Robin Sommer, + Daniel Thayer). + + Overall, Bro now raises the following ICMP events for v4 and v6 as + appropiate: + + event icmp_sent(c: connection, icmp: icmp_conn); + event icmp_echo_request(c: connection, icmp: icmp_conn, id: count, seq: count, payload: string); + event icmp_echo_reply(c: connection, icmp: icmp_conn, id: count, seq: count, payload: string); + event icmp_error_message(c: connection, icmp: icmp_conn, code: count, context: icmp_context); + event icmp_unreachable(c: connection, icmp: icmp_conn, code: count, context: icmp_context); + event icmp_packet_too_big(c: connection, icmp: icmp_conn, code: count, context: icmp_context); + event icmp_time_exceeded(c: connection, icmp: icmp_conn, code: count, context: icmp_context); + event icmp_parameter_problem(c: connection, icmp: icmp_conn, code: count, context: icmp_context); + event icmp_router_solicitation(c: connection, icmp: icmp_conn); + event icmp_router_advertisement(c: connection, icmp: icmp_conn, hop_limit: count, managed: bool, router_lifetime: count, reachable_time: interval, retrans_timer: interval); + event icmp_neighbor_solicitation(c: connection, icmp: icmp_conn, tgt:addr); + event icmp_neighbor_advertisement(c: connection, icmp: icmp_conn, tgt:addr); + event icmp_redirect(c: connection, icmp: icmp_conn, tgt: addr, dest: addr); + + The `icmp_conn` record got a new boolean field 'v6' that indicates + whether the ICMP message is v4 or v6. + + This change also includes further low-level work on existing IP + and ICMP code, including a reorganization of how ICMPv4 is + handled. + +2.0-281 | 2012-04-17 17:40:39 -0700 + + * Small updates for the bittorrent analyzer to support 64bit types + in binpac. (Seth Hall) + + * Removed the attempt at bittorrent resynchronization. (Seth Hall) + +2.0-276 | 2012-04-17 17:35:56 -0700 + + * Add more support for 's that lack some structure + definitions. (Jon Siwek) + +2.0-273 | 2012-04-16 18:08:56 -0700 + + * Removing QR flag from DNS log in response, which should not have + been there in the first place. (Seth Hall) + + * Sync up patricia.c/h with pysubnettree repo. (Daniel Thayer) + + * Adding missing leak groups to a couple tests. Also activating leak + checking for proxy in basic-cluster test. (Robin Sommer) + +2.0-267 | 2012-04-09 17:47:28 -0700 + + * Add support for mobile IPv6 Mobility Header (RFC 6275). (Jon + Siwek) + + - Enabled through a new --enable-mobile-ipv6 configure-time + option. If not enabled, the mobility header (routing type 2) and + Home Address Destination option are ignored. + + - Accessible at script-layer through 'mobile_ipv6_message' event. + + * Refactor IP_Hdr routing header handling, add MobileIPv6 Home + Address handling. Packets that use the Home Address Destination + option use that option's address as the connection's originator. + (Jon Siwek) + + * Revert TCP checksumming to cache common data, like it did before. + (Jon Siwek) + + * Improve handling of IPv6 routing type 0 extension headers. (Jon + Siwek) + + - flow_weird event with name argument value of "routing0_hdr" is raised + for packets containing an IPv6 routing type 0 header because this + type of header is now deprecated according to RFC 5095. + + - Packets with a routing type 0 header and non-zero segments left + now use the last address in that header in order to associate + with a connection/flow and for calculating TCP/UDP checksums. + + - Added a set of IPv4/IPv6 TCP/UDP checksum unit tests (Jon Siwek) + + * Fix table expiry for values assigned in bro_init() when reading + live. (Jon Siwek) + +2.0-257 | 2012-04-05 15:32:43 -0700 + + * Fix CMake from warning about unused ENABLE_PERFTOOLS_DEBUG + variable. (Jon Siwek) + + * Fix handling of IPv6 atomic fragments. (Jon Siwek) + + * Fix that prevents Bro processes that do neither local logging nor + request remote logs from spawning threads. (Robin Sommer) + + * Fixing perftools-debug support. (Robin Sommer) + + * Reverting SocketComm change tuning I/O behaviour. (Robin Sommer) + + * Adding notice_policy.log canonification for external tests. (Robin Sommer) + + +2.0-245 | 2012-04-04 17:25:20 -0700 + + * Internal restructuring of the logging framework: we now spawn + threads doing the I/O. From a user's perspective not much should + change, except that the OS may now show a bunch of Bro threads. + (Gilbert Clark and Robin Sommer). + + * When building Bro, we now always link in tcmalloc if it's found at + configure time. If it's installed but not picked up, + --with-perftools may help. (Robin Sommer) + + * Renaming the configure option --enable-perftools to + --enable-perftool-debug to indicate that the switch is only + relevant for debugging the heap. It's not needed to pick up + tcmalloc for better performance. (Robin Sommer) + +2.0-184 | 2012-03-28 15:11:11 -0700 + + * Improve handling of IPv6 Routing Type 0 headers. (Jon Siwek) + + - For RH0 headers with non-zero segments left, a + "routing0_segleft" flow_weird event is raised (with a + destination indicating the last address in the routing header), + and an "rh0_segleft" event can also be handled if the other + contents of the packet header are of interest. No further + analysis is done as the complexity required to correctly + identify destination endpoints of connections doesn't seem worth + it as RH0 has been deprecated by RFC 5095. + + - For RH0 headers without any segments left, a "routing0_header" + flow_weird event is raised, but further analysis still occurs as + normal. + +2.0-182 | 2012-03-28 15:01:57 -0700 + + * Remove dead tcp_checksum function from net_util. (Jon Siwek) + + * Change routing0_data_to_addrs BIF to return vector of addresses. + The order of addresses in type 0 routing headers is + interesting/important. (Jon Siwek) + + +2.0-179 | 2012-03-23 17:43:31 -0700 + + * Remove the default "tcp or udp or icmp" filter. In default mode, + Bro would load the packet filter script framework which installs a + filter that allows all packets, but in bare mode (the -b option), + this old filter would not follow IPv6 protocol chains and thus + filter out packets with extension headers. (Jon Siwek) + + * Update PacketFilter/Discarder code for IP version independence. + (Jon Siwek) + + * Fix some IPv6 header related bugs. (Jon Siwek) + + * Add IPv6 fragment reassembly. (Jon Siwek) + + * Add handling for IPv6 extension header chains. Addresses #531. + (Jon Siwek) + + - The script-layer 'pkt_hdr' type is extended with a new 'ip6' field + representing the full IPv6 header chain. + + - The 'new_packet' event is now raised for IPv6 packets. Addresses + #523. + + - A new event called 'ipv6_ext_header' is raised for any IPv6 + packet containing extension headers. + + - A new event called 'esp_packet' is raised for any packets using + ESP ('new_packet' and 'ipv6_ext_header' events provide + connection info, but that info can't be provided here since the + upper-layer payload is encrypted). + + - The 'unknown_protocol' weird is now raised more reliably when + Bro sees a transport protocol or IPv6 extension header it can't + handle. Addresses #522. + + * Add unit tests for IPv6 fragment reassembly, ipv6_ext_headers and + esp_packet events. (Jon Siwek) + + * Adapt FreeBSD's inet_ntop implementation for internal use. Now we + get consistent text representations of IPv6 addresses across + platforms. (Jon Siwek) + + * Update documentation for new syntax of IPv6 literals. (Jon Siwek) + + +2.0-150 | 2012-03-13 16:16:22 -0700 + + * Changing the regular expression to allow Site::local_nets in + signatures. (Julien Sentier) + + * Removing a line of dead code. Found by . Closes #786. (Julien + Sentier) + +2.0-146 | 2012-03-13 15:39:38 -0700 + + * Change IPv6 literal constant syntax to require encasing square + brackets. (Jon Siwek) + +2.0-145 | 2012-03-09 15:10:35 -0800 + + * Remove the match expression. 'match' and 'using' are no longer + keywords. Addressed #753. (Jon Siwek) + +2.0-143 | 2012-03-09 15:07:42 -0800 + + * Fix a BRO_PROFILER_FILE/mkstemp portability issue. Addresses #794. + (Jon Siwek) + +2.0-139 | 2012-03-02 09:33:04 -0800 + + * Changes to how script coverage integrates with test suites. (Jon Siwek) + + - BRO_PROFILER_FILE now passes .X* templated filenames to mkstemp + for generating unique coverage state files. + + - Rearranging Makefile targets. The general rule is that if the + all/brief target fails out due to a test failure, then the dependent + coverage target won't run, but can still be invoked directly later. + (e.g. make brief || make coverage) + + * Standardized on the &default function for SSL constants. (Seth + Hall) + + * Adding btest group "leaks" to leak tests. (Robin Sommer) + + * Adding btest group "comm" to communication tests for parallelizing + execution with new btest version. (Robin Sommer) + + * Sorting all output for diffing in the external tests. (Robin + Sommer) + + * Cleaned up dead code from the old SSL analyzers. Reported by + Julien Sentier. (Seth Hall) + + * Update/add tests for broccoli IPv6 addr/subnet support. Addresses + #448. (Jon Siwek) + + * Remove connection compressor. Addresses #559. (Jon Siwek) + + * Refactor IP_Hdr class ctors. Addresses #532. (Jon Siwek) + + +2.0-121 | 2012-02-24 16:34:17 -0800 + + * A number of smaller memory fixes and code cleanups. (Julien + Sentier) + + * Add to_subnet bif. Fixes #782). (Jon Siwek) + + * Fix IPAddr::Mask/ReverseMask not allowing argument of 0. (Jon + Siwek) + + * Refactor IPAddr v4 initialization from string. Fixes #775. (Jon Siwek) + + * Parse the dotted address string directly instead of canonicalizing + and passing to inet_pton. (Jon Siwek) + + +2.0-108 | 2012-02-24 15:21:07 -0800 + + * Refactoring a number of usages of new IPAddr class. (Jon Siwek) + + * Fixed a bug in remask_addr bif. (Jon Siwek) + +2.0-106 | 2012-02-24 15:02:20 -0800 + + * Raise minimum required CMake version to 2.6.3. (Jon Siwek) + +2.0-104 | 2012-02-24 14:59:12 -0800 + + * Add test case for FTP over IPv4. (Daniel Thayer) + + * Fix IPv6 URLs in ftp.log. (Daniel Thayer) + + * Add a test for FTP over IPv6 (Daniel Thayer) + + * Fix parsing of FTP EPRT command and EPSV response. (Daniel Thayer) + +2.0-95 | 2012-02-22 05:27:34 -0800 + + * GeoIP installation documentation update. (Seth Hall) + + * Decrease strictness of parsing IPv4 strings into addrs. Fixes #775. (Jon Siwek) + + * Fix memory leak in DNS manager. Fixes #777. (Jon Siwek) + + * Fix IPAddr/IPPrefix serialization bugs. (Jon Siwek) + + * Fix compile error. (Jon Siwek) + +2.0-86 | 2012-02-17 15:41:06 -0800 + + * Changing ARP detection to always kick in even if no analyzer is + activated. (Robin Sommer) + + * DNS name lookups performed by Bro now also query AAAA records. + DNS_Mgr handles combining the results of the A and AAAA queries + for a given hostname such that at the scripting layer, the name + resolution can yield a set with both IPv4 and IPv6 addresses. (Jon + Siwek) + + * Add counts_to_addr and addr_to_counts conversion BIFs. (Jon Siwek) + + * Change HashKey threshold for using H3 to 36 bytes. (Jon Siwek) + + * Remove mention of --enable-brov6 in docs. (Daniel Thayer) + + * Remove --enable-brov6 from configure usage text (Daniel Thayer) + + * Add a test and baseline for addr_to_ptr_name BiF. (Daniel Thayer) + + * Adding a test and baseline for ptr_name_to_addr BiF. (Seth Hall) + + * Fix the ptr_name_to_addr BiF to work with IPv6 (Daniel Thayer) + + * Fix a memory leak that perftools now complains about. (Jon Siwek) + + * Remove --enable-brov6 flag, IPv6 now supported by default. (Jon Siwek) + + Some script-layer changes of note: + + - dns_AAAA_reply event signature changed: the string representation + of an IPv6 addr is easily derived from the addr value, it doesn't + need to be another parameter. This event also now generated directly + by the DNS analyzer instead of being "faked" into a dns_A_reply event. + + - Removed addr_to_count BIF. It used to return the host-order + count representation of IPv4 addresses only. To make it more + generic, we might later add a BIF to return a vector of counts + in order to support IPv6. + + - Changed the result of enclosing addr variables in vertical pipes + (e.g. |my_addr|) to return the bit-width of the address type which + is 128 for IPv6 and 32 for IPv4. It used to function the same + way as addr_to_count mentioned above. + + - Remove bro_has_ipv6 BIF + +2.0-57 | 2012-02-10 00:02:35 -0800 + + * Fix typos in the documentation. (Daniel Thayer) + + * Fix compiler warning about Brofiler ctor init list order. (Jon Siwek) + + * Fix missing optional field access in webapp signature_match handler. (Jon Siwek) + 2.0-41 | 2012-02-03 04:10:53 -0500 - * Updates to the Software framework to simplify the API. - (Bernhard Amann) + * Updates to the Software framework to simplify the API. (Bernhard + Amann) 2.0-40 | 2012-02-03 01:55:27 -0800 - * Fix typos in documentation. (Daniel Thayer) - - * Fix sorting of lines in Brofiler coverage.log. (Daniel Thayer) - + * Fix typos in documentation. (Daniel Thayer) + + * Fix sorting of lines in Brofiler coverage.log. (Daniel Thayer) + 2.0-38 | 2012-01-31 11:50:53 -0800 * Canonify sorting of lines in Brofiler coverage.log. (Daniel diff --git a/CMakeLists.txt b/CMakeLists.txt index 241a5b29d2..d27fa2d40b 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -1,5 +1,5 @@ project(Bro C CXX) -cmake_minimum_required(VERSION 2.6 FATAL_ERROR) +cmake_minimum_required(VERSION 2.6.3 FATAL_ERROR) include(cmake/CommonCMakeConfig.cmake) ######################################################################## @@ -89,15 +89,29 @@ if (LIBGEOIP_FOUND) endif () set(USE_PERFTOOLS false) -if (ENABLE_PERFTOOLS) - find_package(GooglePerftools) - if (GOOGLEPERFTOOLS_FOUND) - set(USE_PERFTOOLS true) - include_directories(BEFORE ${GooglePerftools_INCLUDE_DIR}) +set(USE_PERFTOOLS_DEBUG false) + +find_package(GooglePerftools) + +if (GOOGLEPERFTOOLS_FOUND) + include_directories(BEFORE ${GooglePerftools_INCLUDE_DIR}) + set(USE_PERFTOOLS true) + + if (ENABLE_PERFTOOLS_DEBUG) + # Enable heap debugging with perftools. + set(USE_PERFTOOLS_DEBUG true) + list(APPEND OPTLIBS ${GooglePerftools_LIBRARIES_DEBUG}) + else () + # Link in tcmalloc for better performance. list(APPEND OPTLIBS ${GooglePerftools_LIBRARIES}) endif () endif () +if (ENABLE_PERFTOOLS_DEBUG) + # Just a no op to prevent CMake from complaining about manually-specified + # ENABLE_PERFTOOLS_DEBUG not being used if google perftools weren't found +endif () + set(brodeps ${BinPAC_LIBRARY} ${PCAP_LIBRARY} @@ -183,6 +197,7 @@ message( "\n" "\nGeoIP: ${USE_GEOIP}" "\nGoogle perftools: ${USE_PERFTOOLS}" + "\n debugging: ${USE_PERFTOOLS_DEBUG}" "\n" "\n================================================================\n" ) diff --git a/INSTALL b/INSTALL index 73b824b2b7..d4aa93d11f 100644 --- a/INSTALL +++ b/INSTALL @@ -8,7 +8,7 @@ Prerequisites Bro relies on the following libraries and tools, which need to be installed before you begin: - * CMake 2.6 or greater http://www.cmake.org + * CMake 2.6.3 or greater http://www.cmake.org * Libpcap (headers and libraries) http://www.tcpdump.org diff --git a/NEWS b/NEWS index 1a257ce18f..c48b243552 100644 --- a/NEWS +++ b/NEWS @@ -5,6 +5,51 @@ Release Notes This document summarizes the most important changes in the current Bro release. For a complete list of changes, see the ``CHANGES`` file. + +Bro 2.1 +------- + +- Dependencies: + + * Bro now requires CMake >= 2.6.3. + + * Bro now links in tcmalloc (part of Google perftools) if found at + configure time. Doing so can significantly improve memory and + CPU use. + +- Bro now supports IPv6 out of the box; the configure switch + --enable-brov6 is gone. + +- DNS name lookups performed by Bro now also query AAAA records. The + results of the A and AAAA queries for a given hostname are combined + such that at the scripting layer, the name resolution can yield a + set with both IPv4 and IPv6 addresses. + +- The connection compressor was already deprecated in 2.0 and has now + been removed from the code base. + +- We removed the "match" statement, which was no longer used by any of + the default scripts, nor was it likely to be used by anybody anytime + soon. With that, "match" and "using" are no longer reserved keywords. + +- The syntax for IPv6 literals changed from "2607:f8b0:4009:802::1012" + to "[2607:f8b0:4009:802::1012]". + +- Bro now spawn threads for doing its logging. From a user's + perspective not much should change, except that the OS may now show + a bunch of Bro threads. + +- We renamed the configure option --enable-perftools to + --enable-perftool-debug to indicate that the switch is only relevant + for debugging the heap. + +- Bro's ICMP analyzer now handles both IPv4 and IPv6 messages with a + joint set of events. The `icmp_conn` record got a new boolean field + 'v6' that indicates whether the ICMP message is v4 or v6. + + +TODO: Extend. + Bro 2.0 ------- @@ -61,4 +106,3 @@ final release are: - diff --git a/VERSION b/VERSION index 9d6521772a..dc9d148493 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -2.0-41 +2.0-306 diff --git a/aux/binpac b/aux/binpac index 43308aab47..71c37019bc 160000 --- a/aux/binpac +++ b/aux/binpac @@ -1 +1 @@ -Subproject commit 43308aab47a3357ca1885e1b6954154a2744d821 +Subproject commit 71c37019bc371eb7863fb6aa47a7daa4540f4f1f diff --git a/aux/bro-aux b/aux/bro-aux index 139cc2e1e0..d885987e79 160000 --- a/aux/bro-aux +++ b/aux/bro-aux @@ -1 +1 @@ -Subproject commit 139cc2e1e049c4e1cc7e95f20866102be1d3d599 +Subproject commit d885987e7968669e34504b0403ac89bd13928e9a diff --git a/aux/broccoli b/aux/broccoli index 930e7c7822..55f368b0ad 160000 --- a/aux/broccoli +++ b/aux/broccoli @@ -1 +1 @@ -Subproject commit 930e7c78221929849086a578308e2fdc99ac3fb8 +Subproject commit 55f368b0ad283b2e7d68ef72922b5d9683e2a880 diff --git a/aux/broctl b/aux/broctl index e908ba686d..ff35c3c144 160000 --- a/aux/broctl +++ b/aux/broctl @@ -1 +1 @@ -Subproject commit e908ba686dceb56065bdf569c18dd0f67f662f6b +Subproject commit ff35c3c144885902c898bf8b47e351c7b8d55e10 diff --git a/aux/btest b/aux/btest index ee87db37b5..045a02749b 160000 --- a/aux/btest +++ b/aux/btest @@ -1 +1 @@ -Subproject commit ee87db37b520b88a55323a9767234c30b801e439 +Subproject commit 045a02749b20b3c5896497959e6fda02d060508f diff --git a/cmake b/cmake index 2cc1055770..49278736c1 160000 --- a/cmake +++ b/cmake @@ -1 +1 @@ -Subproject commit 2cc105577044a2d214124568f3f2496ed2ccbb34 +Subproject commit 49278736c1404cb8c077272b80312c947e68bf52 diff --git a/config.h.in b/config.h.in index 261cd0ccb9..b8e2cb9a88 100644 --- a/config.h.in +++ b/config.h.in @@ -1,6 +1,3 @@ -/* enable IPV6 processing */ -#cmakedefine BROv6 - /* Old libpcap versions (< 0.6.1) need defining pcap_freecode and pcap_compile_nopcap */ #cmakedefine DONT_HAVE_LIBPCAP_PCAP_FREECODE @@ -112,7 +109,10 @@ #cmakedefine HAVE_GEOIP_CITY_EDITION_REV0_V6 /* Use Google's perftools */ -#cmakedefine USE_PERFTOOLS +#cmakedefine USE_PERFTOOLS_DEBUG + +/* Analyze Mobile IPv6 traffic */ +#cmakedefine ENABLE_MOBILE_IPV6 /* Version number of package */ #define VERSION "@VERSION@" @@ -152,3 +152,47 @@ #ifndef HAVE_DLT_PPP_SERIAL #define DLT_PPP_SERIAL @DLT_PPP_SERIAL@ #endif + +/* IPv6 Next Header values defined by RFC 3542 */ +#cmakedefine HAVE_IPPROTO_HOPOPTS +#ifndef HAVE_IPPROTO_HOPOPTS +#define IPPROTO_HOPOPTS 0 +#endif +#cmakedefine HAVE_IPPROTO_IPV6 +#ifndef HAVE_IPPROTO_IPV6 +#define IPPROTO_IPV6 41 +#endif +#cmakedefine HAVE_IPPROTO_ROUTING +#ifndef HAVE_IPPROTO_ROUTING +#define IPPROTO_ROUTING 43 +#endif +#cmakedefine HAVE_IPPROTO_FRAGMENT +#ifndef HAVE_IPPROTO_FRAGMENT +#define IPPROTO_FRAGMENT 44 +#endif +#cmakedefine HAVE_IPPROTO_ESP +#ifndef HAVE_IPPROTO_ESP +#define IPPROTO_ESP 50 +#endif +#cmakedefine HAVE_IPPROTO_AH +#ifndef HAVE_IPPROTO_AH +#define IPPROTO_AH 51 +#endif +#cmakedefine HAVE_IPPROTO_ICMPV6 +#ifndef HAVE_IPPROTO_ICMPV6 +#define IPPROTO_ICMPV6 58 +#endif +#cmakedefine HAVE_IPPROTO_NONE +#ifndef HAVE_IPPROTO_NONE +#define IPPROTO_NONE 59 +#endif +#cmakedefine HAVE_IPPROTO_DSTOPTS +#ifndef HAVE_IPPROTO_DSTOPTS +#define IPPROTO_DSTOPTS 60 +#endif + +/* IPv6 options structure defined by RFC 3542 */ +#cmakedefine HAVE_IP6_OPT + +/* Common IPv6 extension structure */ +#cmakedefine HAVE_IP6_EXT diff --git a/configure b/configure index 0f74674c0f..b6af505ffe 100755 --- a/configure +++ b/configure @@ -24,11 +24,12 @@ Usage: $0 [OPTION]... [VAR=VALUE]... --prefix=PREFIX installation directory [/usr/local/bro] --scriptdir=PATH root installation directory for Bro scripts [PREFIX/share/bro] + --conf-files-dir=PATH config files installation directory [PREFIX/etc] Optional Features: --enable-debug compile in debugging mode - --enable-brov6 enable IPv6 processing - --enable-perftools use Google's perftools + --enable-mobile-ipv6 analyze mobile IPv6 features defined by RFC 6275 + --enable-perftools-debug use Google's perftools for debugging --disable-broccoli don't build or install the Broccoli library --disable-broctl don't install Broctl --disable-auxtools don't build or install auxilliary tools @@ -86,20 +87,22 @@ append_cache_entry () { # set defaults builddir=build +prefix=/usr/local/bro CMakeCacheEntries="" -append_cache_entry CMAKE_INSTALL_PREFIX PATH /usr/local/bro -append_cache_entry BRO_ROOT_DIR PATH /usr/local/bro -append_cache_entry PY_MOD_INSTALL_DIR PATH /usr/local/bro/lib/broctl -append_cache_entry BRO_SCRIPT_INSTALL_PATH STRING /usr/local/bro/share/bro +append_cache_entry CMAKE_INSTALL_PREFIX PATH $prefix +append_cache_entry BRO_ROOT_DIR PATH $prefix +append_cache_entry PY_MOD_INSTALL_DIR PATH $prefix/lib/broctl +append_cache_entry BRO_SCRIPT_INSTALL_PATH STRING $prefix/share/bro +append_cache_entry BRO_ETC_INSTALL_DIR PATH $prefix/etc append_cache_entry ENABLE_DEBUG BOOL false -append_cache_entry BROv6 BOOL false -append_cache_entry ENABLE_PERFTOOLS BOOL false +append_cache_entry ENABLE_PERFTOOLS_DEBUG BOOL false append_cache_entry BinPAC_SKIP_INSTALL BOOL true append_cache_entry BUILD_SHARED_LIBS BOOL true append_cache_entry INSTALL_AUX_TOOLS BOOL true append_cache_entry INSTALL_BROCCOLI BOOL true append_cache_entry INSTALL_BROCTL BOOL true append_cache_entry CPACK_SOURCE_IGNORE_FILES STRING +append_cache_entry ENABLE_MOBILE_IPV6 BOOL false # parse arguments while [ $# -ne 0 ]; do @@ -120,25 +123,27 @@ while [ $# -ne 0 ]; do CMakeGenerator="$optarg" ;; --prefix=*) + prefix=$optarg append_cache_entry CMAKE_INSTALL_PREFIX PATH $optarg append_cache_entry BRO_ROOT_DIR PATH $optarg append_cache_entry PY_MOD_INSTALL_DIR PATH $optarg/lib/broctl - if [ "$user_set_scriptdir" != "true" ]; then - append_cache_entry BRO_SCRIPT_INSTALL_PATH STRING $optarg/share/bro - fi ;; --scriptdir=*) append_cache_entry BRO_SCRIPT_INSTALL_PATH STRING $optarg user_set_scriptdir="true" ;; + --conf-files-dir=*) + append_cache_entry BRO_ETC_INSTALL_DIR PATH $optarg + user_set_conffilesdir="true" + ;; --enable-debug) append_cache_entry ENABLE_DEBUG BOOL true ;; - --enable-brov6) - append_cache_entry BROv6 BOOL true + --enable-mobile-ipv6) + append_cache_entry ENABLE_MOBILE_IPV6 BOOL true ;; - --enable-perftools) - append_cache_entry ENABLE_PERFTOOLS BOOL true + --enable-perftools-debug) + append_cache_entry ENABLE_PERFTOOLS_DEBUG BOOL true ;; --disable-broccoli) append_cache_entry INSTALL_BROCCOLI BOOL false @@ -183,7 +188,6 @@ while [ $# -ne 0 ]; do append_cache_entry LibGeoIP_ROOT_DIR PATH $optarg ;; --with-perftools=*) - append_cache_entry ENABLE_PERFTOOLS BOOL true append_cache_entry GooglePerftools_ROOT_DIR PATH $optarg ;; --with-python=*) @@ -232,6 +236,14 @@ while [ $# -ne 0 ]; do shift done +if [ "$user_set_scriptdir" != "true" ]; then + append_cache_entry BRO_SCRIPT_INSTALL_PATH STRING $prefix/share/bro +fi + +if [ "$user_set_conffilesdir" != "true" ]; then + append_cache_entry BRO_ETC_INSTALL_DIR PATH $prefix/etc +fi + if [ -d $builddir ]; then # If build directory exists, check if it has a CMake cache if [ -f $builddir/CMakeCache.txt ]; then diff --git a/doc/quickstart.rst b/doc/quickstart.rst index 5201420856..cc18956836 100644 --- a/doc/quickstart.rst +++ b/doc/quickstart.rst @@ -103,10 +103,16 @@ Optional Dependencies Bro can use libGeoIP for geo-locating IP addresses, and sendmail for sending emails. -* RPM/RedHat-based Linux: +* RedHat Enterprise Linux: .. console:: + sudo yum install geoip-devel sendmail + +* CentOS Linux: + + .. console:: + sudo yum install GeoIP-devel sendmail * DEB/Debian-based Linux: diff --git a/doc/scripts/builtins.rst b/doc/scripts/builtins.rst index ef6738a1a6..32908f71fd 100644 --- a/doc/scripts/builtins.rst +++ b/doc/scripts/builtins.rst @@ -22,7 +22,7 @@ The Bro scripting language supports the following built-in types. is a string of digits preceded by a ``+`` or ``-`` sign, e.g. ``-42`` or ``+5``. When using type inferencing use care so that the intended type is inferred, e.g. ``local size_difference = 0`` will - infer the :bro:type:`count` while ``local size_difference = +0`` + infer :bro:type:`count`, while ``local size_difference = +0`` will infer :bro:type:`int`. .. bro:type:: count @@ -32,7 +32,7 @@ The Bro scripting language supports the following built-in types. .. bro:type:: counter - An alias to :bro:type:`count` + An alias to :bro:type:`count`. .. TODO: is there anything special about this type? @@ -70,7 +70,7 @@ The Bro scripting language supports the following built-in types. A type used to hold character-string values which represent text. String constants are created by enclosing text in double quotes (") - and the backslash character (\) introduces escape sequences. + and the backslash character (\\) introduces escape sequences. Note that Bro represents strings internally as a count and vector of bytes rather than a NUL-terminated byte string (although string @@ -135,7 +135,7 @@ The Bro scripting language supports the following built-in types. type color: enum { Red, White, Blue, }; - The last comma is after ``Blue`` is optional. + The last comma after ``Blue`` is optional. .. bro:type:: timer @@ -150,21 +150,23 @@ The Bro scripting language supports the following built-in types. followed by one of ``/tcp``, ``/udp``, ``/icmp``, or ``/unknown``. Ports can be compared for equality and also for ordering. When - comparing order across transport-level protocols, ``/unknown`` < - ``/tcp`` < ``/udp`` < ``icmp``, for example ``65535/tcp`` is smaller + comparing order across transport-level protocols, ``unknown`` < + ``tcp`` < ``udp`` < ``icmp``, for example ``65535/tcp`` is smaller than ``0/udp``. .. bro:type:: addr - A type representing an IP address. Currently, Bro defaults to only - supporting IPv4 addresses unless configured/built with - ``--enable-brov6``, in which case, IPv6 addresses are supported. + A type representing an IP address. IPv4 address constants are written in "dotted quad" format, ``A1.A2.A3.A4``, where Ai all lie between 0 and 255. IPv6 address constants are written as colon-separated hexadecimal form - as described by :rfc:`2373`. + as described by :rfc:`2373`, but additionally encased in square brackets. + The mixed notation with embedded IPv4 addresses as dotted-quads in the + lower 32 bits is also allowed. + Some examples: ``[2001:db8::1]``, ``[::ffff:192.168.1.100]``, or + ``[aaaa:bbbb:cccc:dddd:eeee:ffff:1111:2222]``. Hostname constants can also be used, but since a hostname can correspond to multiple IP addresses, the type of such variable is a @@ -198,7 +200,7 @@ The Bro scripting language supports the following built-in types. A type representing a block of IP addresses in CIDR notation. A ``subnet`` constant is written as an :bro:type:`addr` followed by a slash (/) and then the network prefix size specified as a decimal - number. For example, ``192.168.0.0/16``. + number. For example, ``192.168.0.0/16`` or ``[fe80::]/64``. .. bro:type:: any @@ -230,7 +232,7 @@ The Bro scripting language supports the following built-in types. global a: table[count] of table[addr, port] of string; - which declared a table indexed by :bro:type:`count` and yielding + which declares a table indexed by :bro:type:`count` and yielding another :bro:type:`table` which is indexed by an :bro:type:`addr` and :bro:type:`port` to yield a :bro:type:`string`. @@ -392,7 +394,7 @@ The Bro scripting language supports the following built-in types. :bro:attr:`&optional` or have a :bro:attr:`&default` attribute must be specified. - To test for existence of field that is :bro:attr:`&optional`, use the + To test for existence of a field that is :bro:attr:`&optional`, use the ``?$`` operator: .. code:: bro @@ -412,7 +414,7 @@ The Bro scripting language supports the following built-in types. print f, "hello, world"; close(f); - Writing to files like this for logging usually isn't recommend, for better + Writing to files like this for logging usually isn't recommended, for better logging support see :doc:`/logging`. .. bro:type:: func @@ -512,22 +514,22 @@ scripting language supports the following built-in attributes. .. bro:attr:: &optional - Allows record field to be missing. For example the type ``record { + Allows a record field to be missing. For example the type ``record { a: int, b: port &optional }`` could be instantiated both as singleton ``[$a=127.0.0.1]`` or pair ``[$a=127.0.0.1, $b=80/tcp]``. .. bro:attr:: &default Uses a default value for a record field or container elements. For - example, ``table[int] of string &default="foo" }`` would create - table that returns The :bro:type:`string` ``"foo"`` for any + example, ``table[int] of string &default="foo" }`` would create a + table that returns the :bro:type:`string` ``"foo"`` for any non-existing index. .. bro:attr:: &redef Allows for redefinition of initial object values. This is typically used with constants, for example, ``const clever = T &redef;`` would - allow the constant to be redifined at some later point during script + allow the constant to be redefined at some later point during script execution. .. bro:attr:: &rotate_interval @@ -536,7 +538,7 @@ scripting language supports the following built-in attributes. .. bro:attr:: &rotate_size - Rotates af file after it has reached a given size in bytes. + Rotates a file after it has reached a given size in bytes. .. bro:attr:: &add_func @@ -548,7 +550,12 @@ scripting language supports the following built-in attributes. .. bro:attr:: &expire_func - Called right before a container element expires. + Called right before a container element expires. The function's + first parameter is of the same type of the container and the second + parameter the same type of the container's index. The return + value is a :bro:type:`interval` indicating the amount of additional + time to wait before expiring the container element at the given + index (which will trigger another execution of this function). .. bro:attr:: &read_expire diff --git a/scripts/base/frameworks/packet-filter/main.bro b/scripts/base/frameworks/packet-filter/main.bro index 7706c2669f..66a557f53d 100644 --- a/scripts/base/frameworks/packet-filter/main.bro +++ b/scripts/base/frameworks/packet-filter/main.bro @@ -248,7 +248,7 @@ function install() #local ts = current_time(); if ( ! precompile_pcap_filter(DefaultPcapFilter, current_filter) ) { - NOTICE([$note=Compile_Failure, + NOTICE([$note=Compile_Failure, $msg=fmt("Compiling packet filter failed"), $sub=current_filter]); Reporter::fatal(fmt("Bad pcap filter '%s'", current_filter)); @@ -263,7 +263,7 @@ function install() local info: Info; info$ts = network_time(); # If network_time() is 0.0 we're at init time so use the wall clock. - if ( info$ts == 0.0 ) + if ( info$ts == 0.0 ) { info$ts = current_time(); info$init = T; @@ -274,7 +274,7 @@ function install() { # Installing the filter failed for some reason. info$success = F; - NOTICE([$note=Install_Failure, + NOTICE([$note=Install_Failure, $msg=fmt("Installing packet filter failed"), $sub=current_filter]); } diff --git a/scripts/base/init-bare.bro b/scripts/base/init-bare.bro index 81ebf27cc8..fc553d98d8 100644 --- a/scripts/base/init-bare.bro +++ b/scripts/base/init-bare.bro @@ -46,6 +46,13 @@ type index_vec: vector of count; ## then remove this alias. type string_vec: vector of string; +## A vector of addresses. +## +## .. todo:: We need this type definition only for declaring builtin functions via +## ``bifcl``. We should extend ``bifcl`` to understand composite types directly and +## then remove this alias. +type addr_vec: vector of addr; + ## A table of strings indexed by strings. ## ## .. todo:: We need this type definition only for declaring builtin functions via @@ -85,6 +92,8 @@ type icmp_conn: record { itype: count; ##< The ICMP type of the packet that triggered the instantiation of the record. icode: count; ##< The ICMP code of the packet that triggered the instantiation of the record. len: count; ##< The length of the ICMP payload of the packet that triggered the instantiation of the record. + hlim: count; ##< The encapsulating IP header's Hop Limit value. + v6: bool; ##< True if it's an ICMPv6 packet. }; ## Packet context part of an ICMP message. The fields of this record reflect the @@ -93,11 +102,13 @@ type icmp_conn: record { ## .. bro:see:: icmp_time_exceeded icmp_unreachable type icmp_context: record { id: conn_id; ##< The packet's 4-tuple. - len: count; ##< The lenght of the packet's IP header. + len: count; ##< The length of the IP packet (headers + payload). proto: count; ##< The packet's transport-layer protocol. frag_offset: count; ##< The packet's fragementation offset. - ## True if the packet's IP header is fully included in the context. If that is not - ## the case, the other fields will all be set to null values. + ## True if the packet's IP header is not fully included in the context + ## or if there is not enough of the transport header to determine source + ## and destination ports. If that is the cast, the appropriate fields + ## of this record will be set to null values. bad_hdr_len: bool; bad_checksum: bool; ##< True if the packet's IP checksum is not correct. MF: bool; ##< True if the packets *more fragements* flag is set. @@ -303,10 +314,10 @@ type gap_info: record { gap_bytes: count; ##< How many bytes were missing in the gaps. }; -## Deprecated. -## +## Deprecated. +## ## .. todo:: Remove. It's still declared internally but doesn't seem used anywhere -## else. +## else. type packet: record { conn: connection; is_orig: bool; @@ -601,10 +612,10 @@ function add_signature_file(sold: string, snew: string): string } ## Signature files to read. Use ``redef signature_files += "foo.sig"`` to -## extend. Signature files will be searched relative to ``BRO_PATH``. +## extend. Signature files will be searched relative to ``BROPATH``. global signature_files = "" &add_func = add_signature_file; -## ``p0f`` fingerprint file to use. Will be searched relative to ``BRO_PATH``. +## ``p0f`` fingerprint file to use. Will be searched relative to ``BROPATH``. const passive_fingerprint_file = "base/misc/p0f.fp" &redef; # TCP values for :bro:see:`endpoint` *state* field. @@ -926,12 +937,319 @@ const IPPROTO_IGMP = 2; ##< Group management protocol. const IPPROTO_IPIP = 4; ##< IP encapsulation in IP. const IPPROTO_TCP = 6; ##< TCP. const IPPROTO_UDP = 17; ##< User datagram protocol. +const IPPROTO_IPV6 = 41; ##< IPv6 header. +const IPPROTO_ICMPV6 = 58; ##< ICMP for IPv6. const IPPROTO_RAW = 255; ##< Raw IP packet. -## Values extracted from an IP header. +# Definitions for IPv6 extension headers. +const IPPROTO_HOPOPTS = 0; ##< IPv6 hop-by-hop-options header. +const IPPROTO_ROUTING = 43; ##< IPv6 routing header. +const IPPROTO_FRAGMENT = 44; ##< IPv6 fragment header. +const IPPROTO_ESP = 50; ##< IPv6 encapsulating security payload header. +const IPPROTO_AH = 51; ##< IPv6 authentication header. +const IPPROTO_NONE = 59; ##< IPv6 no next header. +const IPPROTO_DSTOPTS = 60; ##< IPv6 destination options header. +const IPPROTO_MOBILITY = 135; ##< IPv6 mobility header. + +## Values extracted from an IPv6 extension header's (e.g. hop-by-hop or +## destination option headers) option field. ## -## .. bro:see:: pkt_hdr discarder_check_ip -type ip_hdr: record { +## .. bro:see:: ip6_hdr ip6_hdr_chain ip6_hopopts ip6_dstopts +type ip6_option: record { + otype: count; ##< Option type. + len: count; ##< Option data length. + data: string; ##< Option data. +}; + +## Values extracted from an IPv6 Hop-by-Hop options extension header. +## +## .. bro:see:: pkt_hdr ip4_hdr ip6_hdr ip6_hdr_chain ip6_option +type ip6_hopopts: record { + ## Protocol number of the next header (RFC 1700 et seq., IANA assigned + ## number), e.g. :bro:id:`IPPROTO_ICMP`. + nxt: count; + ## Length of header in 8-octet units, excluding first unit. + len: count; + ## The TLV encoded options; + options: vector of ip6_option; +}; + +## Values extracted from an IPv6 Destination options extension header. +## +## .. bro:see:: pkt_hdr ip4_hdr ip6_hdr ip6_hdr_chain ip6_option +type ip6_dstopts: record { + ## Protocol number of the next header (RFC 1700 et seq., IANA assigned + ## number), e.g. :bro:id:`IPPROTO_ICMP`. + nxt: count; + ## Length of header in 8-octet units, excluding first unit. + len: count; + ## The TLV encoded options; + options: vector of ip6_option; +}; + +## Values extracted from an IPv6 Routing extension header. +## +## .. bro:see:: pkt_hdr ip4_hdr ip6_hdr ip6_hdr_chain +type ip6_routing: record { + ## Protocol number of the next header (RFC 1700 et seq., IANA assigned + ## number), e.g. :bro:id:`IPPROTO_ICMP`. + nxt: count; + ## Length of header in 8-octet units, excluding first unit. + len: count; + ## Routing type. + rtype: count; + ## Segments left. + segleft: count; + ## Type-specific data. + data: string; +}; + +## Values extracted from an IPv6 Fragment extension header. +## +## .. bro:see:: pkt_hdr ip4_hdr ip6_hdr ip6_hdr_chain +type ip6_fragment: record { + ## Protocol number of the next header (RFC 1700 et seq., IANA assigned + ## number), e.g. :bro:id:`IPPROTO_ICMP`. + nxt: count; + ## 8-bit reserved field. + rsv1: count; + ## Fragmentation offset. + offset: count; + ## 2-bit reserved field. + rsv2: count; + ## More fragments. + more: bool; + ## Fragment identification. + id: count; +}; + +## Values extracted from an IPv6 Authentication extension header. +## +## .. bro:see:: pkt_hdr ip4_hdr ip6_hdr ip6_hdr_chain +type ip6_ah: record { + ## Protocol number of the next header (RFC 1700 et seq., IANA assigned + ## number), e.g. :bro:id:`IPPROTO_ICMP`. + nxt: count; + ## Length of header in 4-octet units, excluding first two units. + len: count; + ## Reserved field. + rsv: count; + ## Security Parameter Index. + spi: count; + ## Sequence number. + seq: count; + ## Authentication data. + data: string; +}; + +## Values extracted from an IPv6 ESP extension header. +## +## .. bro:see:: pkt_hdr ip4_hdr ip6_hdr ip6_hdr_chain +type ip6_esp: record { + ## Security Parameters Index. + spi: count; + ## Sequence number. + seq: count; +}; + +## Values extracted from an IPv6 Mobility Binding Refresh Request message. +## +## .. bro:see:: ip6_mobility_hdr ip6_hdr ip6_hdr_chain ip6_mobility_msg +type ip6_mobility_brr: record { + ## Reserved. + rsv: count; + ## Mobility Options. + options: vector of ip6_option; +}; + +## Values extracted from an IPv6 Mobility Home Test Init message. +## +## .. bro:see:: ip6_mobility_hdr ip6_hdr ip6_hdr_chain ip6_mobility_msg +type ip6_mobility_hoti: record { + ## Reserved. + rsv: count; + ## Home Init Cookie. + cookie: count; + ## Mobility Options. + options: vector of ip6_option; +}; + +## Values extracted from an IPv6 Mobility Care-of Test Init message. +## +## .. bro:see:: ip6_mobility_hdr ip6_hdr ip6_hdr_chain ip6_mobility_msg +type ip6_mobility_coti: record { + ## Reserved. + rsv: count; + ## Care-of Init Cookie. + cookie: count; + ## Mobility Options. + options: vector of ip6_option; +}; + +## Values extracted from an IPv6 Mobility Home Test message. +## +## .. bro:see:: ip6_mobility_hdr ip6_hdr ip6_hdr_chain ip6_mobility_msg +type ip6_mobility_hot: record { + ## Home Nonce Index. + nonce_idx: count; + ## Home Init Cookie. + cookie: count; + ## Home Keygen Token. + token: count; + ## Mobility Options. + options: vector of ip6_option; +}; + +## Values extracted from an IPv6 Mobility Care-of Test message. +## +## .. bro:see:: ip6_mobility_hdr ip6_hdr ip6_hdr_chain ip6_mobility_msg +type ip6_mobility_cot: record { + ## Care-of Nonce Index. + nonce_idx: count; + ## Care-of Init Cookie. + cookie: count; + ## Care-of Keygen Token. + token: count; + ## Mobility Options. + options: vector of ip6_option; +}; + +## Values extracted from an IPv6 Mobility Binding Update message. +## +## .. bro:see:: ip6_mobility_hdr ip6_hdr ip6_hdr_chain ip6_mobility_msg +type ip6_mobility_bu: record { + ## Sequence number. + seq: count; + ## Acknowledge bit. + a: bool; + ## Home Registration bit. + h: bool; + ## Link-Local Address Compatibility bit. + l: bool; + ## Key Management Mobility Capability bit. + k: bool; + ## Lifetime. + life: count; + ## Mobility Options. + options: vector of ip6_option; +}; + +## Values extracted from an IPv6 Mobility Binding Acknowledgement message. +## +## .. bro:see:: ip6_mobility_hdr ip6_hdr ip6_hdr_chain ip6_mobility_msg +type ip6_mobility_back: record { + ## Status. + status: count; + ## Key Management Mobility Capability. + k: bool; + ## Sequence number. + seq: count; + ## Lifetime. + life: count; + ## Mobility Options. + options: vector of ip6_option; +}; + +## Values extracted from an IPv6 Mobility Binding Error message. +## +## .. bro:see:: ip6_mobility_hdr ip6_hdr ip6_hdr_chain ip6_mobility_msg +type ip6_mobility_be: record { + ## Status. + status: count; + ## Home Address. + hoa: addr; + ## Mobility Options. + options: vector of ip6_option; +}; + +## Values extracted from an IPv6 Mobility header's message data. +## +## .. bro:see:: ip6_mobility_hdr ip6_hdr ip6_hdr_chain +type ip6_mobility_msg: record { + ## The type of message from the header's MH Type field. + id: count; + ## Binding Refresh Request. + brr: ip6_mobility_brr &optional; + ## Home Test Init. + hoti: ip6_mobility_hoti &optional; + ## Care-of Test Init. + coti: ip6_mobility_coti &optional; + ## Home Test. + hot: ip6_mobility_hot &optional; + ## Care-of Test. + cot: ip6_mobility_cot &optional; + ## Binding Update. + bu: ip6_mobility_bu &optional; + ## Binding Acknowledgement. + back: ip6_mobility_back &optional; + ## Binding Error. + be: ip6_mobility_be &optional; +}; + +## Values extracted from an IPv6 Mobility header. +## +## .. bro:see:: pkt_hdr ip4_hdr ip6_hdr ip6_hdr_chain +type ip6_mobility_hdr: record { + ## Protocol number of the next header (RFC 1700 et seq., IANA assigned + ## number), e.g. :bro:id:`IPPROTO_ICMP`. + nxt: count; + ## Length of header in 8-octet units, excluding first unit. + len: count; + ## Mobility header type used to identify header's the message. + mh_type: count; + ## Reserved field. + rsv: count; + ## Mobility header checksum. + chksum: count; + ## Mobility header message + msg: ip6_mobility_msg; +}; + +## A general container for a more specific IPv6 extension header. +## +## .. bro:see:: pkt_hdr ip4_hdr ip6_hopopts ip6_dstopts ip6_routing ip6_fragment +## ip6_ah ip6_esp +type ip6_ext_hdr: record { + ## The RFC 1700 et seq. IANA assigned number identifying the type of + ## the extension header. + id: count; + ## Hop-by-hop option extension header. + hopopts: ip6_hopopts &optional; + ## Destination option extension header. + dstopts: ip6_dstopts &optional; + ## Routing extension header. + routing: ip6_routing &optional; + ## Fragment header. + fragment: ip6_fragment &optional; + ## Authentication extension header. + ah: ip6_ah &optional; + ## Encapsulating security payload header. + esp: ip6_esp &optional; + ## Mobility header. + mobility: ip6_mobility_hdr &optional; +}; + +## Values extracted from an IPv6 header. +## +## .. bro:see:: pkt_hdr ip4_hdr ip6_hdr_chain ip6_hopopts ip6_dstopts +## ip6_routing ip6_fragment ip6_ah ip6_esp +type ip6_hdr: record { + class: count; ##< Traffic class. + flow: count; ##< Flow label. + len: count; ##< Payload length. + nxt: count; ##< Protocol number of the next header + ##< (RFC 1700 et seq., IANA assigned number) + ##< e.g. :bro:id:`IPPROTO_ICMP`. + hlim: count; ##< Hop limit. + src: addr; ##< Source address. + dst: addr; ##< Destination address. + exts: vector of ip6_ext_hdr; ##< Extension header chain. +}; + +## Values extracted from an IPv4 header. +## +## .. bro:see:: pkt_hdr ip6_hdr discarder_check_ip +type ip4_hdr: record { hl: count; ##< Header length in bytes. tos: count; ##< Type of service. len: count; ##< Total length. @@ -987,10 +1305,11 @@ type icmp_hdr: record { ## ## .. bro:see:: new_packet type pkt_hdr: record { - ip: ip_hdr; ##< The IP header. - tcp: tcp_hdr &optional; ##< The TCP header if a TCP packet. - udp: udp_hdr &optional; ##< The UDP header if a UDP packet. - icmp: icmp_hdr &optional; ##< The ICMP header if an ICMP packet. + ip: ip4_hdr &optional; ##< The IPv4 header if an IPv4 packet. + ip6: ip6_hdr &optional; ##< The IPv6 header if an IPv6 packet. + tcp: tcp_hdr &optional; ##< The TCP header if a TCP packet. + udp: udp_hdr &optional; ##< The UDP header if a UDP packet. + icmp: icmp_hdr &optional; ##< The ICMP header if an ICMP packet. }; ## Definition of "secondary filters". A secondary filter is a BPF filter given as @@ -1010,7 +1329,7 @@ global discarder_maxlen = 128 &redef; ## analysis. If the function signals to discard a packet, no further processing ## will be performed on it. ## -## i: The IP header of the considered packet. +## p: The IP header of the considered packet. ## ## Returns: True if the packet should not be analyzed any further. ## @@ -1019,15 +1338,15 @@ global discarder_maxlen = 128 &redef; ## ## .. note:: This is very low-level functionality and potentially expensive. ## Avoid using it. -global discarder_check_ip: function(i: ip_hdr): bool; +global discarder_check_ip: function(p: pkt_hdr): bool; ## Function for skipping packets based on their TCP header. If defined, this ## function will be called for all TCP packets before Bro performs any further ## analysis. If the function signals to discard a packet, no further processing ## will be performed on it. ## -## i: The IP header of the considered packet. -## t: The TCP header. +## p: The IP and TCP headers of the considered packet. +## ## d: Up to :bro:see:`discarder_maxlen` bytes of the TCP payload. ## ## Returns: True if the packet should not be analyzed any further. @@ -1037,15 +1356,15 @@ global discarder_check_ip: function(i: ip_hdr): bool; ## ## .. note:: This is very low-level functionality and potentially expensive. ## Avoid using it. -global discarder_check_tcp: function(i: ip_hdr, t: tcp_hdr, d: string): bool; +global discarder_check_tcp: function(p: pkt_hdr, d: string): bool; ## Function for skipping packets based on their UDP header. If defined, this ## function will be called for all UDP packets before Bro performs any further ## analysis. If the function signals to discard a packet, no further processing ## will be performed on it. ## -## i: The IP header of the considered packet. -## t: The UDP header. +## p: The IP and UDP headers of the considered packet. +## ## d: Up to :bro:see:`discarder_maxlen` bytes of the UDP payload. ## ## Returns: True if the packet should not be analyzed any further. @@ -1055,15 +1374,14 @@ global discarder_check_tcp: function(i: ip_hdr, t: tcp_hdr, d: string): bool; ## ## .. note:: This is very low-level functionality and potentially expensive. ## Avoid using it. -global discarder_check_udp: function(i: ip_hdr, u: udp_hdr, d: string): bool; +global discarder_check_udp: function(p: pkt_hdr, d: string): bool; ## Function for skipping packets based on their ICMP header. If defined, this ## function will be called for all ICMP packets before Bro performs any further ## analysis. If the function signals to discard a packet, no further processing ## will be performed on it. ## -## i: The IP header of the considered packet. -## ih: The ICMP header. +## p: The IP and ICMP headers of the considered packet. ## ## Returns: True if the packet should not be analyzed any further. ## @@ -1072,7 +1390,7 @@ global discarder_check_udp: function(i: ip_hdr, u: udp_hdr, d: string): bool; ## ## .. note:: This is very low-level functionality and potentially expensive. ## Avoid using it. -global discarder_check_icmp: function(i: ip_hdr, ih: icmp_hdr): bool; +global discarder_check_icmp: function(p: pkt_hdr): bool; ## Bro's watchdog interval. const watchdog_interval = 10 sec &redef; @@ -1303,7 +1621,7 @@ export { ## NFS file attributes. Field names are based on RFC 1813. ## - ## .. bro:see:: nfs_proc_getattr + ## .. bro:see:: nfs_proc_getattr type fattr_t: record { ftype: file_type_t; ##< File type. mode: count; ##< Mode @@ -1322,8 +1640,8 @@ export { }; ## NFS *readdir* arguments. - ## - ## .. bro:see:: nfs_proc_readdir + ## + ## .. bro:see:: nfs_proc_readdir type diropargs_t : record { dirfh: string; ##< The file handle of the directory. fname: string; ##< The name of the file we are interested in. @@ -1332,7 +1650,7 @@ export { ## NFS lookup reply. If the lookup failed, *dir_attr* may be set. If the lookup ## succeeded, *fh* is always set and *obj_attr* and *dir_attr* may be set. ## - ## .. bro:see:: nfs_proc_lookup + ## .. bro:see:: nfs_proc_lookup type lookup_reply_t: record { fh: string &optional; ##< File handle of object looked up. obj_attr: fattr_t &optional; ##< Optional attributes associated w/ file @@ -1349,7 +1667,7 @@ export { }; ## NFS *read* reply. If the lookup fails, *attr* may be set. If the lookup succeeds, - ## *attr* may be set and all other fields are set. + ## *attr* may be set and all other fields are set. type read_reply_t: record { attr: fattr_t &optional; ##< Attributes. size: count &optional; ##< Number of bytes read. @@ -1358,7 +1676,7 @@ export { }; ## NFS *readline* reply. If the request fails, *attr* may be set. If the request - ## succeeds, *attr* may be set and all other fields are set. + ## succeeds, *attr* may be set and all other fields are set. ## ## .. bro:see:: nfs_proc_readlink type readlink_reply_t: record { @@ -1368,7 +1686,7 @@ export { ## NFS *write* arguments. ## - ## .. bro:see:: nfs_proc_write + ## .. bro:see:: nfs_proc_write type writeargs_t: record { fh: string; ##< File handle to write to. offset: count; ##< Offset in file. @@ -1378,18 +1696,18 @@ export { }; ## NFS *wcc* attributes. - ## + ## ## .. bro:see:: NFS3::write_reply_t type wcc_attr_t: record { - size: count; ##< The dize. + size: count; ##< The dize. atime: time; ##< Access time. mtime: time; ##< Modification time. }; ## NFS *write* reply. If the request fails, *pre|post* attr may be set. If the - ## request succeeds, *pre|post* attr may be set and all other fields are set. + ## request succeeds, *pre|post* attr may be set and all other fields are set. ## - ## .. bro:see:: nfs_proc_write + ## .. bro:see:: nfs_proc_write type write_reply_t: record { preattr: wcc_attr_t &optional; ##< Pre operation attributes. postattr: fattr_t &optional; ##< Post operation attributes. @@ -1400,9 +1718,9 @@ export { ## NFS reply for *create*, *mkdir*, and *symlink*. If the proc ## failed, *dir_\*_attr* may be set. If the proc succeeded, *fh* and the *attr*'s - ## may be set. Note: no guarantee that *fh* is set after success. + ## may be set. Note: no guarantee that *fh* is set after success. ## - ## .. bro:see:: nfs_proc_create nfs_proc_mkdir + ## .. bro:see:: nfs_proc_create nfs_proc_mkdir type newobj_reply_t: record { fh: string &optional; ##< File handle of object created. obj_attr: fattr_t &optional; ##< Optional attributes associated w/ new object. @@ -1410,17 +1728,17 @@ export { dir_post_attr: fattr_t &optional; ##< Optional attributes associated w/ dir. }; - ## NFS reply for *remove*, *rmdir*. Corresponds to *wcc_data* in the spec. + ## NFS reply for *remove*, *rmdir*. Corresponds to *wcc_data* in the spec. ## - ## .. bro:see:: nfs_proc_remove nfs_proc_rmdir + ## .. bro:see:: nfs_proc_remove nfs_proc_rmdir type delobj_reply_t: record { dir_pre_attr: wcc_attr_t &optional; ##< Optional attributes associated w/ dir. dir_post_attr: fattr_t &optional; ##< Optional attributes associated w/ dir. }; ## NFS *readdir* arguments. Used for both *readdir* and *readdirplus*. - ## - ## .. bro:see:: nfs_proc_readdir + ## + ## .. bro:see:: nfs_proc_readdir type readdirargs_t: record { isplus: bool; ##< Is this a readdirplus request? dirfh: string; ##< The directory filehandle. @@ -1433,7 +1751,7 @@ export { ## NFS *direntry*. *fh* and *attr* are used for *readdirplus*. However, even ## for *readdirplus* they may not be filled out. ## - ## .. bro:see:: NFS3::direntry_vec_t NFS3::readdir_reply_t + ## .. bro:see:: NFS3::direntry_vec_t NFS3::readdir_reply_t type direntry_t: record { fileid: count; ##< E.g., inode number. fname: string; ##< Filename. @@ -1444,7 +1762,7 @@ export { ## Vector of NFS *direntry*. ## - ## .. bro:see:: NFS3::readdir_reply_t + ## .. bro:see:: NFS3::readdir_reply_t type direntry_vec_t: vector of direntry_t; ## NFS *readdir* reply. Used for *readdir* and *readdirplus*. If an is @@ -1475,7 +1793,7 @@ module GLOBAL; ## An NTP message. ## -## .. bro:see:: ntp_message +## .. bro:see:: ntp_message type ntp_msg: record { id: count; ##< Message ID. code: count; ##< Message code. @@ -1497,7 +1815,7 @@ global samba_cmds: table[count] of string &redef { return fmt("samba-unknown-%d", c); }; ## An SMB command header. -## +## ## .. bro:see:: smb_com_close smb_com_generic_andx smb_com_logoff_andx ## smb_com_negotiate smb_com_negotiate_response smb_com_nt_create_andx ## smb_com_read_andx smb_com_setup_andx smb_com_trans_mailslot @@ -1516,9 +1834,9 @@ type smb_hdr : record { }; ## An SMB transaction. -## +## ## .. bro:see:: smb_com_trans_mailslot smb_com_trans_pipe smb_com_trans_rap -## smb_com_transaction smb_com_transaction2 +## smb_com_transaction smb_com_transaction2 type smb_trans : record { word_count: count; ##< TODO. total_param_count: count; ##< TODO. @@ -1532,7 +1850,7 @@ type smb_trans : record { param_offset: count; ##< TODO. data_count: count; ##< TODO. data_offset: count; ##< TODO. - setup_count: count; ##< TODO. + setup_count: count; ##< TODO. setup0: count; ##< TODO. setup1: count; ##< TODO. setup2: count; ##< TODO. @@ -1543,19 +1861,19 @@ type smb_trans : record { ## SMB transaction data. -## +## ## .. bro:see:: smb_com_trans_mailslot smb_com_trans_pipe smb_com_trans_rap -## smb_com_transaction smb_com_transaction2 -## +## smb_com_transaction smb_com_transaction2 +## ## .. todo:: Should this really be a record type? type smb_trans_data : record { data : string; ##< The transaction's data. }; -## Deprecated. -## +## Deprecated. +## ## .. todo:: Remove. It's still declared internally but doesn't seem used anywhere -## else. +## else. type smb_tree_connect : record { flags: count; password: string; @@ -1563,21 +1881,21 @@ type smb_tree_connect : record { service: string; }; -## Deprecated. -## +## Deprecated. +## ## .. todo:: Remove. It's still declared internally but doesn't seem used anywhere -## else. +## else. type smb_negotiate : table[count] of string; ## A list of router addresses offered by a DHCP server. ## -## .. bro:see:: dhcp_ack dhcp_offer +## .. bro:see:: dhcp_ack dhcp_offer type dhcp_router_list: table[count] of addr; ## A DHCP message. ## ## .. bro:see:: dhcp_ack dhcp_decline dhcp_discover dhcp_inform dhcp_nak -## dhcp_offer dhcp_release dhcp_request +## dhcp_offer dhcp_release dhcp_request type dhcp_msg: record { op: count; ##< Message OP code. 1 = BOOTREQUEST, 2 = BOOTREPLY m_type: count; ##< The type of DHCP message. @@ -1614,7 +1932,7 @@ type dns_msg: record { ## A DNS SOA record. ## -## .. bro:see:: dns_SOA_reply +## .. bro:see:: dns_SOA_reply type dns_soa: record { mname: string; ##< Primary source of data for zone. rname: string; ##< Mailbox for responsible person. @@ -1627,7 +1945,7 @@ type dns_soa: record { ## An additional DNS EDNS record. ## -## .. bro:see:: dns_EDNS_addl +## .. bro:see:: dns_EDNS_addl type dns_edns_additional: record { query: string; ##< Query. qtype: count; ##< Query type. @@ -1642,7 +1960,7 @@ type dns_edns_additional: record { ## An additional DNS TSIG record. ## -## bro:see:: dns_TSIG_addl +## bro:see:: dns_TSIG_addl type dns_tsig_additional: record { query: string; ##< Query. qtype: count; ##< Query type. @@ -1656,9 +1974,9 @@ type dns_tsig_additional: record { }; # DNS answer types. -# +# # .. .. bro:see:: dns_answerr -# +# # todo::use enum to make them autodoc'able const DNS_QUERY = 0; ##< A query. This shouldn't occur, just for completeness. const DNS_ANS = 1; ##< An answer record. @@ -1672,7 +1990,7 @@ const DNS_ADDL = 3; ##< An additional record. ## dns_TXT_reply dns_WKS_reply type dns_answer: record { ## Answer type. One of :bro:see:`DNS_QUERY`, :bro:see:`DNS_ANS`, - ## :bro:see:`DNS_AUTH` and :bro:see:`DNS_ADDL`. + ## :bro:see:`DNS_AUTH` and :bro:see:`DNS_ADDL`. answer_type: count; query: string; ##< Query. qtype: count; ##< Query type. @@ -1692,46 +2010,35 @@ global dns_skip_auth: set[addr] &redef; ## .. bro:see:: dns_skip_all_addl dns_skip_auth global dns_skip_addl: set[addr] &redef; -## If true, all DNS AUTH records are skipped. +## If true, all DNS AUTH records are skipped. ## ## .. bro:see:: dns_skip_all_addl dns_skip_auth global dns_skip_all_auth = T &redef; -## If true, all DNS ADDL records are skipped. +## If true, all DNS ADDL records are skipped. ## ## .. bro:see:: dns_skip_all_auth dns_skip_addl global dns_skip_all_addl = T &redef; ## If a DNS request includes more than this many queries, assume it's non-DNS -## traffic and do not process it. Set to 0 to turn off this functionality. +## traffic and do not process it. Set to 0 to turn off this functionality. global dns_max_queries = 5; -## The maxiumum size in bytes for an SSL cipher specifcation. If we see a packet -## that has bigger cipherspecs, we won't do a comparisons of cipherspecs. -const ssl_max_cipherspec_size = 68 &redef; - -# todo::Is this still used? -# type X509_extensions: table[count] of string; - ## An X509 certificate. ## -## .. bro:see:: x509_certificate +## .. bro:see:: x509_certificate type X509: record { version: count; ##< Version number. serial: string; ##< Serial number. - subject: string; ##< Subject. + subject: string; ##< Subject. issuer: string; ##< Issuer. not_valid_before: time; ##< Timestamp before when certificate is not valid. not_valid_after: time; ##< Timestamp after when certificate is not valid. }; -# This is indexed with the CA's name and yields a DER (binary) encoded certificate. -# todo::Is this still used? -# const root_ca_certs: table[string] of string = {} &redef; - ## HTTP session statistics. ## -## .. bro:see:: http_stats +## .. bro:see:: http_stats type http_stats_rec: record { num_requests: count; ##< Number of requests. num_replies: count; ##< Number of replies. @@ -1741,7 +2048,7 @@ type http_stats_rec: record { ## HTTP message statistics. ## -## .. bro:see:: http_message_done +## .. bro:see:: http_message_done type http_message_stat: record { ## When the request/reply line was complete. start: time; @@ -1758,26 +2065,26 @@ type http_message_stat: record { }; ## Maximum number of HTTP entity data delivered to events. The amount of data -## can be limited for better performance, zero disables truncation. -## +## can be limited for better performance, zero disables truncation. +## ## .. bro:see:: http_entity_data skip_http_entity_data skip_http_data global http_entity_data_delivery_size = 1500 &redef; ## Skip HTTP data for performance considerations. The skipped -## portion will not go through TCP reassembly. -## +## portion will not go through TCP reassembly. +## ## .. bro:see:: http_entity_data skip_http_entity_data http_entity_data_delivery_size const skip_http_data = F &redef; ## Maximum length of HTTP URIs passed to events. Longer ones will be truncated ## to prevent over-long URIs (usually sent by worms) from slowing down event ## processing. A value of -1 means "do not truncate". -## +## ## .. bro:see:: http_request const truncate_http_URI = -1 &redef; -## IRC join information. -## +## IRC join information. +## ## .. bro:see:: irc_join_list type irc_join_info: record { nick: string; @@ -1788,13 +2095,13 @@ type irc_join_info: record { ## Set of IRC join information. ## -## .. bro:see:: irc_join_message +## .. bro:see:: irc_join_message type irc_join_list: set[irc_join_info]; -## Deprecated. -## +## Deprecated. +## ## .. todo:: Remove. It's still declared internally but doesn't seem used anywhere -## else. +## else. global irc_servers : set[addr] &redef; ## Internal to the stepping stone detector. @@ -1858,7 +2165,7 @@ type backdoor_endp_stats: record { ## Description of a signature match. ## -## .. bro:see:: signature_match +## .. bro:see:: signature_match type signature_state: record { sig_id: string; ##< ID of the matching signature. conn: connection; ##< Matching connection. @@ -1866,10 +2173,10 @@ type signature_state: record { payload_size: count; ##< Payload size of the first matching packet of current endpoint. }; -# Deprecated. -# +# Deprecated. +# # .. todo:: This type is no longer used. Remove any reference of this from the -# core. +# core. type software_version: record { major: int; minor: int; @@ -1877,10 +2184,10 @@ type software_version: record { addl: string; }; -# Deprecated. -# +# Deprecated. +# # .. todo:: This type is no longer used. Remove any reference of this from the -# core. +# core. type software: record { name: string; version: software_version; @@ -1897,7 +2204,7 @@ type OS_version_inference: enum { ## Passive fingerprinting match. ## -## .. bro:see:: OS_version_found +## .. bro:see:: OS_version_found type OS_version: record { genre: string; ##< Linux, Windows, AIX, ... detail: string; ##< Lernel version or such. @@ -1907,20 +2214,20 @@ type OS_version: record { ## Defines for which subnets we should do passive fingerprinting. ## -## .. bro:see:: OS_version_found +## .. bro:see:: OS_version_found global generate_OS_version_event: set[subnet] &redef; # Type used to report load samples via :bro:see:`load_sample`. For now, it's a # set of names (event names, source file names, and perhaps ````, which were seen during the sample. +# number>``, which were seen during the sample. type load_sample_info: set[string]; ## ID for NetFlow header. This is primarily a means to sort together NetFlow -## headers and flow records at the script level. +## headers and flow records at the script level. type nfheader_id: record { ## Name of the NetFlow file (e.g., ``netflow.dat``) or the receiving socket address ## (e.g., ``127.0.0.1:5555``), or an explicit name if specified to - ## ``-y`` or ``-Y``. + ## ``-y`` or ``-Y``. rcvr_id: string; ## A serial number, ignoring any overflows. pdu_id: count; @@ -1928,7 +2235,7 @@ type nfheader_id: record { ## A NetFlow v5 header. ## -## .. bro:see:: netflow_v5_header +## .. bro:see:: netflow_v5_header type nf_v5_header: record { h_id: nfheader_id; ##< ID for sorting. cnt: count; ##< TODO. @@ -1944,7 +2251,7 @@ type nf_v5_header: record { ## A NetFlow v5 record. ## ## .. bro:see:: netflow_v5_record -type nf_v5_record: record { +type nf_v5_record: record { h_id: nfheader_id; ##< ID for sorting. id: conn_id; ##< Connection ID. nexthop: addr; ##< Address of next hop. @@ -1978,7 +2285,7 @@ type bittorrent_peer: record { }; ## A set of BitTorrent peers. -## +## ## .. bro:see:: bt_tracker_response type bittorrent_peer_set: set[bittorrent_peer]; @@ -2001,12 +2308,12 @@ type bittorrent_benc_dir: table[string] of bittorrent_benc_value; ## Header table type used by BitTorrent analyzer. ## ## .. bro:see:: bt_tracker_request bt_tracker_response -## bt_tracker_response_not_ok +## bt_tracker_response_not_ok type bt_tracker_headers: table[string] of string; @load base/event.bif -## BPF filter the user has set via the -f command line options. Empty if none. +## BPF filter the user has set via the -f command line options. Empty if none. const cmd_line_bpf_filter = "" &redef; ## Deprecated. @@ -2024,24 +2331,24 @@ const log_encryption_key = "" &redef; ## Write profiling info into this file in regular intervals. The easiest way to ## activate profiling is loading :doc:`/scripts/policy/misc/profiling`. ## -## .. bro:see:: profiling_interval expensive_profiling_multiple segment_profiling +## .. bro:see:: profiling_interval expensive_profiling_multiple segment_profiling global profiling_file: file &redef; ## Update interval for profiling (0 disables). The easiest way to activate ## profiling is loading :doc:`/scripts/policy/misc/profiling`. ## -## .. bro:see:: profiling_file expensive_profiling_multiple segment_profiling +## .. bro:see:: profiling_file expensive_profiling_multiple segment_profiling const profiling_interval = 0 secs &redef; ## Multiples of profiling_interval at which (more expensive) memory profiling is ## done (0 disables). ## -## .. bro:see:: profiling_interval profiling_file segment_profiling +## .. bro:see:: profiling_interval profiling_file segment_profiling const expensive_profiling_multiple = 0 &redef; ## If true, then write segment profiling information (very high volume!) ## in addition to profiling statistics. -## +## ## .. bro:see:: profiling_interval expensive_profiling_multiple profiling_file const segment_profiling = F &redef; @@ -2080,42 +2387,42 @@ global load_sample_freq = 20 &redef; ## Rate at which to generate :bro:see:`gap_report` events assessing to what degree ## the measurement process appears to exhibit loss. -## +## ## .. bro:see:: gap_report const gap_report_freq = 1.0 sec &redef; ## Whether we want :bro:see:`content_gap` and :bro:see:`gap_report` for partial ## connections. A connection is partial if it is missing a full handshake. Note ## that gap reports for partial connections might not be reliable. -## +## ## .. bro:see:: content_gap gap_report partial_connection const report_gaps_for_partial = F &redef; ## The CA certificate file to authorize remote Bros/Broccolis. -## +## ## .. bro:see:: ssl_private_key ssl_passphrase const ssl_ca_certificate = "" &redef; ## File containing our private key and our certificate. -## +## ## .. bro:see:: ssl_ca_certificate ssl_passphrase const ssl_private_key = "" &redef; ## The passphrase for our private key. Keeping this undefined ## causes Bro to prompt for the passphrase. -## +## ## .. bro:see:: ssl_private_key ssl_ca_certificate const ssl_passphrase = "" &redef; ## Default mode for Bro's user-space dynamic packet filter. If true, packets that -## aren't explicitly allowed through, are dropped from any further processing. -## +## aren't explicitly allowed through, are dropped from any further processing. +## ## .. note:: This is not the BPF packet filter but an additional dynamic filter -## that Bro optionally applies just before normal processing starts. -## -## .. bro:see:: install_dst_addr_filter install_dst_net_filter +## that Bro optionally applies just before normal processing starts. +## +## .. bro:see:: install_dst_addr_filter install_dst_net_filter ## install_src_addr_filter install_src_net_filter uninstall_dst_addr_filter -## uninstall_dst_net_filter uninstall_src_addr_filter uninstall_src_net_filter +## uninstall_dst_net_filter uninstall_src_addr_filter uninstall_src_net_filter const packet_filter_default = F &redef; ## Maximum size of regular expression groups for signature matching. @@ -2127,17 +2434,17 @@ const enable_syslog = F &redef; ## Description transmitted to remote communication peers for identification. const peer_description = "bro" &redef; -## If true, broadcast events received from one peer to all other peers. -## +## If true, broadcast events received from one peer to all other peers. +## ## .. bro:see:: forward_remote_state_changes ## ## .. note:: This option is only temporary and will disappear once we get a more ## sophisticated script-level communication framework. const forward_remote_events = F &redef; -## If true, broadcast state updates received from one peer to all other peers. -## -## .. bro:see:: forward_remote_events +## If true, broadcast state updates received from one peer to all other peers. +## +## .. bro:see:: forward_remote_events ## ## .. note:: This option is only temporary and will disappear once we get a more ## sophisticated script-level communication framework. @@ -2146,26 +2453,6 @@ const forward_remote_state_changes = F &redef; ## Place-holder constant indicating "no peer". const PEER_ID_NONE = 0; -## Deprecated. -## -## .. todo:: The connection compressor is scheduled to be removed from Bro. -const use_connection_compressor = F &redef; - -## Deprecated. -## -## .. todo:: The connection compressor is scheduled to be removed from Bro. -const cc_handle_resets = F &redef; - -## Deprecated. -## -## .. todo:: The connection compressor is scheduled to be removed from Bro. -const cc_handle_only_syns = T &redef; - -## Deprecated. -## -## .. todo:: The connection compressor is scheduled to be removed from Bro. -const cc_instantiate_on_data = F &redef; - # Signature payload pattern types. # todo::use enum to help autodoc # todo::Still used? @@ -2186,23 +2473,23 @@ const REMOTE_SRC_PARENT = 2; ##< Message from the parent process. const REMOTE_SRC_SCRIPT = 3; ##< Message from a policy script. ## Synchronize trace processing at a regular basis in pseudo-realtime mode. -## +## ## .. bro:see:: remote_trace_sync_peers const remote_trace_sync_interval = 0 secs &redef; ## Number of peers across which to synchronize trace processing in -## pseudo-realtime mode. -## +## pseudo-realtime mode. +## ## .. bro:see:: remote_trace_sync_interval const remote_trace_sync_peers = 0 &redef; ## Whether for :bro:attr:`&synchronized` state to send the old value as a -## consistency check. +## consistency check. const remote_check_sync_consistency = F &redef; ## Analyzer tags. The core automatically defines constants ## ``ANALYZER_*``, e.g., ``ANALYZER_HTTP``. -## +## ## .. bro:see:: dpd_config ## ## .. todo::We should autodoc these automaticallty generated constants. @@ -2220,7 +2507,7 @@ type dpd_protocol_config: record { ## This table defines the ports. ## ## .. bro:see:: dpd_reassemble_first_packets dpd_buffer_size -## dpd_match_only_beginning dpd_ignore_ports +## dpd_match_only_beginning dpd_ignore_ports const dpd_config: table[AnalyzerTag] of dpd_protocol_config = {} &redef; ## Reassemble the beginning of all TCP connections before doing @@ -2228,10 +2515,10 @@ const dpd_config: table[AnalyzerTag] of dpd_protocol_config = {} &redef; ## expensive of CPU cycles. ## ## .. bro:see:: dpd_config dpd_buffer_size -## dpd_match_only_beginning dpd_ignore_ports -## +## dpd_match_only_beginning dpd_ignore_ports +## ## .. note:: Despite the name, this option affects *all* signature matching, not -## only signatures used for dynamic protocol detection. +## only signatures used for dynamic protocol detection. const dpd_reassemble_first_packets = T &redef; ## Size of per-connection buffer used for dynamic protocol detection. For each @@ -2240,23 +2527,23 @@ const dpd_reassemble_first_packets = T &redef; ## already passed through (i.e., when a DPD signature matches only later). ## However, once the buffer is full, data is deleted and lost to analyzers that are ## activated afterwards. Then only analyzers that can deal with partial -## connections will be able to analyze the session. +## connections will be able to analyze the session. ## ## .. bro:see:: dpd_reassemble_first_packets dpd_config dpd_match_only_beginning -## dpd_ignore_ports +## dpd_ignore_ports const dpd_buffer_size = 1024 &redef; ## If true, stops signature matching if dpd_buffer_size has been reached. ## ## .. bro:see:: dpd_reassemble_first_packets dpd_buffer_size -## dpd_config dpd_ignore_ports -## +## dpd_config dpd_ignore_ports +## ## .. note:: Despite the name, this option affects *all* signature matching, not -## only signatures used for dynamic protocol detection. +## only signatures used for dynamic protocol detection. const dpd_match_only_beginning = T &redef; ## If true, don't consider any ports for deciding which protocol analyzer to -## use. If so, the value of :bro:see:`dpd_config` is ignored. +## use. If so, the value of :bro:see:`dpd_config` is ignored. ## ## .. bro:see:: dpd_reassemble_first_packets dpd_buffer_size ## dpd_match_only_beginning dpd_config @@ -2264,14 +2551,14 @@ const dpd_ignore_ports = F &redef; ## Ports which the core considers being likely used by servers. For ports in ## this set, is may heuristically decide to flip the direction of the -## connection if it misses the initial handshake. +## connection if it misses the initial handshake. const likely_server_ports: set[port] &redef; ## Deprated. Set of all ports for which we know an analyzer, built by -## :doc:`/scripts/base/frameworks/dpd/main`. +## :doc:`/scripts/base/frameworks/dpd/main`. ## ## .. todo::This should be defined by :doc:`/scripts/base/frameworks/dpd/main` -## itself we still need it. +## itself we still need it. global dpd_analyzer_ports: table[port] of set[AnalyzerTag]; ## Per-incident timer managers are drained after this amount of inactivity. @@ -2284,7 +2571,7 @@ const time_machine_profiling = F &redef; const check_for_unused_event_handlers = F &redef; # If true, dumps all invoked event handlers at startup. -# todo::Still used? +# todo::Still used? # const dump_used_event_handlers = F &redef; ## Deprecated. @@ -2300,7 +2587,7 @@ const trace_output_file = ""; ## of setting this to true is that we can write the packets out before we actually ## process them, which can be helpful for debugging in case the analysis triggers a ## crash. -## +## ## .. bro:see:: trace_output_file const record_all_packets = F &redef; @@ -2313,7 +2600,7 @@ const record_all_packets = F &redef; const ignore_keep_alive_rexmit = F &redef; ## Whether the analysis engine parses IP packets encapsulated in -## UDP tunnels. +## UDP tunnels. ## ## .. bro:see:: tunnel_port const parse_udp_tunnels = F &redef; @@ -2321,6 +2608,6 @@ const parse_udp_tunnels = F &redef; ## Number of bytes per packet to capture from live interfaces. const snaplen = 8192 &redef; -# Load the logging framework here because it uses fairly deep integration with +# Load the logging framework here because it uses fairly deep integration with # BiFs and script-land defined types. @load base/frameworks/logging diff --git a/scripts/base/protocols/dns/main.bro b/scripts/base/protocols/dns/main.bro index 4fa4b229f7..95259fb2e8 100644 --- a/scripts/base/protocols/dns/main.bro +++ b/scripts/base/protocols/dns/main.bro @@ -40,8 +40,6 @@ export { rcode: count &log &optional; ## A descriptive name for the response code value. rcode_name: string &log &optional; - ## Whether the message is a query (F) or response (T). - QR: bool &log &default=F; ## The Authoritative Answer bit for response messages specifies that ## the responding name server is an authority for the domain name ## in the question section. @@ -250,10 +248,13 @@ event dns_TXT_reply(c: connection, msg: dns_msg, ans: dns_answer, str: string) & event DNS::do_reply(c, msg, ans, str); } -event dns_AAAA_reply(c: connection, msg: dns_msg, ans: dns_answer, a: addr, - astr: string) &priority=5 +event dns_AAAA_reply(c: connection, msg: dns_msg, ans: dns_answer, a: addr) &priority=5 + { + event DNS::do_reply(c, msg, ans, fmt("%s", a)); + } + +event dns_A6_reply(c: connection, msg: dns_msg, ans: dns_answer, a: addr) &priority=5 { - # TODO: What should we do with astr? event DNS::do_reply(c, msg, ans, fmt("%s", a)); } diff --git a/scripts/base/protocols/ftp/main.bro b/scripts/base/protocols/ftp/main.bro index 80a0e0a796..52366d28e1 100644 --- a/scripts/base/protocols/ftp/main.bro +++ b/scripts/base/protocols/ftp/main.bro @@ -24,7 +24,7 @@ export { const default_capture_password = F &redef; ## User IDs that can be considered "anonymous". - const guest_ids = { "anonymous", "ftp", "guest" } &redef; + const guest_ids = { "anonymous", "ftp", "ftpuser", "guest" } &redef; type Info: record { ## Time when the command was sent. @@ -160,12 +160,21 @@ function ftp_message(s: Info) # or it's a deliberately logged command. if ( |s$tags| > 0 || (s?$cmdarg && s$cmdarg$cmd in logged_commands) ) { - if ( s?$password && to_lower(s$user) !in guest_ids ) + if ( s?$password && + ! s$capture_password && + to_lower(s$user) !in guest_ids ) + { s$password = ""; + } local arg = s$cmdarg$arg; if ( s$cmdarg$cmd in file_cmds ) - arg = fmt("ftp://%s%s", s$id$resp_h, build_path_compressed(s$cwd, arg)); + { + if ( is_v4_addr(s$id$resp_h) ) + arg = fmt("ftp://%s%s", s$id$resp_h, build_path_compressed(s$cwd, arg)); + else + arg = fmt("ftp://[%s]%s", s$id$resp_h, build_path_compressed(s$cwd, arg)); + } s$ts=s$cmdarg$ts; s$command=s$cmdarg$cmd; @@ -270,7 +279,7 @@ event ftp_reply(c: connection, code: count, msg: string, cont_resp: bool) &prior { c$ftp$passive=T; - if ( code == 229 && data$h == 0.0.0.0 ) + if ( code == 229 && data$h == [::] ) data$h = id$resp_h; ftp_data_expected[data$h, data$p] = c$ftp; diff --git a/scripts/base/protocols/ssl/consts.bro b/scripts/base/protocols/ssl/consts.bro index 9d16ab18ba..6c33e6e438 100644 --- a/scripts/base/protocols/ssl/consts.bro +++ b/scripts/base/protocols/ssl/consts.bro @@ -13,7 +13,7 @@ export { [TLSv10] = "TLSv10", [TLSv11] = "TLSv11", [TLSv12] = "TLSv12", - } &default="UNKNOWN"; + } &default=function(i: count):string { return fmt("unknown-%d", i); }; ## Mapping between numeric codes and human readable strings for alert ## levels. @@ -77,7 +77,9 @@ export { [12] = "srp", [13] = "signature_algorithms", [14] = "use_srtp", + [15] = "heartbeat", [35] = "SessionTicket TLS", + [40] = "extended_random", [13172] = "next_protocol_negotiation", [65281] = "renegotiation_info" } &default=function(i: count):string { return fmt("unknown-%d", i); }; @@ -535,7 +537,7 @@ export { [SSL_RSA_FIPS_WITH_3DES_EDE_CBC_SHA] = "SSL_RSA_FIPS_WITH_3DES_EDE_CBC_SHA", [SSL_RSA_FIPS_WITH_DES_CBC_SHA_2] = "SSL_RSA_FIPS_WITH_DES_CBC_SHA_2", [SSL_RSA_FIPS_WITH_3DES_EDE_CBC_SHA_2] = "SSL_RSA_FIPS_WITH_3DES_EDE_CBC_SHA_2", - } &default="UNKNOWN"; + } &default=function(i: count):string { return fmt("unknown-%d", i); }; ## Mapping between the constants and string values for SSL/TLS errors. const x509_errors: table[count] of string = { @@ -573,6 +575,6 @@ export { [31] = "keyusage no certsign", [32] = "unable to get crl issuer", [33] = "unhandled critical extension", - }; + } &default=function(i: count):string { return fmt("unknown-%d", i); }; } diff --git a/scripts/policy/protocols/http/detect-webapps.bro b/scripts/policy/protocols/http/detect-webapps.bro index afb95074e2..796da5c29a 100644 --- a/scripts/policy/protocols/http/detect-webapps.bro +++ b/scripts/policy/protocols/http/detect-webapps.bro @@ -28,7 +28,7 @@ event signature_match(state: signature_state, msg: string, data: string) &priori local c = state$conn; local si = Software::Info; - si = [$unparsed_version=msg, $host=c$id$resp_h, $host_p=c$id$resp_p, $software_type=WEB_APPLICATION]; + si = [$name=msg, $unparsed_version=msg, $host=c$id$resp_h, $host_p=c$id$resp_p, $software_type=WEB_APPLICATION]; si$url = build_url_http(c$http); if ( c$id$resp_h in Software::tracked && si$name in Software::tracked[c$id$resp_h] ) diff --git a/src/ARP.cc b/src/ARP.cc index 3606ed66d5..7ffd82764c 100644 --- a/src/ARP.cc +++ b/src/ARP.cc @@ -17,7 +17,7 @@ ARP_Analyzer::~ARP_Analyzer() { } -bool ARP_Analyzer::IsARP(const u_char* pkt, int hdr_size) const +bool ARP_Analyzer::IsARP(const u_char* pkt, int hdr_size) { unsigned short network_protocol = *(unsigned short*) (pkt + hdr_size - 2); diff --git a/src/ARP.h b/src/ARP.h index 37f20ced3c..f4b623c513 100644 --- a/src/ARP.h +++ b/src/ARP.h @@ -31,9 +31,6 @@ public: ARP_Analyzer(); virtual ~ARP_Analyzer(); - // Whether a packet is of interest for ARP analysis. - bool IsARP(const u_char* pkt, int hdr_size) const; - void NextPacket(double t, const struct pcap_pkthdr* hdr, const u_char* const pkt, int hdr_size); @@ -41,6 +38,10 @@ public: void RREvent(EventHandlerPtr e, const u_char* src, const u_char* dst, const char* spa, const char* sha, const char* tpa, const char* tha); + + // Whether a packet is of interest for ARP analysis. + static bool IsARP(const u_char* pkt, int hdr_size); + protected: AddrVal* ConstructAddrVal(const void* addr); StringVal* EthAddrToStr(const u_char* addr); diff --git a/src/Analyzer.cc b/src/Analyzer.cc index 83f1f7f821..92ca3ecc50 100644 --- a/src/Analyzer.cc +++ b/src/Analyzer.cc @@ -49,18 +49,6 @@ const Analyzer::Config Analyzer::analyzer_configs[] = { { AnalyzerTag::ICMP, "ICMP", ICMP_Analyzer::InstantiateAnalyzer, ICMP_Analyzer::Available, 0, false }, - { AnalyzerTag::ICMP_TimeExceeded, "ICMP_TIMEEXCEEDED", - ICMP_TimeExceeded_Analyzer::InstantiateAnalyzer, - ICMP_TimeExceeded_Analyzer::Available, 0, false }, - { AnalyzerTag::ICMP_Unreachable, "ICMP_UNREACHABLE", - ICMP_Unreachable_Analyzer::InstantiateAnalyzer, - ICMP_Unreachable_Analyzer::Available, 0, false }, - { AnalyzerTag::ICMP_Echo, "ICMP_ECHO", - ICMP_Echo_Analyzer::InstantiateAnalyzer, - ICMP_Echo_Analyzer::Available, 0, false }, - { AnalyzerTag::ICMP_Redir, "ICMP_REDIR", - ICMP_Redir_Analyzer::InstantiateAnalyzer, - ICMP_Redir_Analyzer::Available, 0, false }, { AnalyzerTag::TCP, "TCP", TCP_Analyzer::InstantiateAnalyzer, TCP_Analyzer::Available, 0, false }, diff --git a/src/AnalyzerTags.h b/src/AnalyzerTags.h index fd31773120..dc10a55f22 100644 --- a/src/AnalyzerTags.h +++ b/src/AnalyzerTags.h @@ -20,9 +20,7 @@ namespace AnalyzerTag { PIA_TCP, PIA_UDP, // Transport-layer analyzers. - ICMP, - ICMP_TimeExceeded, ICMP_Unreachable, ICMP_Echo, ICMP_Redir, - TCP, UDP, + ICMP, TCP, UDP, // Application-layer analyzers (hand-written). BitTorrent, BitTorrentTracker, diff --git a/src/Anon.cc b/src/Anon.cc index 440f8600d5..d2a28a0e08 100644 --- a/src/Anon.cc +++ b/src/Anon.cc @@ -153,7 +153,9 @@ void AnonymizeIPAddr_A50::init() int AnonymizeIPAddr_A50::PreservePrefix(ipaddr32_t input, int num_bits) { - DEBUG_MSG("%s/%d\n", dotted_addr(input), num_bits); + DEBUG_MSG("%s/%d\n", + IPAddr(IPv4, &input, IPAddr::Network).AsString().c_str(), + num_bits); if ( ! before_anonymization ) { diff --git a/src/Attr.cc b/src/Attr.cc index aed9165182..82d9c9ddc7 100644 --- a/src/Attr.cc +++ b/src/Attr.cc @@ -5,7 +5,7 @@ #include "Attr.h" #include "Expr.h" #include "Serializer.h" -#include "LogMgr.h" +#include "threading/SerialTypes.h" const char* attr_name(attr_tag t) { @@ -416,7 +416,7 @@ void Attributes::CheckAttr(Attr* a) break; case ATTR_LOG: - if ( ! LogVal::IsCompatibleType(type) ) + if ( ! threading::Value::IsCompatibleType(type) ) Error("&log applied to a type that cannot be logged"); break; diff --git a/src/BitTorrent.cc b/src/BitTorrent.cc index c58eb4cf65..824e4ec98d 100644 --- a/src/BitTorrent.cc +++ b/src/BitTorrent.cc @@ -66,39 +66,44 @@ void BitTorrent_Analyzer::DeliverStream(int len, const u_char* data, bool orig) void BitTorrent_Analyzer::Undelivered(int seq, int len, bool orig) { - uint64 entry_offset = orig ? - *interp->upflow()->next_message_offset() : - *interp->downflow()->next_message_offset(); - uint64& this_stream_len = orig ? stream_len_orig : stream_len_resp; - bool& this_stop = orig ? stop_orig : stop_resp; - TCP_ApplicationAnalyzer::Undelivered(seq, len, orig); - this_stream_len += len; + // TODO: Code commented out for now. I think that shoving data that + // is definitely wrong into the parser seems like a really bad idea. + // The way it's currently tracking the next message offset isn't + // compatible with new 64bit int support in binpac either. - if ( entry_offset < this_stream_len ) - { // entry point is somewhere in the gap - DeliverWeird("Stopping BitTorrent analysis: cannot recover from content gap", orig); - this_stop = true; - if ( stop_orig && stop_resp ) - ProtocolViolation("BitTorrent: content gap and/or protocol violation"); - } - else - { // fill the gap - try - { - u_char gap[len]; - memset(gap, 0, len); - interp->NewData(orig, gap, gap + len); - } - catch ( binpac::Exception const &e ) - { - DeliverWeird("Stopping BitTorrent analysis: filling content gap failed", orig); - this_stop = true; - if ( stop_orig && stop_resp ) - ProtocolViolation("BitTorrent: content gap and/or protocol violation"); - } - } + //uint64 entry_offset = orig ? + // *interp->upflow()->next_message_offset() : + // *interp->downflow()->next_message_offset(); + //uint64& this_stream_len = orig ? stream_len_orig : stream_len_resp; + //bool& this_stop = orig ? stop_orig : stop_resp; + // + //this_stream_len += len; + // + //if ( entry_offset < this_stream_len ) + // { // entry point is somewhere in the gap + // DeliverWeird("Stopping BitTorrent analysis: cannot recover from content gap", orig); + // this_stop = true; + // if ( stop_orig && stop_resp ) + // ProtocolViolation("BitTorrent: content gap and/or protocol violation"); + // } + //else + // { // fill the gap + // try + // { + // u_char gap[len]; + // memset(gap, 0, len); + // interp->NewData(orig, gap, gap + len); + // } + // catch ( binpac::Exception const &e ) + // { + // DeliverWeird("Stopping BitTorrent analysis: filling content gap failed", orig); + // this_stop = true; + // if ( stop_orig && stop_resp ) + // ProtocolViolation("BitTorrent: content gap and/or protocol violation"); + // } + // } } void BitTorrent_Analyzer::EndpointEOF(TCP_Reassembler* endp) diff --git a/src/BroDoc.cc b/src/BroDoc.cc index b20db727ff..1e2d7d52ea 100644 --- a/src/BroDoc.cc +++ b/src/BroDoc.cc @@ -85,12 +85,13 @@ void BroDoc::AddImport(const std::string& s) if ( ext_pos != std::string::npos ) lname = lname.substr(0, ext_pos); - const char* full_filename = ""; - const char* subpath = ""; + const char* full_filename = NULL; + const char* subpath = NULL; + FILE* f = search_for_file(lname.c_str(), "bro", &full_filename, true, &subpath); - if ( f ) + if ( f && full_filename && subpath ) { fclose(f); @@ -126,12 +127,14 @@ void BroDoc::AddImport(const std::string& s) } delete [] tmp; - delete [] full_filename; - delete [] subpath; } + else fprintf(stderr, "Failed to document '@load %s' in file: %s\n", s.c_str(), reST_filename.c_str()); + + delete [] full_filename; + delete [] subpath; } void BroDoc::SetPacketFilter(const std::string& s) diff --git a/src/Brofiler.cc b/src/Brofiler.cc index 60e57f0964..c9a3505069 100644 --- a/src/Brofiler.cc +++ b/src/Brofiler.cc @@ -1,11 +1,12 @@ #include +#include #include #include #include "Brofiler.h" #include "util.h" Brofiler::Brofiler() - : delim('\t'), ignoring(0) + : ignoring(0), delim('\t') { } @@ -48,10 +49,27 @@ bool Brofiler::WriteStats() char* bf = getenv("BRO_PROFILER_FILE"); if ( ! bf ) return false; - FILE* f = fopen(bf, "w"); + FILE* f; + const char* p = strstr(bf, ".XXXXXX"); + + if ( p && ! p[7] ) + { + int fd = mkstemp(bf); + if ( fd == -1 ) + { + reporter->Error("Failed to generate unique file name from BRO_PROFILER_FILE: %s", bf); + return false; + } + f = fdopen(fd, "w"); + } + else + { + f = fopen(bf, "w"); + } + if ( ! f ) { - reporter->Error("Failed to open BRO_PROFILER_FILE destination '%s' for writing\n", bf); + reporter->Error("Failed to open BRO_PROFILER_FILE destination '%s' for writing", bf); return false; } diff --git a/src/Brofiler.h b/src/Brofiler.h index edbe1e932c..22e5808bf6 100644 --- a/src/Brofiler.h +++ b/src/Brofiler.h @@ -26,7 +26,9 @@ public: /** * Combines usage stats from current run with any read from ReadStats(), * then writes information to file pointed to by environment variable - * BRO_PROFILER_FILE. + * BRO_PROFILER_FILE. If the value of that env. variable ends with + * ".XXXXXX" (exactly 6 X's), then it is first passed through mkstemp + * to get a unique file. * * @return: true when usage info is written, otherwise false. */ diff --git a/src/CMakeLists.txt b/src/CMakeLists.txt index 0e29082db3..ce1b25dd42 100644 --- a/src/CMakeLists.txt +++ b/src/CMakeLists.txt @@ -213,6 +213,8 @@ binpac_target(syslog.pac ######################################################################## ## bro target +find_package (Threads) + # This macro stores associated headers for any C/C++ source files given # as arguments (past _var) as a list in the CMake variable named "_var". macro(COLLECT_HEADERS _var) @@ -288,7 +290,6 @@ set(bro_SRCS ChunkedIO.cc CompHash.cc Conn.cc - ConnCompressor.cc ConnSizeAnalyzer.cc ContentLine.cc DCE_RPC.cc @@ -331,13 +332,11 @@ set(bro_SRCS IntSet.cc InterConn.cc IOSource.cc + IP.cc + IPAddr.cc IRC.cc List.cc Reporter.cc - LogMgr.cc - LogWriter.cc - LogWriterAscii.cc - LogWriterNone.cc Login.cc MIME.cc NCP.cc @@ -402,6 +401,7 @@ set(bro_SRCS XDR.cc ZIP.cc bsd-getopt-long.c + bro_inet_ntop.c cq.c md5.c patricia.c @@ -409,6 +409,18 @@ set(bro_SRCS PacketDumper.cc strsep.c modp_numtoa.c + + threading/BasicThread.cc + threading/Manager.cc + threading/MsgThread.cc + threading/SerialTypes.cc + + logging/Manager.cc + logging/WriterBackend.cc + logging/WriterFrontend.cc + logging/writers/Ascii.cc + logging/writers/None.cc + ${dns_SRCS} ${openssl_SRCS} ) @@ -421,7 +433,7 @@ add_definitions(-DBRO_BUILD_PATH="${CMAKE_CURRENT_BINARY_DIR}") add_executable(bro ${bro_SRCS} ${bro_HEADERS}) -target_link_libraries(bro ${brodeps}) +target_link_libraries(bro ${brodeps} ${CMAKE_THREAD_LIBS_INIT}) install(TARGETS bro DESTINATION bin) install(FILES ${INSTALL_BIF_OUTPUTS} DESTINATION ${BRO_SCRIPT_INSTALL_PATH}/base) diff --git a/src/CompHash.cc b/src/CompHash.cc index 545d3ed8e4..86677f9719 100644 --- a/src/CompHash.cc +++ b/src/CompHash.cc @@ -107,40 +107,18 @@ char* CompositeHash::SingleValHash(int type_check, char* kp0, case TYPE_INTERNAL_ADDR: { - // Use uint32 instead of int, because 'int' is not - // guaranteed to be 32-bit. uint32* kp = AlignAndPadType(kp0); -#ifdef BROv6 - const addr_type av = v->AsAddr(); - kp[0] = av[0]; - kp[1] = av[1]; - kp[2] = av[2]; - kp[3] = av[3]; + v->AsAddr().CopyIPv6(kp); kp1 = reinterpret_cast(kp+4); -#else - *kp = v->AsAddr(); - kp1 = reinterpret_cast(kp+1); -#endif } break; case TYPE_INTERNAL_SUBNET: { uint32* kp = AlignAndPadType(kp0); -#ifdef BROv6 - const subnet_type* sv = v->AsSubNet(); - kp[0] = sv->net[0]; - kp[1] = sv->net[1]; - kp[2] = sv->net[2]; - kp[3] = sv->net[3]; - kp[4] = sv->width; + v->AsSubNet().Prefix().CopyIPv6(kp); + kp[4] = v->AsSubNet().Length(); kp1 = reinterpret_cast(kp+5); -#else - const subnet_type* sv = v->AsSubNet(); - kp[0] = sv->net; - kp[1] = sv->width; - kp1 = reinterpret_cast(kp+2); -#endif } break; @@ -358,26 +336,16 @@ HashKey* CompositeHash::ComputeSingletonHash(const Val* v, int type_check) const if ( type_check && v->Type()->InternalType() != singleton_tag ) return 0; - uint32 tmp_addr; switch ( singleton_tag ) { case TYPE_INTERNAL_INT: case TYPE_INTERNAL_UNSIGNED: return new HashKey(v->ForceAsInt()); case TYPE_INTERNAL_ADDR: -#ifdef BROv6 - return new HashKey(v->AsAddr(), 4); -#else - return new HashKey(v->AsAddr()); -#endif + return v->AsAddr().GetHashKey(); case TYPE_INTERNAL_SUBNET: -#ifdef BROv6 - return new HashKey((const uint32*) v->AsSubNet(), 5); -#else - return new HashKey((const uint32*) v->AsSubNet(), 2); - -#endif + return v->AsSubNet().GetHashKey(); case TYPE_INTERNAL_DOUBLE: return new HashKey(v->InternalDouble()); @@ -425,22 +393,13 @@ int CompositeHash::SingleTypeKeySize(BroType* bt, const Val* v, break; case TYPE_INTERNAL_ADDR: -#ifdef BROv6 sz = SizeAlign(sz, sizeof(uint32)); sz += sizeof(uint32) * 3; // to make a total of 4 words -#else - sz = SizeAlign(sz, sizeof(uint32)); -#endif break; case TYPE_INTERNAL_SUBNET: -#ifdef BROv6 sz = SizeAlign(sz, sizeof(uint32)); sz += sizeof(uint32) * 4; // to make a total of 5 words -#else - sz = SizeAlign(sz, sizeof(uint32)); - sz += sizeof(uint32); // make room for width -#endif break; case TYPE_INTERNAL_DOUBLE: @@ -748,16 +707,13 @@ const char* CompositeHash::RecoverOneVal(const HashKey* k, const char* kp0, case TYPE_INTERNAL_ADDR: { const uint32* const kp = AlignType(kp0); -#ifdef BROv6 - const_addr_type addr_val = kp; kp1 = reinterpret_cast(kp+4); -#else - const_addr_type addr_val = *kp; - kp1 = reinterpret_cast(kp+1); -#endif + + IPAddr addr(IPv6, kp, IPAddr::Network); + switch ( tag ) { case TYPE_ADDR: - pval = new AddrVal(addr_val); + pval = new AddrVal(addr); break; default: @@ -770,12 +726,9 @@ const char* CompositeHash::RecoverOneVal(const HashKey* k, const char* kp0, case TYPE_INTERNAL_SUBNET: { - const subnet_type* const kp = - reinterpret_cast( - AlignType(kp0)); - kp1 = reinterpret_cast(kp+1); - - pval = new SubNetVal(kp->net, kp->width); + const uint32* const kp = AlignType(kp0); + kp1 = reinterpret_cast(kp+5); + pval = new SubNetVal(kp, kp[4]); } break; diff --git a/src/Conn.cc b/src/Conn.cc index df59b1037a..acf17fab3a 100644 --- a/src/Conn.cc +++ b/src/Conn.cc @@ -14,32 +14,6 @@ #include "PIA.h" #include "binpac.h" -HashKey* ConnID::BuildConnKey() const - { - Key key; - - // Lookup up connection based on canonical ordering, which is - // the smaller of and - // followed by the other. - if ( is_one_way || - addr_port_canon_lt(src_addr, src_port, dst_addr, dst_port) ) - { - copy_addr(src_addr, key.ip1); - copy_addr(dst_addr, key.ip2); - key.port1 = src_port; - key.port2 = dst_port; - } - else - { - copy_addr(dst_addr, key.ip1); - copy_addr(src_addr, key.ip2); - key.port1 = dst_port; - key.port2 = src_port; - } - - return new HashKey(&key, sizeof(key)); - } - void ConnectionTimer::Init(Connection* arg_conn, timer_func arg_timer, int arg_do_expire) { @@ -143,8 +117,8 @@ Connection::Connection(NetSessions* s, HashKey* k, double t, const ConnID* id) key = k; start_time = last_time = t; - copy_addr(id->src_addr, orig_addr); - copy_addr(id->dst_addr, resp_addr); + orig_addr = id->src_addr; + resp_addr = id->dst_addr; orig_port = id->src_port; resp_port = id->dst_port; proto = TRANSPORT_UNKNOWN; @@ -521,7 +495,7 @@ Val* Connection::BuildVersionVal(const char* s, int len) return sw; } -int Connection::VersionFoundEvent(const uint32* addr, const char* s, int len, +int Connection::VersionFoundEvent(const IPAddr& addr, const char* s, int len, Analyzer* analyzer) { if ( ! software_version_found && ! software_parse_error ) @@ -559,7 +533,7 @@ int Connection::VersionFoundEvent(const uint32* addr, const char* s, int len, return 1; } -int Connection::UnparsedVersionFoundEvent(const uint32* addr, +int Connection::UnparsedVersionFoundEvent(const IPAddr& addr, const char* full, int len, Analyzer* analyzer) { // Skip leading white space. @@ -693,10 +667,9 @@ TimerMgr* Connection::GetTimerMgr() const void Connection::FlipRoles() { - uint32 tmp_addr[NUM_ADDR_WORDS]; - copy_addr(resp_addr, tmp_addr); - copy_addr(orig_addr, resp_addr); - copy_addr(tmp_addr, orig_addr); + IPAddr tmp_addr = resp_addr; + orig_addr = resp_addr; + resp_addr = tmp_addr; uint32 tmp_port = resp_port; resp_port = orig_port; @@ -752,14 +725,14 @@ void Connection::Describe(ODesc* d) const } d->SP(); - d->Add(dotted_addr(orig_addr)); + d->Add(orig_addr); d->Add(":"); d->Add(ntohs(orig_port)); d->SP(); d->AddSP("->"); - d->Add(dotted_addr(resp_addr)); + d->Add(resp_addr); d->Add(":"); d->Add(ntohs(resp_port)); @@ -782,9 +755,8 @@ bool Connection::DoSerialize(SerialInfo* info) const // First we write the members which are needed to // create the HashKey. - for ( int j = 0; j < NUM_ADDR_WORDS; ++j ) - if ( ! SERIALIZE(orig_addr[j]) || ! SERIALIZE(resp_addr[j]) ) - return false; + if ( ! SERIALIZE(orig_addr) || ! SERIALIZE(resp_addr) ) + return false; if ( ! SERIALIZE(orig_port) || ! SERIALIZE(resp_port) ) return false; @@ -830,21 +802,21 @@ bool Connection::DoUnserialize(UnserialInfo* info) // Build the hash key first. Some of the recursive *::Unserialize() // functions may need it. - for ( int i = 0; i < NUM_ADDR_WORDS; ++i ) - if ( ! UNSERIALIZE(&orig_addr[i]) || ! UNSERIALIZE(&resp_addr[i]) ) - goto error; + ConnID id; + + if ( ! UNSERIALIZE(&orig_addr) || ! UNSERIALIZE(&resp_addr) ) + goto error; if ( ! UNSERIALIZE(&orig_port) || ! UNSERIALIZE(&resp_port) ) goto error; - ConnID id; id.src_addr = orig_addr; id.dst_addr = resp_addr; // This doesn't work for ICMP. But I guess this is not really important. id.src_port = orig_port; id.dst_port = resp_port; id.is_one_way = 0; // ### incorrect for ICMP - key = id.BuildConnKey(); + key = BuildConnIDHashKey(id); int len; if ( ! UNSERIALIZE(&len) ) diff --git a/src/Conn.h b/src/Conn.h index 8e90d6a9c3..b3eb9013d0 100644 --- a/src/Conn.h +++ b/src/Conn.h @@ -12,6 +12,7 @@ #include "PersistenceSerializer.h" #include "RuleMatcher.h" #include "AnalyzerTags.h" +#include "IPAddr.h" class Connection; class ConnectionTimer; @@ -32,52 +33,17 @@ typedef enum { typedef void (Connection::*timer_func)(double t); struct ConnID { - const uint32* src_addr; - const uint32* dst_addr; + IPAddr src_addr; + IPAddr dst_addr; uint32 src_port; uint32 dst_port; - bool is_one_way; // if true, don't canonicalize - - // Returns a ListVal suitable for looking up a connection in - // a hash table. addr/ports are expected to be in network order. - // Unless is_one_way is true, the lookup sorts src and dst, - // so src_addr/src_port and dst_addr/dst_port just have to - // reflect the two different sides of the connection, - // neither has to be the particular source/destination - // or originator/responder. - HashKey* BuildConnKey() const; - - // The structure used internally for hashing. - struct Key { - uint32 ip1[NUM_ADDR_WORDS]; - uint32 ip2[NUM_ADDR_WORDS]; - uint16 port1; - uint16 port2; - }; + bool is_one_way; // if true, don't canonicalize order }; -static inline int addr_port_canon_lt(const uint32* a1, uint32 p1, - const uint32* a2, uint32 p2) +static inline int addr_port_canon_lt(const IPAddr& addr1, uint32 p1, + const IPAddr& addr2, uint32 p2) { -#ifdef BROv6 - // Because it's a canonical ordering, not a strict ordering, - // we can choose to give more weight to the least significant - // word than to the most significant word. This matters - // because for the common case of IPv4 addresses embedded in - // a IPv6 address, the top three words are identical, so we can - // save a few cycles by first testing the bottom word. - return a1[3] < a2[3] || - (a1[3] == a2[3] && - (a1[2] < a2[2] || - (a1[2] == a2[2] && - (a1[1] < a2[1] || - (a1[1] == a2[1] && - (a1[0] < a2[0] || - (a1[0] == a2[0] && - p1 < p2))))))); -#else - return *a1 < *a2 || (*a1 == *a2 && p1 < p2); -#endif + return addr1 < addr2 || (addr1 == addr2 && p1 < p2); } class Analyzer; @@ -119,8 +85,8 @@ public: double LastTime() const { return last_time; } void SetLastTime(double t) { last_time = t; } - const uint32* OrigAddr() const { return orig_addr; } - const uint32* RespAddr() const { return resp_addr; } + const IPAddr& OrigAddr() const { return orig_addr; } + const IPAddr& RespAddr() const { return resp_addr; } uint32 OrigPort() const { return orig_port; } uint32 RespPort() const { return resp_port; } @@ -185,11 +151,11 @@ public: // Raises a software_version_found event based on the // given string (returns false if it's not parseable). - int VersionFoundEvent(const uint32* addr, const char* s, int len, + int VersionFoundEvent(const IPAddr& addr, const char* s, int len, Analyzer* analyzer = 0); // Raises a software_unparsed_version_found event. - int UnparsedVersionFoundEvent(const uint32* addr, + int UnparsedVersionFoundEvent(const IPAddr& addr, const char* full_descr, int len, Analyzer* analyzer); void Event(EventHandlerPtr f, Analyzer* analyzer, const char* name = 0); @@ -273,30 +239,6 @@ public: // Sets the transport protocol in use. void SetTransport(TransportProto arg_proto) { proto = arg_proto; } - // If the connection compressor is activated, we need a special memory - // layout for connections. (See ConnCompressor.h) - void* operator new(size_t size) - { - if ( ! use_connection_compressor ) - return ::operator new(size); - - void* c = ::operator new(size + 4); - - // We have to turn off the is_pending bit. By setting the - // first four bytes to zero, we'll achieve this. - *((uint32*) c) = 0; - - return ((char *) c) + 4; - } - - void operator delete(void* ptr) - { - if ( ! use_connection_compressor ) - ::operator delete(ptr); - else - ::operator delete(((char*) ptr) - 4); - } - void SetUID(uint64 arg_uid) { uid = arg_uid; } protected: @@ -325,8 +267,8 @@ protected: TimerMgr::Tag* conn_timer_mgr; timer_list timers; - uint32 orig_addr[NUM_ADDR_WORDS]; // in network order - uint32 resp_addr[NUM_ADDR_WORDS]; // in network order + IPAddr orig_addr; + IPAddr resp_addr; uint32 orig_port, resp_port; // in network order TransportProto proto; double start_time, last_time; diff --git a/src/ConnCompressor.cc b/src/ConnCompressor.cc deleted file mode 100644 index 7a04cb4f0b..0000000000 --- a/src/ConnCompressor.cc +++ /dev/null @@ -1,1041 +0,0 @@ -#include - -#include "ConnCompressor.h" -#include "Event.h" -#include "ConnSizeAnalyzer.h" -#include "net_util.h" - -// The basic model of the compressor is to wait for an answer before -// instantiating full connection state. Until we see a reply, only a minimal -// amount of state is stored. This has some consequences: -// -// - We try to mimic TCP.cc as close as possible, but this works only to a -// certain degree; e.g., we don't consider any of the wait-a-bit-after- -// the-connection-has-been-closed timers. That means we will get differences -// in connection semantics if the compressor is turned on. On the other -// hand, these differences will occur only for not well-established -// sessions, and experience shows that for these kinds of connections -// semantics are ill-defined in any case. -// -// - If an originator sends multiple different packets before we see a reply, -// we lose the information about additional packets (more precisely, we -// merge the packet headers into one). In particular, we lose any payload. -// This is a major problem if we see only one direction of a connection. -// When analyzing only SYN/FIN/RSTs this leads to differences if we miss -// the SYN/ACK. -// -// To avoid losing payload, there is the option cc_instantiate_on_data: -// if enabled and the originator sends a non-control packet after the -// initial packet, we instantiate full connection state. -// -// - We lose some of the information contained in initial packets (e.g., most -// IP/TCP options and any payload). If you depend on them, you don't -// want to use the compressor. -// -// Optionally, the compressor can take care only of initial SYNs and -// instantiate full connection state for all other connection setups. -// To enable, set cc_handle_only_syns to true. -// -// - The compressor may handle refused connections (i.e., initial packets -// followed by RST from responder) itself. Again, this leads to differences -// from default TCP processing and is therefore turned off by default. -// To enable, set cc_handle_resets to true. -// -// - We don't match signatures on connections which are completely handled -// by the compressor. Matching would require significant additional state -// w/o being very helpful. -// -// - If use_conn_size_analyzer is True, the reported counts for bytes and -// packets may not account for some packets/data that is part of those -// packets which the connection compressor handles. The error, if any, will -// however be small. - - -#ifdef DEBUG -static inline const char* fmt_conn_id(const ConnCompressor::PendingConn* c) - { - if ( c->ip1_is_src ) - return fmt_conn_id(c->key.ip1, c->key.port1, - c->key.ip2, c->key.port2); - else - return fmt_conn_id(c->key.ip2, c->key.port2, - c->key.ip1, c->key.port1); - } - -static inline const char* fmt_conn_id(const Connection* c) - { - return fmt_conn_id(c->OrigAddr(), c->OrigPort(), - c->RespAddr(), c->RespPort()); - } - -static inline const char* fmt_conn_id(const IP_Hdr* ip) - { - const struct tcphdr* tp = (const struct tcphdr*) ip->Payload(); - return fmt_conn_id(ip->SrcAddr(), tp->th_sport, - ip->DstAddr(), tp->th_dport); - } -#endif - -ConnCompressor::ConnCompressor() - { - first_block = last_block = 0; - first_non_expired = 0; - conn_val = 0; - - sizes.connections = sizes.connections_total = 0; - sizes.pending_valid = sizes.pending_total = sizes.pending_in_mem = 0; - sizes.hash_table_size = 0; - sizes.memory = 0; - } - -ConnCompressor::~ConnCompressor() - { - Block* next; - for ( Block* b = first_block; b; b = next ) - { - next = b->next; - delete b; - } - } - -Connection* ConnCompressor::NextPacket(double t, HashKey* key, const IP_Hdr* ip, - const struct pcap_pkthdr* hdr, const u_char* const pkt) - { - // Expire old stuff. - DoExpire(t); - - // Most sanity checks on header sizes are already done ... - const struct tcphdr* tp = (const struct tcphdr*) ip->Payload(); - - // ... except this one. - uint32 tcp_hdr_len = tp->th_off * 4; - if ( tcp_hdr_len > uint32(ip->TotalLen() - ip->HdrLen()) ) - { - sessions->Weird("truncated_header", hdr, pkt); - delete key; - return 0; - } - - bool external = current_iosrc->GetCurrentTag(); - ConnData* c = conns.Lookup(key); - - Unref(conn_val); - conn_val = 0; - - // Do we already have a Connection object? - if ( c && IsConnPtr(c) ) - { - Connection* conn = MakeConnPtr(c); - int consistent = 1; - - if ( external ) - { - // External, and we already have a full connection. - // That means we use the same logic as in NetSessions - // to compare the tags. - consistent = sessions->CheckConnectionTag(conn); - if ( consistent < 0 ) - { - delete key; - return 0; - } - } - - if ( ! consistent || conn->IsReuse(t, ip->Payload()) ) - { - if ( consistent ) - { - DBG_LOG(DBG_COMPRESSOR, "%s reuse", fmt_conn_id(conn)); - conn->Event(connection_reused, 0); - } - - sessions->Remove(conn); - --sizes.connections; - - return Instantiate(t, key, ip); - } - - DBG_LOG(DBG_COMPRESSOR, "%s pass through", fmt_conn_id(conn)); - delete key; - return conn; - } - - PendingConn* pending = c ? MakePendingConnPtr(c) : 0; - - if ( c && external ) - { - // External, but previous packets were not, i.e., they used - // the global timer queue. We finish the old connection - // and instantiate a full one now. - DBG_LOG(DBG_TM, "got packet with tag %s for already" - "known cc connection, instantiating full conn", - current_iosrc->GetCurrentTag()->c_str()); - - Event(pending, 0, connection_attempt, - TCP_ENDPOINT_INACTIVE, 0, TCP_ENDPOINT_INACTIVE); - Event(pending, 0, connection_state_remove, - TCP_ENDPOINT_INACTIVE, 0, TCP_ENDPOINT_INACTIVE); - Remove(key); - - return Instantiate(t, key, ip); - } - - if ( c && pending->invalid && - network_time < pending->time + tcp_session_timer ) - { - // The old connection has terminated sooner than - // tcp_session_timer. We assume this packet to be - // a latecomer, and ignore it. - DBG_LOG(DBG_COMPRESSOR, "%s ignored", fmt_conn_id(pending)); - sessions->DumpPacket(hdr, pkt); - delete key; - return 0; - } - - // Simulate tcp_{reset,close}_delay for initial FINs/RSTs - if ( c && ! pending->invalid && - ((pending->FIN && pending->time + tcp_close_delay < t ) || - (pending->RST && pending->time + tcp_reset_delay < t )) ) - { - DBG_LOG(DBG_COMPRESSOR, "%s closed", fmt_conn_id(pending)); - int orig_state = - pending->FIN ? TCP_ENDPOINT_CLOSED : TCP_ENDPOINT_RESET; - - Event(pending, 0, connection_partial_close, orig_state, - ip->PayloadLen() - (tp->th_off * 4), - TCP_ENDPOINT_INACTIVE); - Event(pending, 0, connection_state_remove, orig_state, - ip->PayloadLen() - (tp->th_off * 4), - TCP_ENDPOINT_INACTIVE); - - Remove(key); - - Connection* tc = FirstFromOrig(t, key, ip, tp); - if ( ! tc ) - { - delete key; - sessions->DumpPacket(hdr, pkt); - } - - return tc; - } - - Connection* tc; - - if ( ! c || pending->invalid ) - { - // First packet of a connection. - if ( c ) - Remove(key); - - if ( external ) - // External, we directly instantiate a full connection. - tc = Instantiate(t, key, ip); - else - tc = FirstFromOrig(t, key, ip, tp); - } - - else if ( addr_eq(ip->SrcAddr(), SrcAddr(pending)) && - tp->th_sport == SrcPort(pending) ) - // Another packet from originator. - tc = NextFromOrig(pending, t, key, ip, tp); - - else - // A reply. - tc = Response(pending, t, key, ip, tp); - - if ( ! tc ) - { - delete key; - sessions->DumpPacket(hdr, pkt); - } - - return tc; - } - -static int parse_tcp_options(unsigned int opt, unsigned int optlen, - const u_char* option, TCP_Analyzer* analyzer, - bool is_orig, void* cookie) - { - ConnCompressor::PendingConn* c = (ConnCompressor::PendingConn*) cookie; - - // We're only interested in window_scale. - if ( opt == 3 ) - c->window_scale = option[2]; - - return 0; - } - -Connection* ConnCompressor::FirstFromOrig(double t, HashKey* key, - const IP_Hdr* ip, const tcphdr* tp) - { - if ( cc_handle_only_syns && ! (tp->th_flags & TH_SYN) ) - return Instantiate(t, key, ip); - - // The first packet of a connection. - PendingConn* pending = MakeNewState(t); - PktHdrToPendingConn(t, key, ip, tp, pending); - - DBG_LOG(DBG_COMPRESSOR, "%s our", fmt_conn_id(pending)); - - // The created DictEntry will point directly into our PendingConn. - // So, we have to be careful when we delete it. - conns.Dictionary::Insert(&pending->key, sizeof(pending->key), - pending->hash, MakeMapPtr(pending), 0); - - // Mimic some of TCP_Analyzer's weirds for SYNs. - // To be completely precise, we'd need to check this at a few - // more locations in NextFromOrig() and Reply(). However, that - // does not really seem worth it, as this is the standard case. - if ( tp->th_flags & TH_SYN ) - { - if ( tp->th_flags & TH_RST ) - Weird(pending, t, "TCP_christmas"); - - if ( tp->th_flags & TH_URG ) - Weird(pending, t, "baroque_SYN"); - - int len = ip->TotalLen() - ip->HdrLen() - tp->th_off * 4; - - if ( len > 0 ) - // T/TCP definitely complicates this. - Weird(pending, t, "SYN_with_data"); - } - - if ( tp->th_flags & TH_FIN ) - { - if ( ! (tp->th_flags & TH_SYN) ) - Weird(pending, t, "spontaneous_FIN"); - } - - if ( tp->th_flags & TH_RST ) - { - if ( ! (tp->th_flags & TH_SYN) ) - Weird(pending, t, "spontaneous_RST"); - } - - ++sizes.pending_valid; - ++sizes.pending_total; - ++sizes.pending_in_mem; - - Event(pending, 0, new_connection, - TCP_ENDPOINT_INACTIVE, 0, TCP_ENDPOINT_INACTIVE); - - if ( current_iosrc->GetCurrentTag() ) - { - Val* tag = - new StringVal(current_iosrc->GetCurrentTag()->c_str()); - Event(pending, 0, connection_external, - TCP_ENDPOINT_INACTIVE, 0, TCP_ENDPOINT_INACTIVE, tag); - } - - return 0; - } - -Connection* ConnCompressor::NextFromOrig(PendingConn* pending, double t, - HashKey* key, const IP_Hdr* ip, - const tcphdr* tp) - { - // Another packet from the same host without seeing an answer so far. - DBG_LOG(DBG_COMPRESSOR, "%s same again", fmt_conn_id(pending)); - - ++pending->num_pkts; - ++pending->num_bytes_ip += ip->PayloadLen(); - - // New window scale overrides old - not great, this is a (subtle) - // evasion opportunity. - if ( TCP_Analyzer::ParseTCPOptions(tp, parse_tcp_options, 0, 0, - pending) < 0 ) - Weird(pending, t, "corrupt_tcp_options"); - - if ( tp->th_flags & TH_SYN ) - // New seq overrides old. - pending->seq = tp->th_seq; - - // Mimic TCP_Endpoint::Size() - int size = ntohl(tp->th_seq) - ntohl(pending->seq); - if ( size != 0 ) - --size; - - if ( size != 0 && (pending->FIN || (tp->th_flags & TH_FIN)) ) - --size; - - if ( size < 0 ) - // We only care about the size for broken connections. - // Surely for those it's more likely that the sequence - // numbers are confused than that they really transferred - // > 2 GB of data. Plus, for 64-bit ints these sign-extend - // up to truly huge, non-sensical unsigned values. - size = 0; - - if ( pending->SYN ) - { - // We're in state SYN_SENT or SYN_ACK_SENT. - if ( tp->th_flags & TH_RST) - { - Event(pending, t, connection_reset, - TCP_ENDPOINT_RESET, size, TCP_ENDPOINT_INACTIVE); - Event(pending, t, connection_state_remove, - TCP_ENDPOINT_RESET, size, TCP_ENDPOINT_INACTIVE); - - Invalidate(key); - return 0; - } - - else if ( tp->th_flags & TH_FIN) - { - Event(pending, t, connection_partial_close, - TCP_ENDPOINT_CLOSED, size, TCP_ENDPOINT_INACTIVE); - Event(pending, t, connection_state_remove, - TCP_ENDPOINT_CLOSED, size, TCP_ENDPOINT_INACTIVE); - Invalidate(key); - return 0; - } - - else if ( tp->th_flags & TH_SYN ) - { - if ( (tp->th_flags & TH_ACK) && ! pending->ACK ) - Weird(pending, t, "repeated_SYN_with_ack"); - } - - else - { - // A data packet without seeing a SYN/ACK first. As - // long as we stick with the principle of instantiating - // state only when we see a reply, we have to throw - // this data away. Optionally we may instantiate a - // real connection now. - - if ( cc_instantiate_on_data ) - return Instantiate(key, pending); - // else - // Weird(pending, t, "data_without_SYN_ACK"); - } - } - - else - { // We're in state INACTIVE. - if ( tp->th_flags & TH_RST) - { - Event(pending, t, connection_reset, - TCP_ENDPOINT_RESET, size, TCP_ENDPOINT_INACTIVE); - Event(pending, t, connection_state_remove, - TCP_ENDPOINT_RESET, size, TCP_ENDPOINT_INACTIVE); - - Invalidate(key); - return 0; - } - - else if ( tp->th_flags & TH_FIN) - { - Event(pending, t, connection_half_finished, - TCP_ENDPOINT_CLOSED, size, TCP_ENDPOINT_INACTIVE); - Event(pending, t, connection_state_remove, - TCP_ENDPOINT_CLOSED, size, TCP_ENDPOINT_INACTIVE); - - Invalidate(key); - return 0; - } - - else if ( tp->th_flags & TH_SYN ) - { - if ( ! tp->th_flags & TH_ACK ) - { - Weird(pending, t, "SYN_after_partial"); - pending->SYN = 1; - } - } - - else - // Another data packet. See discussion above. - if ( cc_instantiate_on_data ) - return Instantiate(key, pending); - - // else - // Weird(pending, t, "data_without_SYN_ACK"); - } - - return 0; - } - -Connection* ConnCompressor::Response(PendingConn* pending, double t, - HashKey* key, const IP_Hdr* ip, - const tcphdr* tp) - { - // The packet comes from the former responder. That means we are - // seeing a reply, so we are going to create a "real" connection now. - DBG_LOG(DBG_COMPRESSOR, "%s response", fmt_conn_id(pending)); - - // Optional: if it's a RST after SYN, we directly generate a - // connection_rejected and throw the state away. - if ( cc_handle_resets && (tp->th_flags & TH_RST) && pending->SYN ) - { - // See discussion of size in DoExpire(). - DBG_LOG(DBG_COMPRESSOR, "%s reset", fmt_conn_id(pending)); - - Event(pending, t, connection_reset, - TCP_ENDPOINT_SYN_SENT, 0, TCP_ENDPOINT_RESET); - Event(pending, t, connection_state_remove, - TCP_ENDPOINT_SYN_SENT, 0, TCP_ENDPOINT_RESET); - - Invalidate(key); - return 0; - } - - // If a connection's initial packet is a RST, Bro's standard TCP - // processing considers the connection done right away. We simulate - // this by instantiating a second connection in this case. The - // first one will time out eventually. - if ( pending->RST && ! pending->SYN ) - { - int orig_state = - pending->RST ? TCP_ENDPOINT_RESET : TCP_ENDPOINT_CLOSED; - Event(pending, 0, connection_attempt, - orig_state, 0, TCP_ENDPOINT_INACTIVE); - Event(pending, 0, connection_state_remove, - orig_state, 0, TCP_ENDPOINT_INACTIVE); - - // Override with current packet. - PktHdrToPendingConn(t, key, ip, tp, pending); - return 0; - } - - return Instantiate(key, pending); - } - -Connection* ConnCompressor::Instantiate(HashKey* key, PendingConn* pending) - { - // Instantantiate a Connection. - ConnID conn_id; - conn_id.src_addr = SrcAddr(pending); - conn_id.dst_addr = DstAddr(pending); - conn_id.src_port = SrcPort(pending); - conn_id.dst_port = DstPort(pending); - - pending->invalid = 1; - --sizes.pending_valid; - --sizes.pending_total; - - // Fake the first packet. - const IP_Hdr* faked_pkt = PendingConnToPacket(pending); - Connection* new_conn = sessions->NewConn(key, pending->time, &conn_id, - faked_pkt->Payload(), IPPROTO_TCP); - - if ( ! new_conn ) - { - // This connection is not to be analyzed (e.g., it may be - // a partial one). - DBG_LOG(DBG_COMPRESSOR, "%s nop", fmt_conn_id(pending)); - return 0; - } - - new_conn->SetUID(pending->uid); - - DBG_LOG(DBG_COMPRESSOR, "%s instantiated", fmt_conn_id(pending)); - - ++sizes.connections; - ++sizes.connections_total; - - if ( new_packet ) - new_conn->Event(new_packet, 0, - sessions->BuildHeader(faked_pkt->IP4_Hdr())); - - // NewConn() may have swapped originator and responder. - int is_orig = addr_eq(conn_id.src_addr, new_conn->OrigAddr()) && - conn_id.src_port == new_conn->OrigPort(); - - // Pass the faked packet to the connection. - const u_char* payload = faked_pkt->Payload(); - - int dummy_record_packet, dummy_record_content; - new_conn->NextPacket(pending->time, is_orig, - faked_pkt, faked_pkt->PayloadLen(), - faked_pkt->PayloadLen(), payload, - dummy_record_packet, dummy_record_content, 0, 0, 0); - - // Removing necessary because the key will be destroyed at some point. - conns.Remove(&pending->key, sizeof(pending->key), pending->hash, true); - conns.Insert(key, MakeMapPtr(new_conn)); - - return new_conn; - } - -Connection* ConnCompressor::Instantiate(double t, HashKey* key, - const IP_Hdr* ip) - { - const struct tcphdr* tp = (const struct tcphdr*) ip->Payload(); - - ConnID conn_id; - conn_id.src_addr = ip->SrcAddr(); - conn_id.dst_addr = ip->DstAddr(); - conn_id.src_port = tp->th_sport; - conn_id.dst_port = tp->th_dport; - - Connection* new_conn = - sessions->NewConn(key, t, &conn_id, ip->Payload(), IPPROTO_TCP); - - if ( ! new_conn ) - { - // This connection is not to be analyzed (e.g., it may be - // a partial one). - DBG_LOG(DBG_COMPRESSOR, "%s nop", fmt_conn_id(ip)); - return 0; - } - - DBG_LOG(DBG_COMPRESSOR, "%s instantiated", fmt_conn_id(ip)); - - conns.Insert(key, MakeMapPtr(new_conn)); - ++sizes.connections; - ++sizes.connections_total; - - if ( new_connection ) - new_conn->Event(new_connection, 0); - - if ( current_iosrc->GetCurrentTag() ) - { - Val* tag = - new StringVal(current_iosrc->GetCurrentTag()->c_str()); - new_conn->Event(connection_external, 0, tag); - } - - return new_conn; - } - -void ConnCompressor::PktHdrToPendingConn(double time, const HashKey* key, - const IP_Hdr* ip, const struct tcphdr* tp, PendingConn* c) - { - memcpy(&c->key, key->Key(), key->Size()); - - c->hash = key->Hash(); - c->ip1_is_src = addr_eq(c->key.ip1, ip->SrcAddr()) && - c->key.port1 == tp->th_sport; - c->time = time; - c->window = tp->th_win; - c->seq = tp->th_seq; - c->ack = tp->th_ack; - c->window_scale = 0; - c->SYN = (tp->th_flags & TH_SYN) != 0; - c->FIN = (tp->th_flags & TH_FIN) != 0; - c->RST = (tp->th_flags & TH_RST) != 0; - c->ACK = (tp->th_flags & TH_ACK) != 0; - c->uid = calculate_unique_id(); - c->num_bytes_ip = ip->TotalLen(); - c->num_pkts = 1; - c->invalid = 0; - - if ( TCP_Analyzer::ParseTCPOptions(tp, parse_tcp_options, 0, 0, c) < 0 ) - sessions->Weird("corrupt_tcp_options", ip); - } - -// Fakes an empty TCP packet based on the information in PendingConn. -const IP_Hdr* ConnCompressor::PendingConnToPacket(const PendingConn* c) - { - static ip* ip = 0; - static tcphdr* tp = 0; - static IP_Hdr* ip_hdr = 0; - - if ( ! ip ) - { // Initialize. ### Note, only handles IPv4 for now. - int packet_length = sizeof(*ip) + sizeof(*tp); - ip = (struct ip*) new char[packet_length]; - tp = (struct tcphdr*) (((char*) ip) + sizeof(*ip)); - ip_hdr = new IP_Hdr(ip); - - // Constant fields. - ip->ip_v = 4; - ip->ip_hl = sizeof(*ip) / 4; // no options - ip->ip_tos = 0; - ip->ip_len = htons(packet_length); - ip->ip_id = 0; - ip->ip_off = 0; - ip->ip_ttl = 255; - ip->ip_p = IPPROTO_TCP; - ip->ip_sum = 0; // is not going to be checked - - tp->th_off = sizeof(*tp) / 4; // no options for now - tp->th_urp = 0; - } - - // Note, do *not* use copy_addr() here. This is because we're - // copying to an IPv4 header, which has room for exactly and - // only an IPv4 address. -#ifdef BROv6 - if ( ! is_v4_addr(c->key.ip1) || ! is_v4_addr(c->key.ip2) ) - reporter->InternalError("IPv6 snuck into connection compressor"); -#endif - *(uint32*) &ip->ip_src = - to_v4_addr(c->ip1_is_src ? c->key.ip1 : c->key.ip2); - *(uint32*) &ip->ip_dst = - to_v4_addr(c->ip1_is_src ? c->key.ip2 : c->key.ip1); - - if ( c->ip1_is_src ) - { - tp->th_sport = c->key.port1; - tp->th_dport = c->key.port2; - } - else - { - tp->th_sport = c->key.port2; - tp->th_dport = c->key.port1; - } - - tp->th_win = c->window; - tp->th_seq = c->seq; - tp->th_ack = c->ack; - tp->th_flags = MakeFlags(c); - tp->th_sum = 0; - tp->th_sum = 0xffff - tcp_checksum(ip, tp, 0); - - // FIXME: Add TCP options. - return ip_hdr; - } - -uint8 ConnCompressor::MakeFlags(const PendingConn* c) const - { - uint8 tcp_flags = 0; - if ( c->SYN ) - tcp_flags |= TH_SYN; - if ( c->FIN ) - tcp_flags |= TH_FIN; - if ( c->RST ) - tcp_flags |= TH_RST; - if ( c->ACK ) - tcp_flags |= TH_ACK; - - return tcp_flags; - } - -ConnCompressor::PendingConn* ConnCompressor::MakeNewState(double t) - { - // See if there is enough space in the current block. - if ( last_block && - int(sizeof(PendingConn)) <= BLOCK_SIZE - last_block->bytes_used ) - { - PendingConn* c = (PendingConn*) &last_block->data[last_block->bytes_used]; - last_block->bytes_used += sizeof(PendingConn); - c->is_pending = true; - return c; - } - - // Get new block. - Block* b = new Block; - b->time = t; - b->bytes_used = sizeof(PendingConn); - b->next = 0; - b->prev = last_block; - - if ( last_block ) - last_block->next = b; - else - first_block = b; - - last_block = b; - - sizes.memory += padded_sizeof(*b); - PendingConn* c = (PendingConn*) &b->data; - c->is_pending = true; - return c; - } - -void ConnCompressor::DoExpire(double t) - { - while ( first_block ) - { - Block* b = first_block; - - unsigned char* p = - first_non_expired ? first_non_expired : b->data; - - while ( p < b->data + b->bytes_used ) - { - Unref(conn_val); - conn_val = 0; - - PendingConn* c = (PendingConn*) p; - if ( t && (c->time + tcp_SYN_timeout > t) ) - { - // All following entries are still - // recent enough. - first_non_expired = p; - return; - } - - if ( ! c->invalid ) - { - // Expired. - DBG_LOG(DBG_COMPRESSOR, "%s expire", fmt_conn_id(c)); - - HashKey key(&c->key, sizeof(c->key), c->hash, true); - - ConnData* cd = conns.Lookup(&key); - if ( cd && ! IsConnPtr(cd) ) - conns.Remove(&c->key, sizeof(c->key), - c->hash, true); - - int orig_state = TCP_ENDPOINT_INACTIVE; - - if ( c->FIN ) - orig_state = TCP_ENDPOINT_CLOSED; - if ( c->RST ) - orig_state = TCP_ENDPOINT_RESET; - if ( c->SYN ) - orig_state = TCP_ENDPOINT_SYN_SENT; - - // We're not able to get the correct size - // here (with "correct" meaning value that - // standard connection processing reports). - // We could if would also store last_seq, but - // doesn't seem worth it. - - Event(c, 0, connection_attempt, - orig_state, 0, TCP_ENDPOINT_INACTIVE); - Event(c, 0, connection_state_remove, - orig_state, 0, TCP_ENDPOINT_INACTIVE); - - c->invalid = 1; - --sizes.pending_valid; - } - - p += sizeof(PendingConn); - --sizes.pending_in_mem; - } - - // Full block expired, so delete it. - first_block = b->next; - - if ( b->next ) - b->next->prev = 0; - else - last_block = 0; - - delete b; - - first_non_expired = 0; - sizes.memory -= padded_sizeof(*b); - } - } - -void ConnCompressor::Event(const PendingConn* pending, double t, - const EventHandlerPtr& event, int orig_state, - int orig_size, int resp_state, Val* arg) - { - if ( ! conn_val ) - { - if ( ! event ) - return; - - // We only raise events if NewConn() would have actually - // instantiated the Connection. - bool flip_roles; - if ( ! sessions->WantConnection(ntohs(SrcPort(pending)), - ntohs(DstPort(pending)), - TRANSPORT_TCP, - MakeFlags(pending), - flip_roles) ) - return; - - conn_val = new RecordVal(connection_type); - RecordVal* id_val = new RecordVal(conn_id); - RecordVal* orig_endp = new RecordVal(endpoint); - RecordVal* resp_endp = new RecordVal(endpoint); - - if ( orig_state == TCP_ENDPOINT_INACTIVE ) - { - if ( pending->SYN ) - orig_state = pending->ACK ? - TCP_ENDPOINT_SYN_ACK_SENT : - TCP_ENDPOINT_SYN_SENT; - else - orig_state = TCP_ENDPOINT_PARTIAL; - } - - int tcp_state = TCP_ENDPOINT_INACTIVE; - - if ( ! flip_roles ) - { - id_val->Assign(0, new AddrVal(SrcAddr(pending))); - id_val->Assign(1, new PortVal(ntohs(SrcPort(pending)), - TRANSPORT_TCP)); - id_val->Assign(2, new AddrVal(DstAddr(pending))); - id_val->Assign(3, new PortVal(ntohs(DstPort(pending)), - TRANSPORT_TCP)); - orig_endp->Assign(0, new Val(orig_size, TYPE_COUNT)); - orig_endp->Assign(1, new Val(orig_state, TYPE_COUNT)); - - if ( ConnSize_Analyzer::Available() ) - { - // Fill in optional fields if ConnSize_Analyzer is on. - orig_endp->Assign(2, new Val(pending->num_pkts, TYPE_COUNT)); - orig_endp->Assign(3, new Val(pending->num_bytes_ip, TYPE_COUNT)); - } - - resp_endp->Assign(0, new Val(0, TYPE_COUNT)); - resp_endp->Assign(1, new Val(resp_state, TYPE_COUNT)); - resp_endp->Assign(2, new Val(0, TYPE_COUNT)); - resp_endp->Assign(3, new Val(0, TYPE_COUNT)); - } - else - { - id_val->Assign(0, new AddrVal(DstAddr(pending))); - id_val->Assign(1, new PortVal(ntohs(DstPort(pending)), - TRANSPORT_TCP)); - id_val->Assign(2, new AddrVal(SrcAddr(pending))); - id_val->Assign(3, new PortVal(ntohs(SrcPort(pending)), - TRANSPORT_TCP)); - - orig_endp->Assign(0, new Val(0, TYPE_COUNT)); - orig_endp->Assign(1, new Val(resp_state, TYPE_COUNT)); - orig_endp->Assign(2, new Val(0, TYPE_COUNT)); - orig_endp->Assign(3, new Val(0, TYPE_COUNT)); - - resp_endp->Assign(0, new Val(orig_size, TYPE_COUNT)); - resp_endp->Assign(1, new Val(orig_state, TYPE_COUNT)); - - if ( ConnSize_Analyzer::Available() ) - { - // Fill in optional fields if ConnSize_Analyzer is on - resp_endp->Assign(2, new Val(pending->num_pkts, TYPE_COUNT)); - resp_endp->Assign(3, new Val(pending->num_bytes_ip, TYPE_COUNT)); - } - - DBG_LOG(DBG_COMPRESSOR, "%s swapped direction", fmt_conn_id(pending)); - } - - conn_val->Assign(0, id_val); - conn_val->Assign(1, orig_endp); - conn_val->Assign(2, resp_endp); - conn_val->Assign(3, new Val(pending->time, TYPE_TIME)); - conn_val->Assign(4, new Val(t > 0 ? t - pending->time : 0, - TYPE_INTERVAL)); // duration - conn_val->Assign(5, new TableVal(string_set)); // service - conn_val->Assign(6, new StringVal("cc=1")); // addl - conn_val->Assign(7, new Val(0, TYPE_COUNT)); // hot - conn_val->Assign(8, new StringVal("")); // history - - char tmp[20]; // uid. - conn_val->Assign(9, new StringVal(uitoa_n(pending->uid, tmp, sizeof(tmp), 62))); - - conn_val->SetOrigin(0); - } - - if ( event == conn_weird ) - { - // Special case to go through the logger. - const char* msg = arg->AsString()->CheckString(); - reporter->Weird(conn_val->Ref(), msg); - return; - } - - val_list* vl = new val_list; - if ( arg ) - vl->append(arg); - vl->append(conn_val->Ref()); - - mgr.QueueEvent(event, vl, SOURCE_LOCAL); - } - -void ConnCompressor::Drain() - { - IterCookie* cookie = conns.InitForIteration(); - ConnData* c; - - DoExpire(0); - - while ( (c = conns.NextEntry(cookie)) ) - { - Unref(conn_val); - conn_val = 0; - - if ( IsConnPtr(c) ) - { - Connection* tc = MakeConnPtr(c); - tc->Done(); - tc->Event(connection_state_remove, 0); - Unref(tc); - --sizes.connections; - } - - else - { - PendingConn* pc = MakePendingConnPtr(c); - if ( ! pc->invalid ) - { - // Same discussion for size here than - // in DoExpire(). - Event(pc, 0, connection_attempt, - TCP_ENDPOINT_INACTIVE, 0, - TCP_ENDPOINT_INACTIVE); - Event(pc, 0, connection_state_remove, - TCP_ENDPOINT_INACTIVE, 0, - TCP_ENDPOINT_INACTIVE); - - --sizes.pending_valid; - pc->invalid = 1; - } - } - } - } - -void ConnCompressor::Invalidate(HashKey* k) - { - ConnData* c = (ConnData*) conns.Lookup(k); - - assert(c && ! IsConnPtr(c)); - PendingConn* pc = MakePendingConnPtr(c); - - DBG_LOG(DBG_COMPRESSOR, "%s invalidate", fmt_conn_id(pc)); - - if ( ! pc->invalid ) - { - conns.Remove(&pc->key, sizeof(pc->key), pc->hash, true); - pc->invalid = 1; - --sizes.pending_valid; - } - } - -Connection* ConnCompressor::Insert(Connection* newconn) - { - HashKey* key = newconn->Key(); - ConnData* c = conns.Lookup(key); - Connection* old = 0; - - // Do we already have a Connection object? - if ( c ) - { - if ( IsConnPtr(c) ) - old = MakeConnPtr(c); - Remove(key); - } - - conns.Insert(key, MakeMapPtr(newconn)); - return old; - } - -bool ConnCompressor::Remove(HashKey* k) - { - ConnData* c = (ConnData*) conns.Lookup(k); - if ( ! c ) - return false; - - if ( IsConnPtr(c) ) - { - DBG_LOG(DBG_COMPRESSOR, "%s remove", fmt_conn_id(MakeConnPtr(c))); - conns.Remove(k); - --sizes.connections; - } - else - { - PendingConn* pc = MakePendingConnPtr(c); - DBG_LOG(DBG_COMPRESSOR, "%s remove", fmt_conn_id(pc)); - - conns.Remove(&pc->key, sizeof(pc->key), pc->hash, true); - - if ( ! pc->invalid ) - { - pc->invalid = 1; - --sizes.pending_valid; - } - } - - return true; - } diff --git a/src/ConnCompressor.h b/src/ConnCompressor.h deleted file mode 100644 index 36959b615c..0000000000 --- a/src/ConnCompressor.h +++ /dev/null @@ -1,235 +0,0 @@ -// The ConnCompressor keeps track of the first packet seen for a conn_id using -// only a minimal amount of memory. This helps us to avoid instantiating -// full Connection objects for never-established sessions. -// -// TCP only. - -#ifndef CONNCOMPRESSOR_H -#define CONNCOMPRESSOR_H - -#include "Conn.h" -#include "Dict.h" -#include "NetVar.h" -#include "TCP.h" - -class ConnCompressor { -public: - ConnCompressor(); - ~ConnCompressor(); - - // Handle next packet. Returns 0 if packet in handled internally. - // Takes ownership of key. - Connection* NextPacket(double t, HashKey* k, const IP_Hdr* ip_hdr, - const struct pcap_pkthdr* hdr, const u_char* const pkt); - - // Look up a connection. Returns non-nil for connections for - // which a Connection object has already been instantiated. - Connection* Lookup(HashKey* k) - { - ConnData* c = conns.Lookup(k); - return c && IsConnPtr(c) ? MakeConnPtr(c) : 0; - } - - // Inserts connection into compressor. If another entry with this key - // already exists, it's replaced. If that was a full connection, it is - // also returned. - Connection* Insert(Connection* c); - - // Remove all state belonging to the given connection. Returns - // true if the connection was found in the compressor's table, - // false if not. - bool Remove(HashKey* k); - - // Flush state. - void Drain(); - - struct Sizes { - // Current number of already fully instantiated connections. - unsigned int connections; - - // Total number of fully instantiated connections. - unsigned int connections_total; - - // Current number of seen but non-yet instantiated connections. - unsigned int pending_valid; - - // Total number of seen but non-yet instantiated connections. - unsigned int pending_total; - - // Total number of all entries in pending list (some a which - // may already been invalid, but not yet removed from memory). - unsigned int pending_in_mem; - - // Total number of hash table entires - // (should equal connections + pending_valid) - unsigned int hash_table_size; - - // Total memory usage; - unsigned int memory; - }; - - const Sizes& Size() - { sizes.hash_table_size = conns.Length(); return sizes; } - - unsigned int MemoryAllocation() const { return sizes.memory; } - - // As long as we have only seen packets from one side, we just - // store a PendingConn. - struct PendingConn { - // True if the block is indeed a PendingConn (see below). - unsigned int is_pending:1; - - // Whether roles in key are flipped. - unsigned int ip1_is_src:1; - - unsigned int invalid:1; // deleted - int window_scale:4; - unsigned int SYN:1; - unsigned int FIN:1; - unsigned int RST:1; - unsigned int ACK:1; - - double time; - ConnID::Key key; - uint32 seq; - uint32 ack; - hash_t hash; - uint16 window; - uint64 uid; - - // The following are set if use_conn_size_analyzer is T. - uint16 num_pkts; - uint16 num_bytes_ip; - }; - -private: - // Helpers to extract addrs/ports from PendingConn. - - const uint32* SrcAddr(const PendingConn* c) - { return c->ip1_is_src ? c->key.ip1 : c->key.ip2; } - const uint32* DstAddr(const PendingConn* c) - { return c->ip1_is_src ? c->key.ip2 : c->key.ip1; } - - uint16 SrcPort(const PendingConn* c) - { return c->ip1_is_src ? c->key.port1 : c->key.port2; } - uint16 DstPort(const PendingConn* c) - { return c->ip1_is_src ? c->key.port2 : c->key.port1; } - - - // Called for the first packet in a connection. - Connection* FirstFromOrig(double t, HashKey* key, - const IP_Hdr* ip, const tcphdr* tp); - - // Called for more packets from the orginator w/o seeing a response. - Connection* NextFromOrig(PendingConn* pending, double t, HashKey* key, - const IP_Hdr* ip, const tcphdr* tp); - - // Called for the first response packet. Instantiates a Connection. - Connection* Response(PendingConn* pending, double t, HashKey* key, - const IP_Hdr* ip, const tcphdr* tp); - - // Instantiates a full TCP connection (invalidates pending connection). - Connection* Instantiate(HashKey* key, PendingConn* pending); - - // Same but based on packet. - Connection* Instantiate(double t, HashKey* key, const IP_Hdr* ip); - - // Fills the attributes of a PendingConn based on the given arguments. - void PktHdrToPendingConn(double time, const HashKey* key, - const IP_Hdr* ip, const struct tcphdr* tp, PendingConn* c); - - // Fakes a TCP packet based on the available information. - const IP_Hdr* PendingConnToPacket(const PendingConn* c); - - // Construct a TCP-flags byte. - uint8 MakeFlags(const PendingConn* c) const; - - // Allocate room for a new (Ext)PendingConn. - PendingConn* MakeNewState(double t); - - // Expire PendingConns. - void DoExpire(double t); - - // Remove all state belonging to the given connection. - void Invalidate(HashKey* k); - - // Sends the given connection_* event. If orig_state is - // TCP_ENDPOINT__INACTIVE, tries to guess a better one based - // on pending. If arg in non-nil, it will be used as the - // *first* argument of the event call (this is for conn_weird()). - void Event(const PendingConn* pending, double t, - const EventHandlerPtr& event, int orig_state, - int orig_size, int resp_state, Val* arg = 0); - - void Weird(const PendingConn* pending, double t, const char* msg) - { - // This will actually go through the Reporter; Event() takes - // care of that. - Event(pending, t, conn_weird, TCP_ENDPOINT_INACTIVE, 0, - TCP_ENDPOINT_INACTIVE, new StringVal(msg)); - } - - static const int BLOCK_SIZE = 16 * 1024; - - // The memory managment for PendConns. - struct Block { - double time; - Block* prev; - Block* next; - int bytes_used; - unsigned char data[BLOCK_SIZE]; - }; - - // In the connection hash table, we store pointers to both PendingConns - // and Connections. Thus, we need a way to differentiate between - // these two types. To avoid an additional indirection, we use a little - // hack: a pointer retrieved from the table is interpreted as a - // PendingConn first. However, if is_pending is false, it's in fact a - // Connection which starts at offset 4. The methods below help to - // implement this scheme transparently. An "operator new" in - // Connection takes care of building Connection's accordingly. - typedef PendingConn ConnData; - declare(PDict, ConnData); - typedef PDict(ConnData) ConnMap; - ConnMap conns; - - static ConnData* MakeMapPtr(PendingConn* c) - { assert(c->is_pending); return c; } - - static ConnData* MakeMapPtr(Connection* c) - { - ConnData* p = (ConnData*) (((char*) c) - 4); - assert(!p->is_pending); - return p; - } - - static PendingConn* MakePendingConnPtr(ConnData* c) - { assert(c->is_pending); return c; } - - static Connection* MakeConnPtr(ConnData* c) - { - assert(!c->is_pending); - return (Connection*) (((char*) c) + 4); - } - - static bool IsConnPtr(ConnData* c) - { return ! c->is_pending; } - - // New blocks are inserted at the end. - Block* first_block; - Block* last_block; - - // If we have already expired some entries in a block, - // this points to the first non-expired. - unsigned char* first_non_expired; - - // Last "connection" that we have build. - RecordVal* conn_val; - - // Statistics. - Sizes sizes; - }; - -extern ConnCompressor* conn_compressor; - -#endif diff --git a/src/DCE_RPC.cc b/src/DCE_RPC.cc index 1d9acaf1fa..21cb3be9a0 100644 --- a/src/DCE_RPC.cc +++ b/src/DCE_RPC.cc @@ -137,12 +137,12 @@ static bool is_mapped_dce_rpc_endpoint(const dce_rpc_endpoint_addr& addr) bool is_mapped_dce_rpc_endpoint(const ConnID* id, TransportProto proto) { -#ifdef BROv6 - if ( ! is_v4_addr(id->dst_addr) ) + if ( id->dst_addr.GetFamily() == IPv6 ) + // TODO: Does the protocol support v6 addresses? #773 return false; -#endif + dce_rpc_endpoint_addr addr; - addr.addr = ntohl(to_v4_addr(id->dst_addr)); + addr.addr = id->dst_addr; addr.port = ntohs(id->dst_port); addr.proto = proto; @@ -160,12 +160,7 @@ static void add_dce_rpc_endpoint(const dce_rpc_endpoint_addr& addr, // of the dce_rpc_endpoints table. // FIXME: Don't hard-code the timeout. - // Convert the address to a v4/v6 address (depending on how - // Bro was configured). This is all based on the address currently - // being a 32-bit host-order v4 address. - AddrVal a(htonl(addr.addr)); - const addr_type at = a.AsAddr(); - dpm->ExpectConnection(0, at, addr.port, addr.proto, + dpm->ExpectConnection(IPAddr(), addr.addr, addr.port, addr.proto, AnalyzerTag::DCE_RPC, 5 * 60, 0); } @@ -418,8 +413,8 @@ void DCE_RPC_Session::DeliverEpmapperMapResponse( break; case binpac::DCE_RPC_Simple::EPM_PROTOCOL_IP: - mapped.addr.addr = - floor->rhs()->data()->ip(); + uint32 hostip = floor->rhs()->data()->ip(); + mapped.addr.addr = IPAddr(IPv4, &hostip, IPAddr::Host); break; } } @@ -433,7 +428,7 @@ void DCE_RPC_Session::DeliverEpmapperMapResponse( vl->append(analyzer->BuildConnVal()); vl->append(new StringVal(mapped.uuid.to_string())); vl->append(new PortVal(mapped.addr.port, mapped.addr.proto)); - vl->append(new AddrVal(htonl(mapped.addr.addr))); + vl->append(new AddrVal(mapped.addr.addr)); analyzer->ConnectionEvent(epm_map_response, vl); } diff --git a/src/DCE_RPC.h b/src/DCE_RPC.h index 63237a151b..acdbf1637d 100644 --- a/src/DCE_RPC.h +++ b/src/DCE_RPC.h @@ -8,6 +8,7 @@ #include "NetVar.h" #include "TCP.h" +#include "IPAddr.h" #include "dce_rpc_simple_pac.h" @@ -34,19 +35,19 @@ const char* uuid_to_string(const u_char* uuid_data); struct dce_rpc_endpoint_addr { // All fields are in host byteorder. - uint32 addr; + IPAddr addr; u_short port; TransportProto proto; dce_rpc_endpoint_addr() { - addr = 0; + addr = IPAddr(); port = 0; proto = TRANSPORT_UNKNOWN; } bool is_valid_addr() const - { return addr != 0 && port != 0 && proto != TRANSPORT_UNKNOWN; } + { return addr != IPAddr() && port != 0 && proto != TRANSPORT_UNKNOWN; } bool operator<(dce_rpc_endpoint_addr const &e) const { @@ -64,7 +65,7 @@ struct dce_rpc_endpoint_addr { { static char buf[128]; snprintf(buf, sizeof(buf), "%s/%d/%s", - dotted_addr(htonl(addr)), port, + addr.AsString().c_str(), port, proto == TRANSPORT_TCP ? "tcp" : (proto == TRANSPORT_UDP ? "udp" : "?")); diff --git a/src/DNS.cc b/src/DNS.cc index c93ea6973d..a3b0b62ef3 100644 --- a/src/DNS.cc +++ b/src/DNS.cc @@ -758,62 +758,37 @@ int DNS_Interpreter::ParseRR_A(DNS_MsgInfo* msg, int DNS_Interpreter::ParseRR_AAAA(DNS_MsgInfo* msg, const u_char*& data, int& len, int rdlength) { - // We need to parse an IPv6 address, high-order byte first. - // ### Currently, we fake an A reply rather than an AAAA reply, - // since for the latter we won't be able to express the full - // address (unless Bro was compiled for IPv6 addresses). We do - // this fake by using just the bottom 4 bytes of the IPv6 address. uint32 addr[4]; - int i; - for ( i = 0; i < 4; ++i ) + for ( int i = 0; i < 4; ++i ) { - addr[i] = ntohl(ExtractLong(data, len)); + addr[i] = htonl(ExtractLong(data, len)); if ( len < 0 ) { - analyzer->Weird("DNS_AAAA_neg_length"); + if ( msg->atype == TYPE_AAAA ) + analyzer->Weird("DNS_AAAA_neg_length"); + else + analyzer->Weird("DNS_A6_neg_length"); return 0; } } - // Currently, dns_AAAA_reply is treated like dns_A_reply, since - // IPv6 addresses are not generally processed. This needs to be - // fixed. ### - if ( dns_A_reply && ! msg->skip_event ) + EventHandlerPtr event; + if ( msg->atype == TYPE_AAAA ) + event = dns_AAAA_reply; + else + event = dns_A6_reply; + if ( event && ! msg->skip_event ) { val_list* vl = new val_list; vl->append(analyzer->BuildConnVal()); vl->append(msg->BuildHdrVal()); vl->append(msg->BuildAnswerVal()); - vl->append(new AddrVal(htonl(addr[3]))); - - analyzer->ConnectionEvent(dns_A_reply, vl); - } - -#if 0 -alternative AAAA code from Chris - if ( dns_AAAA_reply && ! msg->skip_event ) - { - val_list* vl = new val_list; - - vl->append(analyzer->BuildConnVal()); - vl->append(msg->BuildHdrVal()); - vl->append(msg->BuildAnswerVal()); -#ifdef BROv6 - // FIXME: might need to htonl the addr first vl->append(new AddrVal(addr)); -#else - vl->append(new AddrVal((uint32)0x0000)); -#endif - char addrstr[INET6_ADDRSTRLEN]; - inet_ntop(AF_INET6, addr, addrstr, INET6_ADDRSTRLEN); - vl->append(new StringVal(addrstr)); - - analyzer->ConnectionEvent(dns_AAAA_reply, vl); + analyzer->ConnectionEvent(event, vl); } -#endif return 1; } diff --git a/src/DNS_Mgr.cc b/src/DNS_Mgr.cc index 2b9d07a969..9e65d3c9a9 100644 --- a/src/DNS_Mgr.cc +++ b/src/DNS_Mgr.cc @@ -46,13 +46,13 @@ extern int select(int, fd_set *, fd_set *, fd_set *, struct timeval *); class DNS_Mgr_Request { public: - DNS_Mgr_Request(const char* h) { host = copy_string(h); addr = 0; } - DNS_Mgr_Request(uint32 a) { addr = a; host = 0; } + DNS_Mgr_Request(const char* h, int af) { host = copy_string(h); fam = af; } + DNS_Mgr_Request(const IPAddr& a) { addr = a; host = 0; fam = 0; } ~DNS_Mgr_Request() { delete [] host; } // Returns nil if this was an address request. const char* ReqHost() const { return host; } - uint32 ReqAddr() const { return addr; } + const IPAddr& ReqAddr() const { return addr; } int MakeRequest(nb_dns_info* nb_dns); int RequestPending() const { return request_pending; } @@ -61,8 +61,8 @@ public: protected: char* host; // if non-nil, this is a host request - uint32 addr; - uint32 ttl; + int fam; // address family query type for host requests + IPAddr addr; int request_pending; }; @@ -75,15 +75,20 @@ int DNS_Mgr_Request::MakeRequest(nb_dns_info* nb_dns) char err[NB_DNS_ERRSIZE]; if ( host ) - return nb_dns_host_request(nb_dns, host, (void*) this, err) >= 0; + return nb_dns_host_request2(nb_dns, host, fam, (void*) this, err) >= 0; else - return nb_dns_addr_request(nb_dns, addr, (void*) this, err) >= 0; + { + const uint32* bytes; + int len = addr.GetBytes(&bytes); + return nb_dns_addr_request2(nb_dns, (char*) bytes, + len == 1 ? AF_INET : AF_INET6, (void*) this, err) >= 0; + } } class DNS_Mapping { public: DNS_Mapping(const char* host, struct hostent* h, uint32 ttl); - DNS_Mapping(uint32 addr, struct hostent* h, uint32 ttl); + DNS_Mapping(const IPAddr& addr, struct hostent* h, uint32 ttl); DNS_Mapping(FILE* f); int NoMapping() const { return no_mapping; } @@ -93,9 +98,11 @@ public: // Returns nil if this was an address request. const char* ReqHost() const { return req_host; } - uint32 ReqAddr() const { return req_addr; } - const char* ReqStr() const - { return req_host ? req_host : dotted_addr(ReqAddr()); } + IPAddr ReqAddr() const { return req_addr; } + string ReqStr() const + { + return req_host ? req_host : req_addr; + } ListVal* Addrs(); TableVal* AddrsSet(); // addresses returned as a set @@ -109,7 +116,14 @@ public: int Valid() const { return ! failed; } bool Expired() const - { return current_time() > (creation_time + req_ttl); } + { + if ( req_host && num_addrs == 0) + return false; // nothing to expire + + return current_time() > (creation_time + req_ttl); + } + + int Type() const { return map_type; } protected: friend class DNS_Mgr; @@ -121,7 +135,7 @@ protected: int init_failed; char* req_host; - uint32 req_addr; + IPAddr req_addr; uint32 req_ttl; int num_names; @@ -129,11 +143,12 @@ protected: StringVal* host_val; int num_addrs; - uint32* addrs; + IPAddr* addrs; ListVal* addrs_val; int failed; double creation_time; + int map_type; }; void DNS_Mgr_mapping_delete_func(void* v) @@ -154,14 +169,13 @@ DNS_Mapping::DNS_Mapping(const char* host, struct hostent* h, uint32 ttl) { Init(h); req_host = copy_string(host); - req_addr = 0; req_ttl = ttl; if ( names && ! names[0] ) names[0] = copy_string(host); } -DNS_Mapping::DNS_Mapping(uint32 addr, struct hostent* h, uint32 ttl) +DNS_Mapping::DNS_Mapping(const IPAddr& addr, struct hostent* h, uint32 ttl) { Init(h); req_addr = addr; @@ -175,7 +189,6 @@ DNS_Mapping::DNS_Mapping(FILE* f) init_failed = 1; req_host = 0; - req_addr = 0; char buf[512]; @@ -188,14 +201,15 @@ DNS_Mapping::DNS_Mapping(FILE* f) char req_buf[512+1], name_buf[512+1]; int is_req_host; - if ( sscanf(buf, "%lf %d %512s %d %512s %d", &creation_time, &is_req_host, - req_buf, &failed, name_buf, &num_addrs) != 6 ) + if ( sscanf(buf, "%lf %d %512s %d %512s %d %d %"PRIu32, &creation_time, + &is_req_host, req_buf, &failed, name_buf, &map_type, &num_addrs, + &req_ttl) != 8 ) return; if ( is_req_host ) req_host = copy_string(req_buf); else - req_addr = dotted_to_addr(req_buf); + req_addr = IPAddr(req_buf); num_names = 1; names = new char*[num_names]; @@ -203,7 +217,7 @@ DNS_Mapping::DNS_Mapping(FILE* f) if ( num_addrs > 0 ) { - addrs = new uint32[num_addrs]; + addrs = new IPAddr[num_addrs]; for ( int i = 0; i < num_addrs; ++i ) { @@ -217,7 +231,7 @@ DNS_Mapping::DNS_Mapping(FILE* f) if ( newline ) *newline = '\0'; - addrs[i] = dotted_to_addr(buf); + addrs[i] = IPAddr(buf); } } else @@ -280,14 +294,6 @@ StringVal* DNS_Mapping::Host() return host_val; } -// Converts an array of 4 bytes in network order to the corresponding -// 32-bit network long. -static uint32 raw_bytes_to_addr(const unsigned char b[4]) - { - uint32 l = (b[0] << 24) | (b[1] << 16) | (b[2] << 8) | b[3]; - return uint32(htonl(l)); - } - void DNS_Mapping::Init(struct hostent* h) { no_mapping = 0; @@ -296,12 +302,13 @@ void DNS_Mapping::Init(struct hostent* h) host_val = 0; addrs_val = 0; - if ( ! h || h->h_addrtype != AF_INET || h->h_length != 4 ) + if ( ! h ) { Clear(); return; } + map_type = h->h_addrtype; num_names = 1; // for now, just use official name names = new char*[num_names]; names[0] = h->h_name ? copy_string(h->h_name) : 0; @@ -311,10 +318,14 @@ void DNS_Mapping::Init(struct hostent* h) if ( num_addrs > 0 ) { - addrs = new uint32[num_addrs]; + addrs = new IPAddr[num_addrs]; for ( int i = 0; i < num_addrs; ++i ) - addrs[i] = raw_bytes_to_addr( - (unsigned char*)h->h_addr_list[i]); + if ( h->h_addrtype == AF_INET ) + addrs[i] = IPAddr(IPv4, (uint32*)h->h_addr_list[i], + IPAddr::Network); + else if ( h->h_addrtype == AF_INET6 ) + addrs[i] = IPAddr(IPv6, (uint32*)h->h_addr_list[i], + IPAddr::Network); } else addrs = 0; @@ -330,18 +341,19 @@ void DNS_Mapping::Clear() host_val = 0; addrs_val = 0; no_mapping = 0; + map_type = 0; failed = 1; } void DNS_Mapping::Save(FILE* f) const { - fprintf(f, "%.0f %d %s %d %s %d\n", creation_time, req_host != 0, - req_host ? req_host : dotted_addr(req_addr), + fprintf(f, "%.0f %d %s %d %s %d %d %"PRIu32"\n", creation_time, req_host != 0, + req_host ? req_host : req_addr.AsString().c_str(), failed, (names && names[0]) ? names[0] : "*", - num_addrs); + map_type, num_addrs, req_ttl); for ( int i = 0; i < num_addrs; ++i ) - fprintf(f, "%s\n", dotted_addr(addrs[i])); + fprintf(f, "%s\n", addrs[i].AsString().c_str()); } @@ -351,9 +363,6 @@ DNS_Mgr::DNS_Mgr(DNS_MgrMode arg_mode) mode = arg_mode; - host_mappings.SetDeleteFunc(DNS_Mgr_mapping_delete_func); - addr_mappings.SetDeleteFunc(DNS_Mgr_mapping_delete_func); - char err[NB_DNS_ERRSIZE]; nb_dns = nb_dns_init(err); @@ -440,24 +449,34 @@ TableVal* DNS_Mgr::LookupHost(const char* name) if ( mode != DNS_PRIME ) { - DNS_Mapping* d = host_mappings.Lookup(name); + HostMap::iterator it = host_mappings.find(name); - if ( d ) + if ( it != host_mappings.end() ) { - if ( d->Valid() ) - return d->Addrs()->ConvertToSet(); - else + DNS_Mapping* d4 = it->second.first; + DNS_Mapping* d6 = it->second.second; + + if ( (d4 && d4->Failed()) || (d6 && d6->Failed()) ) { reporter->Warning("no such host: %s", name); return empty_addr_set(); } + else if ( d4 && d6 ) + { + TableVal* tv4 = d4->AddrsSet(); + TableVal* tv6 = d6->AddrsSet(); + tv4->AddTo(tv6, false); + Unref(tv4); + return tv6; + } } } // Not found, or priming. switch ( mode ) { case DNS_PRIME: - requests.append(new DNS_Mgr_Request(name)); + requests.append(new DNS_Mgr_Request(name, AF_INET)); + requests.append(new DNS_Mgr_Request(name, AF_INET6)); return empty_addr_set(); case DNS_FORCE: @@ -465,7 +484,8 @@ TableVal* DNS_Mgr::LookupHost(const char* name) return 0; case DNS_DEFAULT: - requests.append(new DNS_Mgr_Request(name)); + requests.append(new DNS_Mgr_Request(name, AF_INET)); + requests.append(new DNS_Mgr_Request(name, AF_INET6)); Resolve(); return LookupHost(name); @@ -475,24 +495,25 @@ TableVal* DNS_Mgr::LookupHost(const char* name) } } -Val* DNS_Mgr::LookupAddr(uint32 addr) +Val* DNS_Mgr::LookupAddr(const IPAddr& addr) { if ( ! did_init ) Init(); if ( mode != DNS_PRIME ) { - HashKey h(&addr, 1); - DNS_Mapping* d = addr_mappings.Lookup(&h); + AddrMap::iterator it = addr_mappings.find(addr); - if ( d ) + if ( it != addr_mappings.end() ) { + DNS_Mapping* d = it->second; if ( d->Valid() ) return d->Host(); else { - reporter->Warning("can't resolve IP address: %s", dotted_addr(addr)); - return new StringVal(dotted_addr(addr)); + string s(addr); + reporter->Warning("can't resolve IP address: %s", s.c_str()); + return new StringVal(s.c_str()); } } } @@ -505,7 +526,7 @@ Val* DNS_Mgr::LookupAddr(uint32 addr) case DNS_FORCE: reporter->FatalError("can't find DNS entry for %s in cache", - dotted_addr(addr)); + addr.AsString().c_str()); return 0; case DNS_DEFAULT: @@ -681,28 +702,53 @@ void DNS_Mgr::AddResult(DNS_Mgr_Request* dr, struct nb_dns_result* r) if ( dr->ReqHost() ) { new_dm = new DNS_Mapping(dr->ReqHost(), h, ttl); - prev_dm = host_mappings.Insert(dr->ReqHost(), new_dm); + prev_dm = 0; + + HostMap::iterator it = host_mappings.find(dr->ReqHost()); + if ( it == host_mappings.end() ) + { + host_mappings[dr->ReqHost()].first = + new_dm->Type() == AF_INET ? new_dm : 0; + host_mappings[dr->ReqHost()].second = + new_dm->Type() == AF_INET ? 0 : new_dm; + } + + else + { + if ( new_dm->Type() == AF_INET ) + { + prev_dm = it->second.first; + it->second.first = new_dm; + } + else + { + prev_dm = it->second.second; + it->second.second = new_dm; + } + } if ( new_dm->Failed() && prev_dm && prev_dm->Valid() ) { // Put previous, valid entry back - CompareMappings // will generate a corresponding warning. - (void) host_mappings.Insert(dr->ReqHost(), prev_dm); + if ( prev_dm->Type() == AF_INET ) + host_mappings[dr->ReqHost()].first = prev_dm; + else + host_mappings[dr->ReqHost()].second = prev_dm; + ++keep_prev; } } else { new_dm = new DNS_Mapping(dr->ReqAddr(), h, ttl); - uint32 tmp_addr = dr->ReqAddr(); - HashKey k(&tmp_addr, 1); - prev_dm = addr_mappings.Insert(&k, new_dm); + AddrMap::iterator it = addr_mappings.find(dr->ReqAddr()); + prev_dm = (it == addr_mappings.end()) ? 0 : it->second; + addr_mappings[dr->ReqAddr()] = new_dm; if ( new_dm->Failed() && prev_dm && prev_dm->Valid() ) { - uint32 tmp_addr = dr->ReqAddr(); - HashKey k2(&tmp_addr, 1); - (void) addr_mappings.Insert(&k2, prev_dm); + addr_mappings[dr->ReqAddr()] = prev_dm; ++keep_prev; } } @@ -774,17 +820,13 @@ ListVal* DNS_Mgr::AddrListDelta(ListVal* al1, ListVal* al2) for ( int i = 0; i < al1->Length(); ++i ) { - addr_type al1_i = al1->Index(i)->AsAddr(); + const IPAddr& al1_i = al1->Index(i)->AsAddr(); int j; for ( j = 0; j < al2->Length(); ++j ) { - addr_type al2_j = al2->Index(j)->AsAddr(); -#ifdef BROv6 - if ( addr_eq(al1_i, al2_j) ) -#else + const IPAddr& al2_j = al2->Index(j)->AsAddr(); if ( al1_i == al2_j ) -#endif break; } @@ -800,8 +842,8 @@ void DNS_Mgr::DumpAddrList(FILE* f, ListVal* al) { for ( int i = 0; i < al->Length(); ++i ) { - addr_type al_i = al->Index(i)->AsAddr(); - fprintf(f, "%s%s", i > 0 ? "," : "", dotted_addr(al_i)); + const IPAddr& al_i = al->Index(i)->AsAddr(); + fprintf(f, "%s%s", i > 0 ? "," : "", al_i.AsString().c_str()); } } @@ -814,12 +856,20 @@ void DNS_Mgr::LoadCache(FILE* f) for ( ; ! m->NoMapping() && ! m->InitFailed(); m = new DNS_Mapping(f) ) { if ( m->ReqHost() ) - host_mappings.Insert(m->ReqHost(), m); + { + if ( host_mappings.find(m->ReqHost()) == host_mappings.end() ) + { + host_mappings[m->ReqHost()].first = 0; + host_mappings[m->ReqHost()].second = 0; + } + if ( m->Type() == AF_INET ) + host_mappings[m->ReqHost()].first = m; + else + host_mappings[m->ReqHost()].second = m; + } else { - uint32 tmp_addr = m->ReqAddr(); - HashKey h(&tmp_addr, 1); - addr_mappings.Insert(&h, m); + addr_mappings[m->ReqAddr()] = m; } } @@ -830,26 +880,41 @@ void DNS_Mgr::LoadCache(FILE* f) fclose(f); } -void DNS_Mgr::Save(FILE* f, PDict(DNS_Mapping)& m) +void DNS_Mgr::Save(FILE* f, const AddrMap& m) { - IterCookie* cookie = m.InitForIteration(); - DNS_Mapping* dm; - - while ( (dm = m.NextEntry(cookie)) ) - dm->Save(f); + for ( AddrMap::const_iterator it = m.begin(); it != m.end(); ++it ) + { + if ( it->second ) + it->second->Save(f); + } } -const char* DNS_Mgr::LookupAddrInCache(dns_mgr_addr_type addr) +void DNS_Mgr::Save(FILE* f, const HostMap& m) { - HashKey h(&addr, 1); - DNS_Mapping* d = dns_mgr->addr_mappings.Lookup(&h); + HostMap::const_iterator it; - if ( ! d ) + for ( it = m.begin(); it != m.end(); ++it ) + { + if ( it->second.first ) + it->second.first->Save(f); + + if ( it->second.second ) + it->second.second->Save(f); + } + } + +const char* DNS_Mgr::LookupAddrInCache(const IPAddr& addr) + { + AddrMap::iterator it = dns_mgr->addr_mappings.find(addr); + + if ( it == addr_mappings.end() ) return 0; + DNS_Mapping* d = it->second; + if ( d->Expired() ) { - dns_mgr->addr_mappings.Remove(&h); + dns_mgr->addr_mappings.erase(it); delete d; return 0; } @@ -861,23 +926,32 @@ const char* DNS_Mgr::LookupAddrInCache(dns_mgr_addr_type addr) TableVal* DNS_Mgr::LookupNameInCache(string name) { - DNS_Mapping* d = dns_mgr->host_mappings.Lookup(name.c_str()); - - if ( ! d || ! d->names ) + HostMap::iterator it = dns_mgr->host_mappings.find(name); + if ( it == dns_mgr->host_mappings.end() ) return 0; - if ( d->Expired() ) + DNS_Mapping* d4 = it->second.first; + DNS_Mapping* d6 = it->second.second; + + if ( ! d4 || ! d4->names || ! d6 || ! d6->names ) + return 0; + + if ( d4->Expired() || d6->Expired() ) { - HashKey h(name.c_str()); - dns_mgr->host_mappings.Remove(&h); - delete d; + dns_mgr->host_mappings.erase(it); + delete d4; + delete d6; return 0; } - return d->AddrsSet(); + TableVal* tv4 = d4->AddrsSet(); + TableVal* tv6 = d6->AddrsSet(); + tv4->AddTo(tv6, false); + Unref(tv4); + return tv6; } -void DNS_Mgr::AsyncLookupAddr(dns_mgr_addr_type host, LookupCallback* callback) +void DNS_Mgr::AsyncLookupAddr(const IPAddr& host, LookupCallback* callback) { if ( ! did_init ) Init(); @@ -956,10 +1030,15 @@ void DNS_Mgr::IssueAsyncRequests() ++num_requests; DNS_Mgr_Request* dr; + DNS_Mgr_Request* dr6 = 0; + if ( req->IsAddrReq() ) dr = new DNS_Mgr_Request(req->host); else - dr = new DNS_Mgr_Request(req->name.c_str()); + { + dr = new DNS_Mgr_Request(req->name.c_str(), AF_INET); + dr6 = new DNS_Mgr_Request(req->name.c_str(), AF_INET6); + } if ( ! dr->MakeRequest(nb_dns) ) { @@ -969,6 +1048,14 @@ void DNS_Mgr::IssueAsyncRequests() continue; } + if ( dr6 && ! dr6->MakeRequest(nb_dns) ) + { + reporter->Warning("can't issue DNS request"); + ++failed; + req->Timeout(); + continue; + } + req->time = current_time(); asyncs_timeouts.push(req); @@ -987,7 +1074,7 @@ double DNS_Mgr::NextTimestamp(double* network_time) return asyncs_timeouts.size() ? timer_mgr->Time() : -1.0; } -void DNS_Mgr::CheckAsyncAddrRequest(dns_mgr_addr_type addr, bool timeout) +void DNS_Mgr::CheckAsyncAddrRequest(const IPAddr& addr, bool timeout) { // Note that this code is a mirror of that for CheckAsyncHostRequest. @@ -1060,11 +1147,18 @@ void DNS_Mgr::Flush() { DoProcess(false); - IterCookie* cookie = addr_mappings.InitForIteration(); - DNS_Mapping* dm; + HostMap::iterator it; + for ( it = host_mappings.begin(); it != host_mappings.end(); ++it ) + { + delete it->second.first; + delete it->second.second; + } - host_mappings.Clear(); - addr_mappings.Clear(); + for ( AddrMap::iterator it2 = addr_mappings.begin(); it2 != addr_mappings.end(); ++it2 ) + delete it2->second; + + host_mappings.clear(); + addr_mappings.clear(); } void DNS_Mgr::Process() @@ -1107,6 +1201,14 @@ void DNS_Mgr::DoProcess(bool flush) else if ( status > 0 ) { DNS_Mgr_Request* dr = (DNS_Mgr_Request*) r.cookie; + + bool do_host_timeout = true; + if ( dr->ReqHost() && + host_mappings.find(dr->ReqHost()) == host_mappings.end() ) + // Don't timeout when this is the first result in an expected pair + // (one result each for A and AAAA queries). + do_host_timeout = false; + if ( dr->RequestPending() ) { AddResult(dr, &r); @@ -1116,7 +1218,7 @@ void DNS_Mgr::DoProcess(bool flush) if ( ! dr->ReqHost() ) CheckAsyncAddrRequest(dr->ReqAddr(), true); else - CheckAsyncHostRequest(dr->ReqHost(), true); + CheckAsyncHostRequest(dr->ReqHost(), do_host_timeout); IssueAsyncRequests(); @@ -1167,7 +1269,7 @@ void DNS_Mgr::GetStats(Stats* stats) stats->successful = successful; stats->failed = failed; stats->pending = asyncs_pending; - stats->cached_hosts = host_mappings.Length(); - stats->cached_addresses = addr_mappings.Length(); + stats->cached_hosts = host_mappings.size(); + stats->cached_addresses = addr_mappings.size(); } diff --git a/src/DNS_Mgr.h b/src/DNS_Mgr.h index def608d064..7983a91573 100644 --- a/src/DNS_Mgr.h +++ b/src/DNS_Mgr.h @@ -6,12 +6,14 @@ #include #include #include +#include #include "util.h" #include "BroList.h" #include "Dict.h" #include "EventHandler.h" #include "IOSource.h" +#include "IPAddr.h" class Val; class ListVal; @@ -27,7 +29,6 @@ struct nb_dns_result; declare(PDict,ListVal); class DNS_Mapping; -declare(PDict,DNS_Mapping); enum DNS_MgrMode { DNS_PRIME, // used to prime the cache @@ -39,10 +40,6 @@ enum DNS_MgrMode { // Number of seconds we'll wait for a reply. #define DNS_TIMEOUT 5 -// ### For now, we don't support IPv6 lookups. When we do, this -// should become addr_type. -typedef uint32 dns_mgr_addr_type; - class DNS_Mgr : public IOSource { public: DNS_Mgr(DNS_MgrMode mode); @@ -55,7 +52,7 @@ public: // a set of addr. TableVal* LookupHost(const char* host); - Val* LookupAddr(uint32 addr); + Val* LookupAddr(const IPAddr& addr); // Define the directory where to store the data. void SetDir(const char* arg_dir) { dir = copy_string(arg_dir); } @@ -64,7 +61,7 @@ public: void Resolve(); int Save(); - const char* LookupAddrInCache(dns_mgr_addr_type addr); + const char* LookupAddrInCache(const IPAddr& addr); TableVal* LookupNameInCache(string name); // Support for async lookups. @@ -78,7 +75,7 @@ public: virtual void Timeout() = 0; }; - void AsyncLookupAddr(dns_mgr_addr_type host, LookupCallback* callback); + void AsyncLookupAddr(const IPAddr& host, LookupCallback* callback); void AsyncLookupName(string name, LookupCallback* callback); struct Stats { @@ -107,8 +104,11 @@ protected: ListVal* AddrListDelta(ListVal* al1, ListVal* al2); void DumpAddrList(FILE* f, ListVal* al); + typedef map > HostMap; + typedef map AddrMap; void LoadCache(FILE* f); - void Save(FILE* f, PDict(DNS_Mapping)& m); + void Save(FILE* f, const AddrMap& m); + void Save(FILE* f, const HostMap& m); // Selects on the fd to see if there is an answer available (timeout // is secs). Returns 0 on timeout, -1 on EINTR or other error, and 1 @@ -120,7 +120,7 @@ protected: // Finish the request if we have a result. If not, time it out if // requested. - void CheckAsyncAddrRequest(dns_mgr_addr_type addr, bool timeout); + void CheckAsyncAddrRequest(const IPAddr& addr, bool timeout); void CheckAsyncHostRequest(const char* host, bool timeout); // Process outstanding requests. @@ -136,8 +136,8 @@ protected: PDict(ListVal) services; - PDict(DNS_Mapping) host_mappings; - PDict(DNS_Mapping) addr_mappings; + HostMap host_mappings; + AddrMap addr_mappings; DNS_mgr_request_list requests; @@ -163,7 +163,7 @@ protected: struct AsyncRequest { double time; - dns_mgr_addr_type host; + IPAddr host; string name; CallbackList callbacks; @@ -204,7 +204,7 @@ protected: }; - typedef map AsyncRequestAddrMap; + typedef map AsyncRequestAddrMap; AsyncRequestAddrMap asyncs_addrs; typedef map AsyncRequestNameMap; diff --git a/src/DPM.cc b/src/DPM.cc index 345741dfc8..d7e5cd25ef 100644 --- a/src/DPM.cc +++ b/src/DPM.cc @@ -11,53 +11,28 @@ #include "ConnSizeAnalyzer.h" -ExpectedConn::ExpectedConn(const uint32* _orig, const uint32* _resp, +ExpectedConn::ExpectedConn(const IPAddr& _orig, const IPAddr& _resp, uint16 _resp_p, uint16 _proto) { - if ( orig ) - copy_addr(_orig, orig); + if ( _orig == IPAddr(string("0.0.0.0")) ) + // don't use the IPv4 mapping, use the literal unspecified address + // to indicate a wildcard + orig = IPAddr(string("::")); else - { - for ( int i = 0; i < NUM_ADDR_WORDS; ++i ) - orig[i] = 0; - } - - copy_addr(_resp, resp); - - resp_p = _resp_p; - proto = _proto; - } - -ExpectedConn::ExpectedConn(uint32 _orig, uint32 _resp, - uint16 _resp_p, uint16 _proto) - { -#ifdef BROv6 - // Use the IPv4-within-IPv6 convention, as this is what's - // needed when we mix uint32's (like in this construction) - // with addr_type's (for example, when looking up expected - // connections). - - orig[0] = orig[1] = orig[2] = 0; - resp[0] = resp[1] = resp[2] = 0; - orig[3] = _orig; - resp[3] = _resp; -#else - orig[0] = _orig; - resp[0] = _resp; -#endif + orig = _orig; + resp = _resp; resp_p = _resp_p; proto = _proto; } ExpectedConn::ExpectedConn(const ExpectedConn& c) { - copy_addr(c.orig, orig); - copy_addr(c.resp, resp); + orig = c.orig; + resp = c.resp; resp_p = c.resp_p; proto = c.proto; } - DPM::DPM() : expected_conns_queue(AssignedAnalyzer::compare) { @@ -99,7 +74,7 @@ void DPM::PostScriptInit() void DPM::AddConfig(const Analyzer::Config& cfg) { -#ifdef USE_PERFTOOLS +#ifdef USE_PERFTOOLS_DEBUG HeapLeakChecker::Disabler disabler; #endif @@ -158,23 +133,18 @@ AnalyzerTag::Tag DPM::GetExpected(int proto, const Connection* conn) ExpectedConn c(conn->OrigAddr(), conn->RespAddr(), ntohs(conn->RespPort()), proto); - // Can't use sizeof(c) due to potential alignment issues. - // FIXME: I guess this is still not portable ... - HashKey key(&c, sizeof(c.orig) + sizeof(c.resp) + - sizeof(c.resp_p) + sizeof(c.proto)); - - AssignedAnalyzer* a = expected_conns.Lookup(&key); + HashKey* key = BuildExpectedConnHashKey(c); + AssignedAnalyzer* a = expected_conns.Lookup(key); + delete key; if ( ! a ) { // Wildcard for originator. - for ( int i = 0; i < NUM_ADDR_WORDS; ++i ) - c.orig[i] = 0; + c.orig = IPAddr(string("::")); - HashKey key(&c, sizeof(c.orig) + sizeof(c.resp) + - sizeof(c.resp_p) + sizeof(c.proto)); - - a = expected_conns.Lookup(&key); + HashKey* key = BuildExpectedConnHashKey(c); + a = expected_conns.Lookup(key); + delete key; } if ( ! a ) @@ -215,46 +185,8 @@ bool DPM::BuildInitialAnalyzerTree(TransportProto proto, Connection* conn, break; case TRANSPORT_ICMP: { - const struct icmp* icmpp = (const struct icmp *) data; - switch ( icmpp->icmp_type ) { - - case ICMP_ECHO: - case ICMP_ECHOREPLY: - if ( ICMP_Echo_Analyzer::Available() ) - { - root = icmp = new ICMP_Echo_Analyzer(conn); - DBG_DPD(conn, "activated ICMP Echo analyzer"); - } - break; - - case ICMP_REDIRECT: - if ( ICMP_Redir_Analyzer::Available() ) - { - root = new ICMP_Redir_Analyzer(conn); - DBG_DPD(conn, "activated ICMP Redir analyzer"); - } - break; - - case ICMP_UNREACH: - if ( ICMP_Unreachable_Analyzer::Available() ) - { - root = icmp = new ICMP_Unreachable_Analyzer(conn); - DBG_DPD(conn, "activated ICMP Unreachable analyzer"); - } - break; - - case ICMP_TIMXCEED: - if ( ICMP_TimeExceeded_Analyzer::Available() ) - { - root = icmp = new ICMP_TimeExceeded_Analyzer(conn); - DBG_DPD(conn, "activated ICMP Time Exceeded analyzer"); - } - break; - } - - if ( ! root ) - root = icmp = new ICMP_Analyzer(conn); - + root = icmp = new ICMP_Analyzer(conn); + DBG_DPD(conn, "activated ICMP analyzer"); analyzed = true; break; } @@ -404,7 +336,8 @@ bool DPM::BuildInitialAnalyzerTree(TransportProto proto, Connection* conn, return true; } -void DPM::ExpectConnection(addr_type orig, addr_type resp, uint16 resp_p, +void DPM::ExpectConnection(const IPAddr& orig, const IPAddr& resp, + uint16 resp_p, TransportProto proto, AnalyzerTag::Tag analyzer, double timeout, void* cookie) { @@ -416,11 +349,7 @@ void DPM::ExpectConnection(addr_type orig, addr_type resp, uint16 resp_p, { if ( ! a->deleted ) { - HashKey* key = new HashKey(&a->conn, - sizeof(a->conn.orig) + - sizeof(a->conn.resp) + - sizeof(a->conn.resp_p) + - sizeof(a->conn.proto)); + HashKey* key = BuildExpectedConnHashKey(a->conn); expected_conns.Remove(key); delete key; } @@ -439,10 +368,9 @@ void DPM::ExpectConnection(addr_type orig, addr_type resp, uint16 resp_p, ExpectedConn c(orig, resp, resp_p, proto); - HashKey key(&c, sizeof(c.orig) + sizeof(c.resp) + - sizeof(c.resp_p) + sizeof(c.proto)); + HashKey* key = BuildExpectedConnHashKey(c); - AssignedAnalyzer* a = expected_conns.Lookup(&key); + AssignedAnalyzer* a = expected_conns.Lookup(key); if ( a ) a->deleted = true; @@ -454,8 +382,9 @@ void DPM::ExpectConnection(addr_type orig, addr_type resp, uint16 resp_p, a->timeout = network_time + timeout; a->deleted = false; - expected_conns.Insert(&key, a); + expected_conns.Insert(key, a); expected_conns_queue.push(a); + delete key; } void DPM::Done() @@ -466,11 +395,7 @@ void DPM::Done() AssignedAnalyzer* a = expected_conns_queue.top(); if ( ! a->deleted ) { - HashKey* key = new HashKey(&a->conn, - sizeof(a->conn.orig) + - sizeof(a->conn.resp) + - sizeof(a->conn.resp_p) + - sizeof(a->conn.proto)); + HashKey* key = BuildExpectedConnHashKey(a->conn); expected_conns.Remove(key); delete key; } diff --git a/src/DPM.h b/src/DPM.h index 4bacbfbeea..f59d21dbfc 100644 --- a/src/DPM.h +++ b/src/DPM.h @@ -27,19 +27,13 @@ // Map to assign expected connections to analyzers. class ExpectedConn { public: - // This form can be used for IPv6 as well as IPv4. - ExpectedConn(const uint32* _orig, const uint32* _resp, + ExpectedConn(const IPAddr& _orig, const IPAddr& _resp, uint16 _resp_p, uint16 _proto); - // This form only works for expecting an IPv4 connection. Note - // that we do the right thing whether we're built IPv4-only or - // BROv6. - ExpectedConn(uint32 _orig, uint32 _resp, uint16 _resp_p, uint16 _proto); - ExpectedConn(const ExpectedConn& c); - uint32 orig[NUM_ADDR_WORDS]; - uint32 resp[NUM_ADDR_WORDS]; + IPAddr orig; + IPAddr resp; uint16 resp_p; uint16 proto; }; @@ -90,7 +84,7 @@ public: // Schedules a particular analyzer for an upcoming connection. // 0 acts as a wildcard for orig. (Cookie is currently unused. // Eventually, we may pass it on to the analyzer). - void ExpectConnection(addr_type orig, addr_type resp, uint16 resp_p, + void ExpectConnection(const IPAddr& orig, const IPAddr& resp, uint16 resp_p, TransportProto proto, AnalyzerTag::Tag analyzer, double timeout, void* cookie); diff --git a/src/Debug.cc b/src/Debug.cc index 1b849fff7e..ea9c52f77e 100644 --- a/src/Debug.cc +++ b/src/Debug.cc @@ -142,7 +142,7 @@ int TraceState::LogTrace(const char* fmt, ...) if ( ! loc.filename ) { - loc.filename = ""; + loc.filename = copy_string(""); loc.last_line = 0; } @@ -735,7 +735,7 @@ string get_context_description(const Stmt* stmt, const Frame* frame) loc = *stmt->GetLocationInfo(); else { - loc.filename = ""; + loc.filename = copy_string(""); loc.last_line = 0; } diff --git a/src/DebugLogger.cc b/src/DebugLogger.cc index d60fdd70c8..c41a0552c6 100644 --- a/src/DebugLogger.cc +++ b/src/DebugLogger.cc @@ -15,7 +15,7 @@ DebugLogger::Stream DebugLogger::streams[NUM_DBGS] = { { "compressor", 0, false }, {"string", 0, false }, { "notifiers", 0, false }, { "main-loop", 0, false }, { "dpd", 0, false }, { "tm", 0, false }, - { "logging", 0, false } + { "logging", 0, false }, { "threading", 0, false } }; DebugLogger::DebugLogger(const char* filename) diff --git a/src/DebugLogger.h b/src/DebugLogger.h index a2dece5b3c..71e21bfa26 100644 --- a/src/DebugLogger.h +++ b/src/DebugLogger.h @@ -24,6 +24,7 @@ enum DebugStream { DBG_DPD, // Dynamic application detection framework DBG_TM, // Time-machine packet input via Brocolli DBG_LOGGING, // Logging streams + DBG_THREADING, // Threading system NUM_DBGS // Has to be last }; diff --git a/src/Desc.cc b/src/Desc.cc index 12b4a524eb..9d94321427 100644 --- a/src/Desc.cc +++ b/src/Desc.cc @@ -157,6 +157,16 @@ void ODesc::Add(double d) } } +void ODesc::Add(const IPAddr& addr) + { + Add(addr.AsString()); + } + +void ODesc::Add(const IPPrefix& prefix) + { + Add(prefix.AsString()); + } + void ODesc::AddCS(const char* s) { int n = strlen(s); diff --git a/src/Desc.h b/src/Desc.h index 27cbd4fa01..9c60c68106 100644 --- a/src/Desc.h +++ b/src/Desc.h @@ -6,6 +6,7 @@ #include #include #include + #include "BroString.h" typedef enum { @@ -21,6 +22,8 @@ typedef enum { } desc_style; class BroFile; +class IPAddr; +class IPPrefix; class ODesc { public: @@ -68,11 +71,14 @@ public: void Add(const char* s, int do_indent=1); void AddN(const char* s, int len) { AddBytes(s, len); } + void Add(const string& s) { AddBytes(s.data(), s.size()); } void Add(int i); void Add(uint32 u); void Add(int64 i); void Add(uint64 u); void Add(double d); + void Add(const IPAddr& addr); + void Add(const IPPrefix& prefix); // Add s as a counted string. void AddCS(const char* s); diff --git a/src/Discard.cc b/src/Discard.cc index a71b810601..edfeea1408 100644 --- a/src/Discard.cc +++ b/src/Discard.cc @@ -10,11 +10,6 @@ Discarder::Discarder() { - ip_hdr = internal_type("ip_hdr")->AsRecordType(); - tcp_hdr = internal_type("tcp_hdr")->AsRecordType(); - udp_hdr = internal_type("udp_hdr")->AsRecordType(); - icmp_hdr = internal_type("icmp_hdr")->AsRecordType(); - check_ip = internal_func("discarder_check_ip"); check_tcp = internal_func("discarder_check_tcp"); check_udp = internal_func("discarder_check_udp"); @@ -36,12 +31,10 @@ int Discarder::NextPacket(const IP_Hdr* ip, int len, int caplen) { int discard_packet = 0; - const struct ip* ip4 = ip->IP4_Hdr(); - if ( check_ip ) { val_list* args = new val_list; - args->append(BuildHeader(ip4)); + args->append(ip->BuildPktHdrVal()); try { @@ -59,19 +52,18 @@ int Discarder::NextPacket(const IP_Hdr* ip, int len, int caplen) return discard_packet; } - int proto = ip4->ip_p; + int proto = ip->NextProto(); if ( proto != IPPROTO_TCP && proto != IPPROTO_UDP && proto != IPPROTO_ICMP ) // This is not a protocol we understand. return 0; // XXX shall we only check the first packet??? - uint32 frag_field = ntohs(ip4->ip_off); - if ( (frag_field & 0x3fff) != 0 ) + if ( ip->IsFragment() ) // Never check any fragment. return 0; - int ip_hdr_len = ip4->ip_hl * 4; + int ip_hdr_len = ip->HdrLen(); len -= ip_hdr_len; // remove IP header caplen -= ip_hdr_len; @@ -87,7 +79,7 @@ int Discarder::NextPacket(const IP_Hdr* ip, int len, int caplen) // Where the data starts - if this is a protocol we know about, // this gets advanced past the transport header. - const u_char* data = ((u_char*) ip4 + ip_hdr_len); + const u_char* data = ip->Payload(); if ( is_tcp ) { @@ -97,8 +89,7 @@ int Discarder::NextPacket(const IP_Hdr* ip, int len, int caplen) int th_len = tp->th_off * 4; val_list* args = new val_list; - args->append(BuildHeader(ip4)); - args->append(BuildHeader(tp, len)); + args->append(ip->BuildPktHdrVal()); args->append(BuildData(data, th_len, len, caplen)); try @@ -123,8 +114,7 @@ int Discarder::NextPacket(const IP_Hdr* ip, int len, int caplen) int uh_len = sizeof (struct udphdr); val_list* args = new val_list; - args->append(BuildHeader(ip4)); - args->append(BuildHeader(up)); + args->append(ip->BuildPktHdrVal()); args->append(BuildData(data, uh_len, len, caplen)); try @@ -148,8 +138,7 @@ int Discarder::NextPacket(const IP_Hdr* ip, int len, int caplen) const struct icmp* ih = (const struct icmp*) data; val_list* args = new val_list; - args->append(BuildHeader(ip4)); - args->append(BuildHeader(ih)); + args->append(ip->BuildPktHdrVal()); try { @@ -168,62 +157,6 @@ int Discarder::NextPacket(const IP_Hdr* ip, int len, int caplen) return discard_packet; } -Val* Discarder::BuildHeader(const struct ip* ip) - { - RecordVal* hdr = new RecordVal(ip_hdr); - - hdr->Assign(0, new Val(ip->ip_hl * 4, TYPE_COUNT)); - hdr->Assign(1, new Val(ip->ip_tos, TYPE_COUNT)); - hdr->Assign(2, new Val(ntohs(ip->ip_len), TYPE_COUNT)); - hdr->Assign(3, new Val(ntohs(ip->ip_id), TYPE_COUNT)); - hdr->Assign(4, new Val(ip->ip_ttl, TYPE_COUNT)); - hdr->Assign(5, new Val(ip->ip_p, TYPE_COUNT)); - hdr->Assign(6, new AddrVal(ip->ip_src.s_addr)); - hdr->Assign(7, new AddrVal(ip->ip_dst.s_addr)); - - return hdr; - } - -Val* Discarder::BuildHeader(const struct tcphdr* tp, int tcp_len) - { - RecordVal* hdr = new RecordVal(tcp_hdr); - - hdr->Assign(0, new PortVal(ntohs(tp->th_sport), TRANSPORT_TCP)); - hdr->Assign(1, new PortVal(ntohs(tp->th_dport), TRANSPORT_TCP)); - hdr->Assign(2, new Val(uint32(ntohl(tp->th_seq)), TYPE_COUNT)); - hdr->Assign(3, new Val(uint32(ntohl(tp->th_ack)), TYPE_COUNT)); - - int tcp_hdr_len = tp->th_off * 4; - - hdr->Assign(4, new Val(tcp_hdr_len, TYPE_COUNT)); - hdr->Assign(5, new Val(tcp_len - tcp_hdr_len, TYPE_COUNT)); - - hdr->Assign(6, new Val(tp->th_flags, TYPE_COUNT)); - hdr->Assign(7, new Val(ntohs(tp->th_win), TYPE_COUNT)); - - return hdr; - } - -Val* Discarder::BuildHeader(const struct udphdr* up) - { - RecordVal* hdr = new RecordVal(udp_hdr); - - hdr->Assign(0, new PortVal(ntohs(up->uh_sport), TRANSPORT_UDP)); - hdr->Assign(1, new PortVal(ntohs(up->uh_dport), TRANSPORT_UDP)); - hdr->Assign(2, new Val(ntohs(up->uh_ulen), TYPE_COUNT)); - - return hdr; - } - -Val* Discarder::BuildHeader(const struct icmp* icmp) - { - RecordVal* hdr = new RecordVal(icmp_hdr); - - hdr->Assign(0, new Val(icmp->icmp_type, TYPE_COUNT)); - - return hdr; - } - Val* Discarder::BuildData(const u_char* data, int hdrlen, int len, int caplen) { len -= hdrlen; diff --git a/src/Discard.h b/src/Discard.h index 16f7a58e6e..f4daabefa7 100644 --- a/src/Discard.h +++ b/src/Discard.h @@ -25,17 +25,8 @@ public: int NextPacket(const IP_Hdr* ip, int len, int caplen); protected: - Val* BuildHeader(const struct ip* ip); - Val* BuildHeader(const struct tcphdr* tp, int tcp_len); - Val* BuildHeader(const struct udphdr* up); - Val* BuildHeader(const struct icmp* icmp); Val* BuildData(const u_char* data, int hdrlen, int len, int caplen); - RecordType* ip_hdr; - RecordType* tcp_hdr; - RecordType* udp_hdr; - RecordType* icmp_hdr; - Func* check_ip; Func* check_tcp; Func* check_udp; diff --git a/src/EventHandler.h b/src/EventHandler.h index 2aebe87584..a86b8a285c 100644 --- a/src/EventHandler.h +++ b/src/EventHandler.h @@ -7,7 +7,6 @@ #include "List.h" #include "BroList.h" -#include "net_util.h" class Func; class FuncType; diff --git a/src/Expr.cc b/src/Expr.cc index f9669f86fc..58f5db3fd1 100644 --- a/src/Expr.cc +++ b/src/Expr.cc @@ -14,6 +14,7 @@ #include "Net.h" #include "Traverse.h" #include "Trigger.h" +#include "IPAddr.h" const char* expr_name(BroExprTag t) { @@ -834,30 +835,30 @@ Val* BinaryExpr::StringFold(Val* v1, Val* v2) const Val* BinaryExpr::AddrFold(Val* v1, Val* v2) const { - addr_type a1 = v1->AsAddr(); - addr_type a2 = v2->AsAddr(); + IPAddr a1 = v1->AsAddr(); + IPAddr a2 = v2->AsAddr(); int result = 0; switch ( tag ) { -#undef DO_FOLD -#ifdef BROv6 -#define DO_FOLD(sense) { result = memcmp(a1, a2, 16) sense 0; break; } -#else -#define DO_FOLD(sense) \ - { \ - a1 = ntohl(a1); \ - a2 = ntohl(a2); \ - result = (a1 < a2 ? -1 : (a1 == a2 ? 0 : 1)) sense 0; \ - break; \ - } -#endif - case EXPR_LT: DO_FOLD(<) - case EXPR_LE: DO_FOLD(<=) - case EXPR_EQ: DO_FOLD(==) - case EXPR_NE: DO_FOLD(!=) - case EXPR_GE: DO_FOLD(>=) - case EXPR_GT: DO_FOLD(>) + case EXPR_LT: + result = a1 < a2; + break; + case EXPR_LE: + result = a1 < a2 || a1 == a2; + break; + case EXPR_EQ: + result = a1 == a2; + break; + case EXPR_NE: + result = a1 != a2; + break; + case EXPR_GE: + result = ! ( a1 < a2 ); + break; + case EXPR_GT: + result = ( ! ( a1 < a2 ) ) && ( a1 != a2 ); + break; default: BadTag("BinaryExpr::AddrFold", expr_name(tag)); @@ -868,20 +869,13 @@ Val* BinaryExpr::AddrFold(Val* v1, Val* v2) const Val* BinaryExpr::SubNetFold(Val* v1, Val* v2) const { - subnet_type* n1 = v1->AsSubNet(); - subnet_type* n2 = v2->AsSubNet(); + const IPPrefix& n1 = v1->AsSubNet(); + const IPPrefix& n2 = v2->AsSubNet(); - if ( n1->width != n2->width ) + if ( n1 == n2 ) + return new Val(1, TYPE_BOOL); + else return new Val(0, TYPE_BOOL); - -#ifdef BROv6 - if ( memcmp(n1->net, n2->net, 16) ) -#else - if ( n1->net != n2->net ) -#endif - return new Val(0, TYPE_BOOL); - - return new Val(1, TYPE_BOOL); } void BinaryExpr::SwapOps() @@ -1681,15 +1675,13 @@ DivideExpr::DivideExpr(Expr* arg_op1, Expr* arg_op2) Val* DivideExpr::AddrFold(Val* v1, Val* v2) const { - addr_type a1 = v1->AsAddr(); - uint32 mask; if ( v2->Type()->Tag() == TYPE_COUNT ) mask = static_cast(v2->InternalUnsigned()); else mask = static_cast(v2->InternalInt()); - return new SubNetVal(a1, mask); + return new SubNetVal(v1->AsAddr(), mask); } Expr* DivideExpr::DoSimplify() @@ -2672,8 +2664,6 @@ void AssignExpr::EvalIntoAggregate(const BroType* t, Val* aggr, Frame* f) const Error("bad table insertion"); TableVal* tv = aggr->AsTableVal(); - const TableType* tt = tv->Type()->AsTableType(); - const BroType* yt = tv->Type()->YieldType(); Val* index = op1->Eval(f); Val* v = op2->Eval(f); @@ -3643,151 +3633,6 @@ bool FieldAssignExpr::DoUnserialize(UnserialInfo* info) return true; } -RecordMatchExpr::RecordMatchExpr(Expr* op1 /* record to match */, - Expr* op2 /* cases to match against */) -: BinaryExpr(EXPR_MATCH, op1, op2) - { - BroType* result_type = 0; - - // Make sure the second argument is of a suitable type. - if ( ! op2->Type()->IsSet() ) - { - ExprError("matching must be done against a set of match records"); - return; - } - - type_list* elt_types = op2->Type()->AsSetType()->Indices()->Types(); - - if ( ! elt_types->length() || - (*elt_types)[0]->Tag() != TYPE_RECORD ) - { - ExprError("matching must be done against a set of match records"); - return; - } - - RecordType* case_rec_type = (*elt_types)[0]->AsRecordType(); - - // NOTE: The "result" and "pred" field names are hardcoded here. - result_field_index = case_rec_type->FieldOffset("result"); - - if ( result_field_index < 0 ) - { - ExprError("match records must have a $result field"); - return; - } - - result_type = case_rec_type->FieldType("result")->Ref(); - - // Check that pred exists, and that the first argument matches it. - if ( (pred_field_index = case_rec_type->FieldOffset("pred")) < 0 || - case_rec_type->FieldType("pred")->Tag() != TYPE_FUNC ) - { - ExprError("match records must have a $pred' field of function type"); - return; - } - - FuncType* pred_type = case_rec_type->FieldType("pred")->AsFuncType(); - type_list* pred_arg_types = pred_type->ArgTypes()->Types(); - if ( pred_arg_types->length() != 1 || - ! check_and_promote_expr(op1, (*pred_arg_types)[0]) ) - ExprError("record to match does not have the same type as predicate argument"); - - // NOTE: The "priority" field name is hardcoded here. - if ( (priority_field_index = case_rec_type->FieldOffset("priority")) >= 0 && - ! IsArithmetic(case_rec_type->FieldType("priority")->Tag()) ) - ExprError("$priority field must have a numeric type"); - - SetType(result_type); - } - -void RecordMatchExpr::ExprDescribe(ODesc* d) const - { - if ( d->IsReadable() ) - { - d->Add("match "); - op1->Describe(d); - d->Add(" using "); - op2->Describe(d); - } - } - -Val* RecordMatchExpr::Fold(Val* v1, Val* v2) const - { - TableVal* match_set = v2->AsTableVal(); - if ( ! match_set ) - Internal("non-table in RecordMatchExpr"); - - Val* return_val = 0; - double highest_priority = -1e100; - - ListVal* match_recs = match_set->ConvertToList(TYPE_ANY); - for ( int i = 0; i < match_recs->Length(); ++i ) - { - val_list args(1); - args.append(v1->Ref()); - - double this_priority = 0; - - // ### Get rid of the double Index if TYPE_ANY->TYPE_RECORD. - Val* v = match_recs->Index(i)->AsListVal()->Index(0); - - const RecordVal* match_rec = v->AsRecordVal(); - if ( ! match_rec ) - Internal("Element of match set is not a record"); - - if ( priority_field_index >= 0 ) - { - this_priority = - match_rec->Lookup(priority_field_index)->CoerceToDouble(); - if ( this_priority <= highest_priority ) - { - Unref(v1); - continue; - } - } - - // No try/catch here; we pass exceptions upstream. - Val* pred_val = - match_rec->Lookup(pred_field_index)->AsFunc()->Call(&args); - bool is_zero = pred_val->IsZero(); - Unref(pred_val); - - if ( ! is_zero ) - { - Val* new_return_val = - match_rec->Lookup(result_field_index); - - Unref(return_val); - return_val = new_return_val->Ref(); - - if ( priority_field_index >= 0 ) - highest_priority = this_priority; - else - break; - } - } - - Unref(match_recs); - - return return_val; - } - -IMPLEMENT_SERIAL(RecordMatchExpr, SER_RECORD_MATCH_EXPR); - -bool RecordMatchExpr::DoSerialize(SerialInfo* info) const - { - DO_SERIALIZE(SER_RECORD_MATCH_EXPR, BinaryExpr); - return SERIALIZE(pred_field_index) && SERIALIZE(result_field_index) && - SERIALIZE(priority_field_index); - } - -bool RecordMatchExpr::DoUnserialize(UnserialInfo* info) - { - DO_UNSERIALIZE(BinaryExpr); - return UNSERIALIZE(&pred_field_index) && UNSERIALIZE(&result_field_index) && - UNSERIALIZE(&priority_field_index); - } - ArithCoerceExpr::ArithCoerceExpr(Expr* arg_op, TypeTag t) : UnaryExpr(EXPR_ARITH_COERCE, arg_op) { @@ -4918,6 +4763,7 @@ Val* ListExpr::Eval(Frame* f) const if ( ! ev ) { Error("uninitialized list value"); + Unref(v); return 0; } diff --git a/src/Expr.h b/src/Expr.h index 8676a1ad7e..f0798359c2 100644 --- a/src/Expr.h +++ b/src/Expr.h @@ -823,32 +823,6 @@ protected: string field_name; }; -class RecordMatchExpr : public BinaryExpr { -public: - RecordMatchExpr(Expr* op1 /* record to match */, - Expr* op2 /* cases to match against */); - -protected: - friend class Expr; - RecordMatchExpr() - { - pred_field_index = result_field_index = - priority_field_index = 0; - } - - virtual Val* Fold(Val* v1, Val* v2) const; - void ExprDescribe(ODesc*) const; - - DECLARE_SERIAL(RecordMatchExpr); - - // The following are used to hold the field offset of - // $pred, $result, $priority, so the names only need to - // be looked up at compile-time. - int pred_field_index; - int result_field_index; - int priority_field_index; -}; - class ArithCoerceExpr : public UnaryExpr { public: ArithCoerceExpr(Expr* op, TypeTag t); diff --git a/src/File.cc b/src/File.cc index 080923ad37..d4e31bcc16 100644 --- a/src/File.cc +++ b/src/File.cc @@ -232,7 +232,7 @@ BroFile::~BroFile() delete [] access; delete [] cipher_buffer; -#ifdef USE_PERFTOOLS +#ifdef USE_PERFTOOLS_DEBUG heap_checker->UnIgnoreObject(this); #endif } @@ -255,7 +255,7 @@ void BroFile::Init() cipher_ctx = 0; cipher_buffer = 0; -#ifdef USE_PERFTOOLS +#ifdef USE_PERFTOOLS_DEBUG heap_checker->IgnoreObject(this); #endif } diff --git a/src/Frag.cc b/src/Frag.cc index b72fac4b16..d873f5bc0c 100644 --- a/src/Frag.cc +++ b/src/Frag.cc @@ -27,21 +27,30 @@ void FragTimer::Dispatch(double t, int /* is_expire */) FragReassembler::FragReassembler(NetSessions* arg_s, const IP_Hdr* ip, const u_char* pkt, - uint32 frag_field, HashKey* k, double t) -: Reassembler(0, ip->DstAddr(), REASSEM_IP) + HashKey* k, double t) + : Reassembler(0, REASSEM_IP) { s = arg_s; key = k; + const struct ip* ip4 = ip->IP4_Hdr(); - proto_hdr_len = ip4->ip_hl * 4; - proto_hdr = (struct ip*) new u_char[64]; // max IP header + slop - // Don't do a structure copy - need to pick up options, too. - memcpy((void*) proto_hdr, (const void*) ip4, proto_hdr_len); + if ( ip4 ) + { + proto_hdr_len = ip->HdrLen(); + proto_hdr = new u_char[64]; // max IP header + slop + // Don't do a structure copy - need to pick up options, too. + memcpy((void*) proto_hdr, (const void*) ip4, proto_hdr_len); + } + else + { + proto_hdr_len = ip->HdrLen() - 8; // minus length of fragment header + proto_hdr = new u_char[proto_hdr_len]; + memcpy(proto_hdr, ip->IP6_Hdr(), proto_hdr_len); + } reassembled_pkt = 0; frag_size = 0; // flag meaning "not known" - - AddFragment(t, ip, pkt, frag_field); + next_proto = ip->NextProto(); if ( frag_timeout != 0.0 ) { @@ -50,6 +59,8 @@ FragReassembler::FragReassembler(NetSessions* arg_s, } else expire_timer = 0; + + AddFragment(t, ip, pkt); } FragReassembler::~FragReassembler() @@ -60,28 +71,42 @@ FragReassembler::~FragReassembler() delete key; } -void FragReassembler::AddFragment(double t, const IP_Hdr* ip, const u_char* pkt, - uint32 frag_field) +void FragReassembler::AddFragment(double t, const IP_Hdr* ip, const u_char* pkt) { const struct ip* ip4 = ip->IP4_Hdr(); - if ( ip4->ip_p != proto_hdr->ip_p || ip4->ip_hl != proto_hdr->ip_hl ) + if ( ip4 ) + { + if ( ip4->ip_p != ((const struct ip*)proto_hdr)->ip_p || + ip4->ip_hl != ((const struct ip*)proto_hdr)->ip_hl ) // || ip4->ip_tos != proto_hdr->ip_tos // don't check TOS, there's at least one stack that actually // uses different values, and it's hard to see an associated // attack. s->Weird("fragment_protocol_inconsistency", ip); + } + else + { + if ( ip->NextProto() != next_proto || + ip->HdrLen() - 8 != proto_hdr_len ) + s->Weird("fragment_protocol_inconsistency", ip); + // TODO: more detailed unfrag header consistency checks? + } - if ( frag_field & 0x4000 ) + if ( ip->DF() ) // Linux MTU discovery for UDP can do this, for example. s->Weird("fragment_with_DF", ip); - int offset = (ntohs(ip4->ip_off) & 0x1fff) * 8; - int len = ntohs(ip4->ip_len); - int hdr_len = proto_hdr->ip_hl * 4; + int offset = ip->FragOffset(); + int len = ip->TotalLen(); + int hdr_len = ip->HdrLen(); int upper_seq = offset + len - hdr_len; - if ( (frag_field & 0x2000) == 0 ) + if ( ! offset ) + // Make sure to use the first fragment header's next field. + next_proto = ip->NextProto(); + + if ( ! ip->MF() ) { // Last fragment. if ( frag_size == 0 ) @@ -125,7 +150,7 @@ void FragReassembler::AddFragment(double t, const IP_Hdr* ip, const u_char* pkt, void FragReassembler::Overlap(const u_char* b1, const u_char* b2, int n) { - IP_Hdr proto_h((const struct ip*) proto_hdr); + IP_Hdr proto_h(proto_hdr, false, proto_hdr_len); if ( memcmp((const void*) b1, (const void*) b2, n) ) s->Weird("fragment_inconsistency", &proto_h); @@ -157,7 +182,7 @@ void FragReassembler::BlockInserted(DataBlock* /* start_block */) // can happen for benign reasons when we're // intermingling parts of two fragmented packets. - IP_Hdr proto_h((const struct ip*) proto_hdr); + IP_Hdr proto_h(proto_hdr, false, proto_hdr_len); s->Weird("fragment_size_inconsistency", &proto_h); // We decide to analyze the contiguous portion now. @@ -171,7 +196,7 @@ void FragReassembler::BlockInserted(DataBlock* /* start_block */) else if ( last_block->upper > frag_size ) { - IP_Hdr proto_h((const struct ip*) proto_hdr); + IP_Hdr proto_h(proto_hdr, false, proto_hdr_len); s->Weird("fragment_size_inconsistency", &proto_h); frag_size = last_block->upper; } @@ -193,8 +218,7 @@ void FragReassembler::BlockInserted(DataBlock* /* start_block */) u_char* pkt = new u_char[n]; memcpy((void*) pkt, (const void*) proto_hdr, proto_hdr_len); - struct ip* reassem4 = (struct ip*) pkt; - reassem4->ip_len = htons(frag_size + proto_hdr_len); + u_char* pkt_start = pkt; pkt += proto_hdr_len; @@ -214,7 +238,27 @@ void FragReassembler::BlockInserted(DataBlock* /* start_block */) } delete reassembled_pkt; - reassembled_pkt = new IP_Hdr(reassem4); + + if ( ((const struct ip*)pkt_start)->ip_v == 4 ) + { + struct ip* reassem4 = (struct ip*) pkt_start; + reassem4->ip_len = htons(frag_size + proto_hdr_len); + reassembled_pkt = new IP_Hdr(reassem4, true); + } + + else if ( ((const struct ip*)pkt_start)->ip_v == 6 ) + { + struct ip6_hdr* reassem6 = (struct ip6_hdr*) pkt_start; + reassem6->ip6_plen = htons(frag_size + proto_hdr_len - 40); + const IPv6_Hdr_Chain* chain = new IPv6_Hdr_Chain(reassem6, next_proto, n); + reassembled_pkt = new IP_Hdr(reassem6, true, n, chain); + } + + else + { + reporter->InternalError("bad IP version in fragment reassembly"); + } + DeleteTimer(); } diff --git a/src/Frag.h b/src/Frag.h index 92bf1b3bbd..86cf3a9dd4 100644 --- a/src/Frag.h +++ b/src/Frag.h @@ -20,11 +20,10 @@ typedef void (FragReassembler::*frag_timer_func)(double t); class FragReassembler : public Reassembler { public: FragReassembler(NetSessions* s, const IP_Hdr* ip, const u_char* pkt, - uint32 frag_field, HashKey* k, double t); + HashKey* k, double t); ~FragReassembler(); - void AddFragment(double t, const IP_Hdr* ip, const u_char* pkt, - uint32 frag_field); + void AddFragment(double t, const IP_Hdr* ip, const u_char* pkt); void Expire(double t); void DeleteTimer(); @@ -37,11 +36,12 @@ protected: void BlockInserted(DataBlock* start_block); void Overlap(const u_char* b1, const u_char* b2, int n); - struct ip* proto_hdr; + u_char* proto_hdr; IP_Hdr* reassembled_pkt; int proto_hdr_len; NetSessions* s; int frag_size; // size of fully reassembled fragment + uint16 next_proto; // first IPv6 fragment header's next proto field HashKey* key; FragTimer* expire_timer; diff --git a/src/Gnutella.cc b/src/Gnutella.cc index 448c8dcb3b..6b5e901bc5 100644 --- a/src/Gnutella.cc +++ b/src/Gnutella.cc @@ -42,6 +42,12 @@ Gnutella_Analyzer::Gnutella_Analyzer(Connection* conn) resp_msg_state = new GnutellaMsgState(); } +Gnutella_Analyzer::~Gnutella_Analyzer() + { + delete orig_msg_state; + delete resp_msg_state; + } + void Gnutella_Analyzer::Done() { TCP_ApplicationAnalyzer::Done(); diff --git a/src/Gnutella.h b/src/Gnutella.h index f06c816c90..455876462d 100644 --- a/src/Gnutella.h +++ b/src/Gnutella.h @@ -35,6 +35,7 @@ public: class Gnutella_Analyzer : public TCP_ApplicationAnalyzer { public: Gnutella_Analyzer(Connection* conn); + ~Gnutella_Analyzer(); virtual void Done (); virtual void DeliverStream(int len, const u_char* data, bool orig); diff --git a/src/HTTP.cc b/src/HTTP.cc index b41933156d..9d9f01be64 100644 --- a/src/HTTP.cc +++ b/src/HTTP.cc @@ -1543,7 +1543,7 @@ void HTTP_Analyzer::HTTP_Header(int is_orig, MIME_Header* h) } } -void HTTP_Analyzer::ParseVersion(data_chunk_t ver, const uint32* host, +void HTTP_Analyzer::ParseVersion(data_chunk_t ver, const IPAddr& host, bool user_agent) { int len = ver.length; diff --git a/src/HTTP.h b/src/HTTP.h index 00524da20d..c9d8ae55d1 100644 --- a/src/HTTP.h +++ b/src/HTTP.h @@ -8,6 +8,7 @@ #include "MIME.h" #include "binpac_bro.h" #include "ZIP.h" +#include "IPAddr.h" enum CHUNKED_TRANSFER_STATE { NON_CHUNKED_TRANSFER, @@ -212,7 +213,7 @@ protected: const BroString* UnansweredRequestMethod(); - void ParseVersion(data_chunk_t ver, const uint32* host, bool user_agent); + void ParseVersion(data_chunk_t ver, const IPAddr& host, bool user_agent); int HTTP_ReplyCode(const char* code_str); int ExpectReplyMessageBody(); diff --git a/src/Hash.h b/src/Hash.h index 3a1b42084c..00db53d075 100644 --- a/src/Hash.h +++ b/src/Hash.h @@ -7,7 +7,7 @@ #include "BroString.h" -#define UHASH_KEY_SIZE 32 +#define UHASH_KEY_SIZE 36 typedef uint64 hash_t; diff --git a/src/ICMP.cc b/src/ICMP.cc index bc081ace51..dd2108ebf0 100644 --- a/src/ICMP.cc +++ b/src/ICMP.cc @@ -9,6 +9,8 @@ #include "Event.h" #include "ICMP.h" +#include + ICMP_Analyzer::ICMP_Analyzer(Connection* c) : TransportLayerAnalyzer(AnalyzerTag::ICMP, c) { @@ -32,7 +34,7 @@ void ICMP_Analyzer::Done() matcher_state.FinishEndpointMatcher(); } -void ICMP_Analyzer::DeliverPacket(int arg_len, const u_char* data, +void ICMP_Analyzer::DeliverPacket(int len, const u_char* data, bool is_orig, int seq, const IP_Hdr* ip, int caplen) { assert(ip); @@ -46,13 +48,33 @@ void ICMP_Analyzer::DeliverPacket(int arg_len, const u_char* data, PacketContents(data + 8, min(len, caplen) - 8); const struct icmp* icmpp = (const struct icmp*) data; - len = arg_len; - if ( ! ignore_checksums && caplen >= len && - icmp_checksum(icmpp, len) != 0xffff ) + assert(caplen >= len); // Should have been caught earlier already. + + if ( ! ignore_checksums ) { - Weird("bad_ICMP_checksum"); - return; + int chksum = 0; + + switch ( ip->NextProto() ) + { + case IPPROTO_ICMP: + chksum = icmp_checksum(icmpp, len); + break; + + case IPPROTO_ICMPV6: + chksum = icmp6_checksum(icmpp, ip, len); + break; + + default: + reporter->InternalError("unexpected IP proto in ICMP analyzer"); + break; + } + + if ( chksum != 0xffff ) + { + Weird("bad_ICMP_checksum"); + return; + } } Conn()->SetLastTime(current_timestamp); @@ -77,7 +99,13 @@ void ICMP_Analyzer::DeliverPacket(int arg_len, const u_char* data, else len_stat += len; - NextICMP(current_timestamp, icmpp, len, caplen, data); + if ( ip->NextProto() == IPPROTO_ICMP ) + NextICMP4(current_timestamp, icmpp, len, caplen, data, ip); + else if ( ip->NextProto() == IPPROTO_ICMPV6 ) + NextICMP6(current_timestamp, icmpp, len, caplen, data, ip); + else + reporter->InternalError("unexpected next protocol in ICMP::DeliverPacket()"); + if ( caplen >= len ) ForwardPacket(len, data, is_orig, seq, ip, caplen); @@ -87,26 +115,91 @@ void ICMP_Analyzer::DeliverPacket(int arg_len, const u_char* data, false, false, true); } -void ICMP_Analyzer::NextICMP(double /* t */, const struct icmp* /* icmpp */, - int /* len */, int /* caplen */, - const u_char*& /* data */) +void ICMP_Analyzer::NextICMP4(double t, const struct icmp* icmpp, int len, int caplen, + const u_char*& data, const IP_Hdr* ip_hdr ) { - ICMPEvent(icmp_sent); + switch ( icmpp->icmp_type ) + { + case ICMP_ECHO: + case ICMP_ECHOREPLY: + Echo(t, icmpp, len, caplen, data, ip_hdr); + break; + + case ICMP_UNREACH: + case ICMP_TIMXCEED: + Context4(t, icmpp, len, caplen, data, ip_hdr); + break; + + default: + ICMPEvent(icmp_sent, icmpp, len, 0, ip_hdr); + break; + } } -void ICMP_Analyzer::ICMPEvent(EventHandlerPtr f) +void ICMP_Analyzer::NextICMP6(double t, const struct icmp* icmpp, int len, int caplen, + const u_char*& data, const IP_Hdr* ip_hdr ) { + switch ( icmpp->icmp_type ) + { + // Echo types. + case ICMP6_ECHO_REQUEST: + case ICMP6_ECHO_REPLY: + Echo(t, icmpp, len, caplen, data, ip_hdr); + break; + + // Error messages all have the same structure for their context, + // and are handled by the same function. + case ICMP6_PARAM_PROB: + case ICMP6_TIME_EXCEEDED: + case ICMP6_PACKET_TOO_BIG: + case ICMP6_DST_UNREACH: + Context6(t, icmpp, len, caplen, data, ip_hdr); + break; + + // Router related messages. + case ND_REDIRECT: + Redirect(t, icmpp, len, caplen, data, ip_hdr); + break; + case ND_ROUTER_ADVERT: + RouterAdvert(t, icmpp, len, caplen, data, ip_hdr); + break; + case ND_NEIGHBOR_ADVERT: + NeighborAdvert(t, icmpp, len, caplen, data, ip_hdr); + break; + case ND_NEIGHBOR_SOLICIT: + NeighborSolicit(t, icmpp, len, caplen, data, ip_hdr); + break; + case ND_ROUTER_SOLICIT: + case ICMP6_ROUTER_RENUMBERING: + Router(t, icmpp, len, caplen, data, ip_hdr); + break; + +#if 0 + // Currently not specifically implemented. + case MLD_LISTENER_QUERY: + case MLD_LISTENER_REPORT: + case MLD_LISTENER_REDUCTION: +#endif + default: + ICMPEvent(icmp_sent, icmpp, len, 1, ip_hdr); + break; + } + } + +void ICMP_Analyzer::ICMPEvent(EventHandlerPtr f, const struct icmp* icmpp, + int len, int icmpv6, const IP_Hdr* ip_hdr) + { if ( ! f ) return; val_list* vl = new val_list; vl->append(BuildConnVal()); - vl->append(BuildICMPVal()); - + vl->append(BuildICMPVal(icmpp, len, icmpv6, ip_hdr)); ConnectionEvent(f, vl); } -RecordVal* ICMP_Analyzer::BuildICMPVal() +RecordVal* ICMP_Analyzer::BuildICMPVal(const struct icmp* icmpp, int len, + int icmpv6, const IP_Hdr* ip_hdr) { if ( ! icmp_conn_val ) { @@ -114,9 +207,11 @@ RecordVal* ICMP_Analyzer::BuildICMPVal() icmp_conn_val->Assign(0, new AddrVal(Conn()->OrigAddr())); icmp_conn_val->Assign(1, new AddrVal(Conn()->RespAddr())); - icmp_conn_val->Assign(2, new Val(type, TYPE_COUNT)); - icmp_conn_val->Assign(3, new Val(code, TYPE_COUNT)); + icmp_conn_val->Assign(2, new Val(icmpp->icmp_type, TYPE_COUNT)); + icmp_conn_val->Assign(3, new Val(icmpp->icmp_code, TYPE_COUNT)); icmp_conn_val->Assign(4, new Val(len, TYPE_COUNT)); + icmp_conn_val->Assign(5, new Val(ip_hdr->TTL(), TYPE_COUNT)); + icmp_conn_val->Assign(6, new Val(icmpv6, TYPE_BOOL)); } Ref(icmp_conn_val); @@ -124,91 +219,115 @@ RecordVal* ICMP_Analyzer::BuildICMPVal() return icmp_conn_val; } -RecordVal* ICMP_Analyzer::ExtractICMPContext(int len, const u_char*& data) +TransportProto ICMP_Analyzer::GetContextProtocol(const IP_Hdr* ip_hdr, uint32* src_port, uint32* dst_port) { - const struct ip* ip = (const struct ip *) data; - uint32 ip_hdr_len = ip->ip_hl * 4; + const u_char* transport_hdr; + uint32 ip_hdr_len = ip_hdr->HdrLen(); + bool ip4 = ip_hdr->IP4_Hdr(); + + if ( ip4 ) + transport_hdr = ((u_char *) ip_hdr->IP4_Hdr() + ip_hdr_len); + else + transport_hdr = ((u_char *) ip_hdr->IP6_Hdr() + ip_hdr_len); + + TransportProto proto; + + switch ( ip_hdr->NextProto() ) { + case 1: proto = TRANSPORT_ICMP; break; + case 6: proto = TRANSPORT_TCP; break; + case 17: proto = TRANSPORT_UDP; break; + case 58: proto = TRANSPORT_ICMP; break; + default: proto = TRANSPORT_UNKNOWN; break; + } + + switch ( proto ) { + case TRANSPORT_ICMP: + { + const struct icmp* icmpp = + (const struct icmp *) transport_hdr; + bool is_one_way; // dummy + *src_port = ntohs(icmpp->icmp_type); + + if ( ip4 ) + *dst_port = ntohs(ICMP4_counterpart(icmpp->icmp_type, + icmpp->icmp_code, is_one_way)); + else + *dst_port = ntohs(ICMP6_counterpart(icmpp->icmp_type, + icmpp->icmp_code, is_one_way)); + + break; + } + + case TRANSPORT_TCP: + { + const struct tcphdr* tp = + (const struct tcphdr *) transport_hdr; + *src_port = ntohs(tp->th_sport); + *dst_port = ntohs(tp->th_dport); + break; + } + + case TRANSPORT_UDP: + { + const struct udphdr* up = + (const struct udphdr *) transport_hdr; + *src_port = ntohs(up->uh_sport); + *dst_port = ntohs(up->uh_dport); + break; + } + + default: + *src_port = *dst_port = ntohs(0); + break; + } + + return proto; + } + +RecordVal* ICMP_Analyzer::ExtractICMP4Context(int len, const u_char*& data) + { + const IP_Hdr ip_hdr_data((const struct ip*) data, false); + const IP_Hdr* ip_hdr = &ip_hdr_data; + + uint32 ip_hdr_len = ip_hdr->HdrLen(); uint32 ip_len, frag_offset; TransportProto proto = TRANSPORT_UNKNOWN; int DF, MF, bad_hdr_len, bad_checksum; - uint32 src_addr, dst_addr; + IPAddr src_addr, dst_addr; uint32 src_port, dst_port; - if ( ip_hdr_len < sizeof(struct ip) || ip_hdr_len > uint32(len) ) - { // We don't have an entire IP header. + if ( len < (int)sizeof(struct ip) || ip_hdr_len > uint32(len) ) + { + // We don't have an entire IP header. bad_hdr_len = 1; ip_len = frag_offset = 0; DF = MF = bad_checksum = 0; - src_addr = dst_addr = 0; src_port = dst_port = 0; } else { bad_hdr_len = 0; - ip_len = ntohs(ip->ip_len); - bad_checksum = ones_complement_checksum((void*) ip, ip_hdr_len, 0) != 0xffff; + ip_len = ip_hdr->TotalLen(); + bad_checksum = (ones_complement_checksum((void*) ip_hdr->IP4_Hdr(), ip_hdr_len, 0) != 0xffff); - src_addr = uint32(ip->ip_src.s_addr); - dst_addr = uint32(ip->ip_dst.s_addr); + src_addr = ip_hdr->SrcAddr(); + dst_addr = ip_hdr->DstAddr(); - switch ( ip->ip_p ) { - case 1: proto = TRANSPORT_ICMP; break; - case 6: proto = TRANSPORT_TCP; break; - case 17: proto = TRANSPORT_UDP; break; + DF = ip_hdr->DF(); + MF = ip_hdr->MF(); + frag_offset = ip_hdr->FragOffset(); - // Default uses TRANSPORT_UNKNOWN, per initialization above. - } - - uint32 frag_field = ntohs(ip->ip_off); - DF = frag_field & 0x4000; - MF = frag_field & 0x2000; - frag_offset = frag_field & /* IP_OFFMASK not portable */ 0x1fff; - const u_char* transport_hdr = ((u_char *) ip + ip_hdr_len); - - if ( uint32(len) < ip_hdr_len + 4 ) + if ( uint32(len) >= ip_hdr_len + 4 ) + proto = GetContextProtocol(ip_hdr, &src_port, &dst_port); + else { // 4 above is the magic number meaning that both // port numbers are included in the ICMP. - bad_hdr_len = 1; src_port = dst_port = 0; + bad_hdr_len = 1; } - - switch ( proto ) { - case TRANSPORT_ICMP: - { - const struct icmp* icmpp = - (const struct icmp *) transport_hdr; - bool is_one_way; // dummy - src_port = ntohs(icmpp->icmp_type); - dst_port = ntohs(ICMP_counterpart(icmpp->icmp_type, - icmpp->icmp_code, - is_one_way)); - } - break; - - case TRANSPORT_TCP: - { - const struct tcphdr* tp = - (const struct tcphdr *) transport_hdr; - src_port = ntohs(tp->th_sport); - dst_port = ntohs(tp->th_dport); - } - break; - - case TRANSPORT_UDP: - { - const struct udphdr* up = - (const struct udphdr *) transport_hdr; - src_port = ntohs(up->uh_sport); - dst_port = ntohs(up->uh_dport); - } - break; - - default: - src_port = dst_port = ntohs(0); - } } RecordVal* iprec = new RecordVal(icmp_context); @@ -218,8 +337,8 @@ RecordVal* ICMP_Analyzer::ExtractICMPContext(int len, const u_char*& data) id_val->Assign(1, new PortVal(src_port, proto)); id_val->Assign(2, new AddrVal(dst_addr)); id_val->Assign(3, new PortVal(dst_port, proto)); - iprec->Assign(0, id_val); + iprec->Assign(0, id_val); iprec->Assign(1, new Val(ip_len, TYPE_COUNT)); iprec->Assign(2, new Val(proto, TYPE_COUNT)); iprec->Assign(3, new Val(frag_offset, TYPE_COUNT)); @@ -231,6 +350,66 @@ RecordVal* ICMP_Analyzer::ExtractICMPContext(int len, const u_char*& data) return iprec; } +RecordVal* ICMP_Analyzer::ExtractICMP6Context(int len, const u_char*& data) + { + int DF = 0, MF = 0, bad_hdr_len = 0; + TransportProto proto = TRANSPORT_UNKNOWN; + + IPAddr src_addr; + IPAddr dst_addr; + uint32 ip_len, frag_offset = 0; + uint32 src_port, dst_port; + + if ( len < (int)sizeof(struct ip6_hdr) ) + { + bad_hdr_len = 1; + ip_len = 0; + src_port = dst_port = 0; + } + else + { + const IP_Hdr ip_hdr_data((const struct ip6_hdr*) data, false, len); + const IP_Hdr* ip_hdr = &ip_hdr_data; + + ip_len = ip_hdr->TotalLen(); + src_addr = ip_hdr->SrcAddr(); + dst_addr = ip_hdr->DstAddr(); + frag_offset = ip_hdr->FragOffset(); + MF = ip_hdr->MF(); + DF = ip_hdr->DF(); + + if ( uint32(len) >= uint32(ip_hdr->HdrLen() + 4) ) + proto = GetContextProtocol(ip_hdr, &src_port, &dst_port); + else + { + // 4 above is the magic number meaning that both + // port numbers are included in the ICMP. + src_port = dst_port = 0; + bad_hdr_len = 1; + } + } + + RecordVal* iprec = new RecordVal(icmp_context); + RecordVal* id_val = new RecordVal(conn_id); + + id_val->Assign(0, new AddrVal(src_addr)); + id_val->Assign(1, new PortVal(src_port, proto)); + id_val->Assign(2, new AddrVal(dst_addr)); + id_val->Assign(3, new PortVal(dst_port, proto)); + + iprec->Assign(0, id_val); + iprec->Assign(1, new Val(ip_len, TYPE_COUNT)); + iprec->Assign(2, new Val(proto, TYPE_COUNT)); + iprec->Assign(3, new Val(frag_offset, TYPE_COUNT)); + iprec->Assign(4, new Val(bad_hdr_len, TYPE_BOOL)); + // bad_checksum is always false since IPv6 layer doesn't have a checksum. + iprec->Assign(5, new Val(0, TYPE_BOOL)); + iprec->Assign(6, new Val(MF, TYPE_BOOL)); + iprec->Assign(7, new Val(DF, TYPE_BOOL)); + + return iprec; + } + bool ICMP_Analyzer::IsReuse(double /* t */, const u_char* /* pkt */) { return 0; @@ -243,7 +422,7 @@ void ICMP_Analyzer::Describe(ODesc* d) const d->Add(Conn()->LastTime()); d->AddSP(")"); - d->Add(dotted_addr(Conn()->OrigAddr())); + d->Add(Conn()->OrigAddr()); d->Add("."); d->Add(type); d->Add("."); @@ -252,7 +431,7 @@ void ICMP_Analyzer::Describe(ODesc* d) const d->SP(); d->AddSP("->"); - d->Add(dotted_addr(Conn()->RespAddr())); + d->Add(Conn()->RespAddr()); } void ICMP_Analyzer::UpdateConnVal(RecordVal *conn_val) @@ -294,15 +473,20 @@ unsigned int ICMP_Analyzer::MemoryAllocation() const + (icmp_conn_val ? icmp_conn_val->MemoryAllocation() : 0); } -ICMP_Echo_Analyzer::ICMP_Echo_Analyzer(Connection* c) -: ICMP_Analyzer(AnalyzerTag::ICMP_Echo, c) - { - } -void ICMP_Echo_Analyzer::NextICMP(double t, const struct icmp* icmpp, int len, - int caplen, const u_char*& data) +void ICMP_Analyzer::Echo(double t, const struct icmp* icmpp, int len, + int caplen, const u_char*& data, const IP_Hdr* ip_hdr) { - EventHandlerPtr f = type == ICMP_ECHO ? icmp_echo_request : icmp_echo_reply; + // For handling all Echo related ICMP messages + EventHandlerPtr f = 0; + + if ( ip_hdr->NextProto() == IPPROTO_ICMPV6 ) + f = (icmpp->icmp_type == ICMP6_ECHO_REQUEST) + ? icmp_echo_request : icmp_echo_reply; + else + f = (icmpp->icmp_type == ICMP_ECHO) + ? icmp_echo_request : icmp_echo_reply; + if ( ! f ) return; @@ -313,7 +497,7 @@ void ICMP_Echo_Analyzer::NextICMP(double t, const struct icmp* icmpp, int len, val_list* vl = new val_list; vl->append(BuildConnVal()); - vl->append(BuildICMPVal()); + vl->append(BuildICMPVal(icmpp, len, ip_hdr->NextProto() != IPPROTO_ICMP, ip_hdr)); vl->append(new Val(iid, TYPE_COUNT)); vl->append(new Val(iseq, TYPE_COUNT)); vl->append(new StringVal(payload)); @@ -321,66 +505,232 @@ void ICMP_Echo_Analyzer::NextICMP(double t, const struct icmp* icmpp, int len, ConnectionEvent(f, vl); } -ICMP_Redir_Analyzer::ICMP_Redir_Analyzer(Connection* c) -: ICMP_Analyzer(AnalyzerTag::ICMP_Redir, c) - { - } -void ICMP_Redir_Analyzer::NextICMP(double t, const struct icmp* icmpp, int len, - int caplen, const u_char*& data) +void ICMP_Analyzer::RouterAdvert(double t, const struct icmp* icmpp, int len, + int caplen, const u_char*& data, const IP_Hdr* ip_hdr) { - uint32 addr = ntohl(icmpp->icmp_hun.ih_void); + EventHandlerPtr f = icmp_router_advertisement; + uint32 reachable, retrans; + + memcpy(&reachable, data, sizeof(reachable)); + memcpy(&retrans, data + sizeof(reachable), sizeof(retrans)); val_list* vl = new val_list; vl->append(BuildConnVal()); - vl->append(BuildICMPVal()); - vl->append(new AddrVal(htonl(addr))); + vl->append(BuildICMPVal(icmpp, len, 1, ip_hdr)); + vl->append(new Val(icmpp->icmp_num_addrs, TYPE_COUNT)); // Cur Hop Limit + vl->append(new Val(icmpp->icmp_wpa & 0x80, TYPE_BOOL)); // Managed + vl->append(new Val(icmpp->icmp_wpa & 0x40, TYPE_BOOL)); // Other + vl->append(new Val(icmpp->icmp_wpa & 0x20, TYPE_BOOL)); // Home Agent + vl->append(new Val((icmpp->icmp_wpa & 0x18)>>3, TYPE_COUNT)); // Pref + vl->append(new Val(icmpp->icmp_wpa & 0x04, TYPE_BOOL)); // Proxy + vl->append(new Val(icmpp->icmp_wpa & 0x02, TYPE_COUNT)); // Reserved + vl->append(new IntervalVal((double)ntohs(icmpp->icmp_lifetime), Seconds)); + vl->append(new IntervalVal((double)ntohl(reachable), Milliseconds)); + vl->append(new IntervalVal((double)ntohl(retrans), Milliseconds)); - ConnectionEvent(icmp_redirect, vl); + ConnectionEvent(f, vl); } -void ICMP_Context_Analyzer::NextICMP(double t, const struct icmp* icmpp, - int len, int caplen, const u_char*& data) +void ICMP_Analyzer::NeighborAdvert(double t, const struct icmp* icmpp, int len, + int caplen, const u_char*& data, const IP_Hdr* ip_hdr) + { + EventHandlerPtr f = icmp_neighbor_advertisement; + in6_addr tgtaddr; + + memcpy(&tgtaddr.s6_addr, data, sizeof(tgtaddr.s6_addr)); + + val_list* vl = new val_list; + vl->append(BuildConnVal()); + vl->append(BuildICMPVal(icmpp, len, 1, ip_hdr)); + vl->append(new Val(icmpp->icmp_num_addrs & 0x80, TYPE_BOOL)); // Router + vl->append(new Val(icmpp->icmp_num_addrs & 0x40, TYPE_BOOL)); // Solicited + vl->append(new Val(icmpp->icmp_num_addrs & 0x20, TYPE_BOOL)); // Override + vl->append(new AddrVal(IPAddr(tgtaddr))); + + ConnectionEvent(f, vl); + } + + +void ICMP_Analyzer::NeighborSolicit(double t, const struct icmp* icmpp, int len, + int caplen, const u_char*& data, const IP_Hdr* ip_hdr) + { + EventHandlerPtr f = icmp_neighbor_solicitation; + in6_addr tgtaddr; + + memcpy(&tgtaddr.s6_addr, data, sizeof(tgtaddr.s6_addr)); + + val_list* vl = new val_list; + vl->append(BuildConnVal()); + vl->append(BuildICMPVal(icmpp, len, 1, ip_hdr)); + vl->append(new AddrVal(IPAddr(tgtaddr))); + + ConnectionEvent(f, vl); + } + + +void ICMP_Analyzer::Redirect(double t, const struct icmp* icmpp, int len, + int caplen, const u_char*& data, const IP_Hdr* ip_hdr) + { + EventHandlerPtr f = icmp_redirect; + in6_addr tgtaddr, dstaddr; + + memcpy(&tgtaddr.s6_addr, data, sizeof(tgtaddr.s6_addr)); + memcpy(&dstaddr.s6_addr, data + sizeof(tgtaddr.s6_addr), sizeof(dstaddr.s6_addr)); + + val_list* vl = new val_list; + vl->append(BuildConnVal()); + vl->append(BuildICMPVal(icmpp, len, 1, ip_hdr)); + vl->append(new AddrVal(IPAddr(tgtaddr))); + vl->append(new AddrVal(IPAddr(dstaddr))); + + ConnectionEvent(f, vl); + } + + +void ICMP_Analyzer::Router(double t, const struct icmp* icmpp, int len, + int caplen, const u_char*& data, const IP_Hdr* ip_hdr) { EventHandlerPtr f = 0; - switch ( type ) { - case ICMP_UNREACH: f = icmp_unreachable; break; - case ICMP_TIMXCEED: f = icmp_time_exceeded; break; + + switch ( icmpp->icmp_type ) + { + case ND_ROUTER_SOLICIT: + f = icmp_router_solicitation; + break; + case ICMP6_ROUTER_RENUMBERING: + default: + ICMPEvent(icmp_sent, icmpp, len, 1, ip_hdr); + return; + } + + val_list* vl = new val_list; + vl->append(BuildConnVal()); + vl->append(BuildICMPVal(icmpp, len, 1, ip_hdr)); + + ConnectionEvent(f, vl); } + +void ICMP_Analyzer::Context4(double t, const struct icmp* icmpp, + int len, int caplen, const u_char*& data, const IP_Hdr* ip_hdr) + { + EventHandlerPtr f = 0; + + switch ( icmpp->icmp_type ) + { + case ICMP_UNREACH: + f = icmp_unreachable; + break; + + case ICMP_TIMXCEED: + f = icmp_time_exceeded; + break; + } + if ( f ) { val_list* vl = new val_list; vl->append(BuildConnVal()); - vl->append(BuildICMPVal()); - vl->append(new Val(code, TYPE_COUNT)); - vl->append(ExtractICMPContext(caplen, data)); - + vl->append(BuildICMPVal(icmpp, len, 0, ip_hdr)); + vl->append(new Val(icmpp->icmp_code, TYPE_COUNT)); + vl->append(ExtractICMP4Context(caplen, data)); ConnectionEvent(f, vl); } } -int ICMP_counterpart(int icmp_type, int icmp_code, bool& is_one_way) +void ICMP_Analyzer::Context6(double t, const struct icmp* icmpp, + int len, int caplen, const u_char*& data, const IP_Hdr* ip_hdr) + { + EventHandlerPtr f = 0; + + switch ( icmpp->icmp_type ) + { + case ICMP6_DST_UNREACH: + f = icmp_unreachable; + break; + + case ICMP6_PARAM_PROB: + f = icmp_parameter_problem; + break; + + case ICMP6_TIME_EXCEEDED: + f = icmp_time_exceeded; + break; + + case ICMP6_PACKET_TOO_BIG: + f = icmp_packet_too_big; + break; + } + + if ( f ) + { + val_list* vl = new val_list; + vl->append(BuildConnVal()); + vl->append(BuildICMPVal(icmpp, len, 1, ip_hdr)); + vl->append(new Val(icmpp->icmp_code, TYPE_COUNT)); + vl->append(ExtractICMP6Context(caplen, data)); + ConnectionEvent(f, vl); + } + } + +int ICMP4_counterpart(int icmp_type, int icmp_code, bool& is_one_way) { is_one_way = false; - // return the counterpart type if one exists. This allows us + // Return the counterpart type if one exists. This allows us // to track corresponding ICMP requests/replies. // Note that for the two-way ICMP messages, icmp_code is // always 0 (RFC 792). switch ( icmp_type ) { case ICMP_ECHO: return ICMP_ECHOREPLY; case ICMP_ECHOREPLY: return ICMP_ECHO; + case ICMP_TSTAMP: return ICMP_TSTAMPREPLY; case ICMP_TSTAMPREPLY: return ICMP_TSTAMP; + case ICMP_IREQ: return ICMP_IREQREPLY; case ICMP_IREQREPLY: return ICMP_IREQ; + case ICMP_ROUTERSOLICIT: return ICMP_ROUTERADVERT; + case ICMP_MASKREQ: return ICMP_MASKREPLY; case ICMP_MASKREPLY: return ICMP_MASKREQ; default: is_one_way = true; return icmp_code; } } + +int ICMP6_counterpart(int icmp_type, int icmp_code, bool& is_one_way) + { + is_one_way = false; + + switch ( icmp_type ) { + case ICMP6_ECHO_REQUEST: return ICMP6_ECHO_REPLY; + case ICMP6_ECHO_REPLY: return ICMP6_ECHO_REQUEST; + + case ND_ROUTER_SOLICIT: return ND_ROUTER_ADVERT; + case ND_ROUTER_ADVERT: return ND_ROUTER_SOLICIT; + + case ND_NEIGHBOR_SOLICIT: return ND_NEIGHBOR_ADVERT; + case ND_NEIGHBOR_ADVERT: return ND_NEIGHBOR_SOLICIT; + + case MLD_LISTENER_QUERY: return MLD_LISTENER_REPORT; + case MLD_LISTENER_REPORT: return MLD_LISTENER_QUERY; + + // ICMP node information query and response respectively (not defined in + // icmp6.h) + case 139: return 140; + case 140: return 139; + + // Home Agent Address Discovery Request Message and reply + case 144: return 145; + case 145: return 144; + + // TODO: Add further counterparts. + + default: is_one_way = true; return icmp_code; + } + } diff --git a/src/ICMP.h b/src/ICMP.h index ad43d7b948..33773b9762 100644 --- a/src/ICMP.h +++ b/src/ICMP.h @@ -33,21 +33,51 @@ protected: virtual bool IsReuse(double t, const u_char* pkt); virtual unsigned int MemoryAllocation() const; - void ICMPEvent(EventHandlerPtr f); + void ICMPEvent(EventHandlerPtr f, const struct icmp* icmpp, int len, + int icmpv6, const IP_Hdr* ip_hdr); + + void Echo(double t, const struct icmp* icmpp, int len, + int caplen, const u_char*& data, const IP_Hdr* ip_hdr); + void Context(double t, const struct icmp* icmpp, int len, + int caplen, const u_char*& data, const IP_Hdr* ip_hdr); + void Redirect(double t, const struct icmp* icmpp, int len, + int caplen, const u_char*& data, const IP_Hdr* ip_hdr); + void RouterAdvert(double t, const struct icmp* icmpp, int len, + int caplen, const u_char*& data, const IP_Hdr* ip_hdr); + void NeighborAdvert(double t, const struct icmp* icmpp, int len, + int caplen, const u_char*& data, const IP_Hdr* ip_hdr); + void NeighborSolicit(double t, const struct icmp* icmpp, int len, + int caplen, const u_char*& data, const IP_Hdr* ip_hdr); + void Router(double t, const struct icmp* icmpp, int len, + int caplen, const u_char*& data, const IP_Hdr* ip_hdr); + void Describe(ODesc* d) const; - RecordVal* BuildICMPVal(); + RecordVal* BuildICMPVal(const struct icmp* icmpp, int len, int icmpv6, + const IP_Hdr* ip_hdr); - virtual void NextICMP(double t, const struct icmp* icmpp, - int len, int caplen, const u_char*& data); + void NextICMP4(double t, const struct icmp* icmpp, int len, int caplen, + const u_char*& data, const IP_Hdr* ip_hdr ); - RecordVal* ExtractICMPContext(int len, const u_char*& data); + RecordVal* ExtractICMP4Context(int len, const u_char*& data); + + void Context4(double t, const struct icmp* icmpp, int len, int caplen, + const u_char*& data, const IP_Hdr* ip_hdr); + + TransportProto GetContextProtocol(const IP_Hdr* ip_hdr, uint32* src_port, + uint32* dst_port); + + void NextICMP6(double t, const struct icmp* icmpp, int len, int caplen, + const u_char*& data, const IP_Hdr* ip_hdr ); + + RecordVal* ExtractICMP6Context(int len, const u_char*& data); + + void Context6(double t, const struct icmp* icmpp, int len, int caplen, + const u_char*& data, const IP_Hdr* ip_hdr); RecordVal* icmp_conn_val; int type; int code; - int len; - int request_len, reply_len; RuleMatcherState matcher_state; @@ -56,81 +86,9 @@ private: void UpdateEndpointVal(RecordVal* endp, int is_orig); }; -class ICMP_Echo_Analyzer : public ICMP_Analyzer { -public: - ICMP_Echo_Analyzer(Connection* conn); - - static Analyzer* InstantiateAnalyzer(Connection* conn) - { return new ICMP_Echo_Analyzer(conn); } - - static bool Available() { return icmp_echo_request || icmp_echo_reply; } - -protected: - ICMP_Echo_Analyzer() { } - - virtual void NextICMP(double t, const struct icmp* icmpp, - int len, int caplen, const u_char*& data); -}; - -class ICMP_Redir_Analyzer : public ICMP_Analyzer { -public: - ICMP_Redir_Analyzer(Connection* conn); - - static Analyzer* InstantiateAnalyzer(Connection* conn) - { return new ICMP_Redir_Analyzer(conn); } - - static bool Available() { return icmp_redirect; } - -protected: - ICMP_Redir_Analyzer() { } - - virtual void NextICMP(double t, const struct icmp* icmpp, - int len, int caplen, const u_char*& data); -}; - -class ICMP_Context_Analyzer : public ICMP_Analyzer { -public: - ICMP_Context_Analyzer(AnalyzerTag::Tag tag, Connection* conn) - : ICMP_Analyzer(tag, conn) { } - -protected: - ICMP_Context_Analyzer() { } - - virtual void NextICMP(double t, const struct icmp* icmpp, - int len, int caplen, const u_char*& data); -}; - -class ICMP_TimeExceeded_Analyzer : public ICMP_Context_Analyzer { -public: - ICMP_TimeExceeded_Analyzer(Connection* conn) - : ICMP_Context_Analyzer(AnalyzerTag::ICMP_TimeExceeded, conn) { } - - static Analyzer* InstantiateAnalyzer(Connection* conn) - { return new ICMP_TimeExceeded_Analyzer(conn); } - - static bool Available() { return icmp_time_exceeded; } - -protected: - ICMP_TimeExceeded_Analyzer() { } -}; - -class ICMP_Unreachable_Analyzer : public ICMP_Context_Analyzer { -public: - ICMP_Unreachable_Analyzer(Connection* conn) - : ICMP_Context_Analyzer(AnalyzerTag::ICMP_Unreachable, conn) { } - - static Analyzer* InstantiateAnalyzer(Connection* conn) - { return new ICMP_Unreachable_Analyzer(conn); } - - static bool Available() { return icmp_unreachable; } - -protected: - ICMP_Unreachable_Analyzer() { } -}; - - // Returns the counterpart type to the given type (e.g., the counterpart // to ICMP_ECHOREPLY is ICMP_ECHO). -extern int ICMP_counterpart(int icmp_type, int icmp_code, bool& is_one_way); +extern int ICMP4_counterpart(int icmp_type, int icmp_code, bool& is_one_way); +extern int ICMP6_counterpart(int icmp_type, int icmp_code, bool& is_one_way); #endif diff --git a/src/ID.cc b/src/ID.cc index 3f5c76ca1d..a70aa3fd0e 100644 --- a/src/ID.cc +++ b/src/ID.cc @@ -372,7 +372,7 @@ ID* ID::Unserialize(UnserialInfo* info) Ref(id); global_scope()->Insert(id->Name(), id); -#ifdef USE_PERFTOOLS +#ifdef USE_PERFTOOLS_DEBUG heap_checker->IgnoreObject(id); #endif } diff --git a/src/IP.cc b/src/IP.cc new file mode 100644 index 0000000000..f5598600d5 --- /dev/null +++ b/src/IP.cc @@ -0,0 +1,627 @@ +// See the file "COPYING" in the main distribution directory for copyright. + +#include "IP.h" +#include "Type.h" +#include "Val.h" +#include "Var.h" + +static RecordType* ip4_hdr_type = 0; +static RecordType* ip6_hdr_type = 0; +static RecordType* ip6_ext_hdr_type = 0; +static RecordType* ip6_option_type = 0; +static RecordType* ip6_hopopts_type = 0; +static RecordType* ip6_dstopts_type = 0; +static RecordType* ip6_routing_type = 0; +static RecordType* ip6_fragment_type = 0; +static RecordType* ip6_ah_type = 0; +static RecordType* ip6_esp_type = 0; +static RecordType* ip6_mob_type = 0; +static RecordType* ip6_mob_msg_type = 0; +static RecordType* ip6_mob_brr_type = 0; +static RecordType* ip6_mob_hoti_type = 0; +static RecordType* ip6_mob_coti_type = 0; +static RecordType* ip6_mob_hot_type = 0; +static RecordType* ip6_mob_cot_type = 0; +static RecordType* ip6_mob_bu_type = 0; +static RecordType* ip6_mob_back_type = 0; +static RecordType* ip6_mob_be_type = 0; + +static inline RecordType* hdrType(RecordType*& type, const char* name) + { + if ( ! type ) + type = internal_type(name)->AsRecordType(); + + return type; + } + +static VectorVal* BuildOptionsVal(const u_char* data, int len) + { + VectorVal* vv = new VectorVal(new VectorType( + hdrType(ip6_option_type, "ip6_option")->Ref())); + + while ( len > 0 ) + { + const struct ip6_opt* opt = (const struct ip6_opt*) data; + RecordVal* rv = new RecordVal(ip6_option_type); + rv->Assign(0, new Val(opt->ip6o_type, TYPE_COUNT)); + + if ( opt->ip6o_type == 0 ) + { + // Pad1 option + rv->Assign(1, new Val(0, TYPE_COUNT)); + rv->Assign(2, new StringVal("")); + data += sizeof(uint8); + len -= sizeof(uint8); + } + else + { + // PadN or other option + uint16 off = 2 * sizeof(uint8); + rv->Assign(1, new Val(opt->ip6o_len, TYPE_COUNT)); + rv->Assign(2, new StringVal( + new BroString(data + off, opt->ip6o_len, 1))); + data += opt->ip6o_len + off; + len -= opt->ip6o_len + off; + } + + vv->Assign(vv->Size(), rv, 0); + } + + return vv; + } + +RecordVal* IPv6_Hdr::BuildRecordVal(VectorVal* chain) const + { + RecordVal* rv = 0; + + switch ( type ) { + case IPPROTO_IPV6: + { + rv = new RecordVal(hdrType(ip6_hdr_type, "ip6_hdr")); + const struct ip6_hdr* ip6 = (const struct ip6_hdr*)data; + rv->Assign(0, new Val((ntohl(ip6->ip6_flow) & 0x0ff00000)>>20, TYPE_COUNT)); + rv->Assign(1, new Val(ntohl(ip6->ip6_flow) & 0x000fffff, TYPE_COUNT)); + rv->Assign(2, new Val(ntohs(ip6->ip6_plen), TYPE_COUNT)); + rv->Assign(3, new Val(ip6->ip6_nxt, TYPE_COUNT)); + rv->Assign(4, new Val(ip6->ip6_hlim, TYPE_COUNT)); + rv->Assign(5, new AddrVal(IPAddr(ip6->ip6_src))); + rv->Assign(6, new AddrVal(IPAddr(ip6->ip6_dst))); + if ( ! chain ) + chain = new VectorVal(new VectorType( + hdrType(ip6_ext_hdr_type, "ip6_ext_hdr")->Ref())); + rv->Assign(7, chain); + } + break; + + case IPPROTO_HOPOPTS: + { + rv = new RecordVal(hdrType(ip6_hopopts_type, "ip6_hopopts")); + const struct ip6_hbh* hbh = (const struct ip6_hbh*)data; + rv->Assign(0, new Val(hbh->ip6h_nxt, TYPE_COUNT)); + rv->Assign(1, new Val(hbh->ip6h_len, TYPE_COUNT)); + uint16 off = 2 * sizeof(uint8); + rv->Assign(2, BuildOptionsVal(data + off, Length() - off)); + + } + break; + + case IPPROTO_DSTOPTS: + { + rv = new RecordVal(hdrType(ip6_dstopts_type, "ip6_dstopts")); + const struct ip6_dest* dst = (const struct ip6_dest*)data; + rv->Assign(0, new Val(dst->ip6d_nxt, TYPE_COUNT)); + rv->Assign(1, new Val(dst->ip6d_len, TYPE_COUNT)); + uint16 off = 2 * sizeof(uint8); + rv->Assign(2, BuildOptionsVal(data + off, Length() - off)); + } + break; + + case IPPROTO_ROUTING: + { + rv = new RecordVal(hdrType(ip6_routing_type, "ip6_routing")); + const struct ip6_rthdr* rt = (const struct ip6_rthdr*)data; + rv->Assign(0, new Val(rt->ip6r_nxt, TYPE_COUNT)); + rv->Assign(1, new Val(rt->ip6r_len, TYPE_COUNT)); + rv->Assign(2, new Val(rt->ip6r_type, TYPE_COUNT)); + rv->Assign(3, new Val(rt->ip6r_segleft, TYPE_COUNT)); + uint16 off = 4 * sizeof(uint8); + rv->Assign(4, new StringVal(new BroString(data + off, Length() - off, 1))); + } + break; + + case IPPROTO_FRAGMENT: + { + rv = new RecordVal(hdrType(ip6_fragment_type, "ip6_fragment")); + const struct ip6_frag* frag = (const struct ip6_frag*)data; + rv->Assign(0, new Val(frag->ip6f_nxt, TYPE_COUNT)); + rv->Assign(1, new Val(frag->ip6f_reserved, TYPE_COUNT)); + rv->Assign(2, new Val((ntohs(frag->ip6f_offlg) & 0xfff8)>>3, TYPE_COUNT)); + rv->Assign(3, new Val((ntohs(frag->ip6f_offlg) & 0x0006)>>1, TYPE_COUNT)); + rv->Assign(4, new Val(ntohs(frag->ip6f_offlg) & 0x0001, TYPE_BOOL)); + rv->Assign(5, new Val(ntohl(frag->ip6f_ident), TYPE_COUNT)); + } + break; + + case IPPROTO_AH: + { + rv = new RecordVal(hdrType(ip6_ah_type, "ip6_ah")); + rv->Assign(0, new Val(((ip6_ext*)data)->ip6e_nxt, TYPE_COUNT)); + rv->Assign(1, new Val(((ip6_ext*)data)->ip6e_len, TYPE_COUNT)); + rv->Assign(2, new Val(ntohs(((uint16*)data)[1]), TYPE_COUNT)); + rv->Assign(3, new Val(ntohl(((uint32*)data)[1]), TYPE_COUNT)); + rv->Assign(4, new Val(ntohl(((uint32*)data)[2]), TYPE_COUNT)); + uint16 off = 3 * sizeof(uint32); + rv->Assign(5, new StringVal(new BroString(data + off, Length() - off, 1))); + } + break; + + case IPPROTO_ESP: + { + rv = new RecordVal(hdrType(ip6_esp_type, "ip6_esp")); + const uint32* esp = (const uint32*)data; + rv->Assign(0, new Val(ntohl(esp[0]), TYPE_COUNT)); + rv->Assign(1, new Val(ntohl(esp[1]), TYPE_COUNT)); + } + break; + +#ifdef ENABLE_MOBILE_IPV6 + case IPPROTO_MOBILITY: + { + rv = new RecordVal(hdrType(ip6_mob_type, "ip6_mobility_hdr")); + const struct ip6_mobility* mob = (const struct ip6_mobility*) data; + rv->Assign(0, new Val(mob->ip6mob_payload, TYPE_COUNT)); + rv->Assign(1, new Val(mob->ip6mob_len, TYPE_COUNT)); + rv->Assign(2, new Val(mob->ip6mob_type, TYPE_COUNT)); + rv->Assign(3, new Val(mob->ip6mob_rsv, TYPE_COUNT)); + rv->Assign(4, new Val(ntohs(mob->ip6mob_chksum), TYPE_COUNT)); + + RecordVal* msg = new RecordVal(hdrType(ip6_mob_msg_type, "ip6_mobility_msg")); + msg->Assign(0, new Val(mob->ip6mob_type, TYPE_COUNT)); + + uint16 off = sizeof(ip6_mobility); + const u_char* msg_data = data + off; + + switch ( mob->ip6mob_type ) { + case 0: + { + RecordVal* m = new RecordVal(hdrType(ip6_mob_brr_type, "ip6_mobility_brr")); + m->Assign(0, new Val(ntohs(*((uint16*)msg_data)), TYPE_COUNT)); + off += sizeof(uint16); + m->Assign(1, BuildOptionsVal(data + off, Length() - off)); + msg->Assign(1, m); + } + break; + + case 1: + { + RecordVal* m = new RecordVal(hdrType(ip6_mob_brr_type, "ip6_mobility_hoti")); + m->Assign(0, new Val(ntohs(*((uint16*)msg_data)), TYPE_COUNT)); + m->Assign(1, new Val(ntohll(*((uint64*)(msg_data + sizeof(uint16)))), TYPE_COUNT)); + off += sizeof(uint16) + sizeof(uint64); + m->Assign(2, BuildOptionsVal(data + off, Length() - off)); + msg->Assign(2, m); + break; + } + + case 2: + { + RecordVal* m = new RecordVal(hdrType(ip6_mob_brr_type, "ip6_mobility_coti")); + m->Assign(0, new Val(ntohs(*((uint16*)msg_data)), TYPE_COUNT)); + m->Assign(1, new Val(ntohll(*((uint64*)(msg_data + sizeof(uint16)))), TYPE_COUNT)); + off += sizeof(uint16) + sizeof(uint64); + m->Assign(2, BuildOptionsVal(data + off, Length() - off)); + msg->Assign(3, m); + break; + } + + case 3: + { + RecordVal* m = new RecordVal(hdrType(ip6_mob_brr_type, "ip6_mobility_hot")); + m->Assign(0, new Val(ntohs(*((uint16*)msg_data)), TYPE_COUNT)); + m->Assign(1, new Val(ntohll(*((uint64*)(msg_data + sizeof(uint16)))), TYPE_COUNT)); + m->Assign(2, new Val(ntohll(*((uint64*)(msg_data + sizeof(uint16) + sizeof(uint64)))), TYPE_COUNT)); + off += sizeof(uint16) + 2 * sizeof(uint64); + m->Assign(3, BuildOptionsVal(data + off, Length() - off)); + msg->Assign(4, m); + break; + } + + case 4: + { + RecordVal* m = new RecordVal(hdrType(ip6_mob_brr_type, "ip6_mobility_cot")); + m->Assign(0, new Val(ntohs(*((uint16*)msg_data)), TYPE_COUNT)); + m->Assign(1, new Val(ntohll(*((uint64*)(msg_data + sizeof(uint16)))), TYPE_COUNT)); + m->Assign(2, new Val(ntohll(*((uint64*)(msg_data + sizeof(uint16) + sizeof(uint64)))), TYPE_COUNT)); + off += sizeof(uint16) + 2 * sizeof(uint64); + m->Assign(3, BuildOptionsVal(data + off, Length() - off)); + msg->Assign(5, m); + break; + } + + case 5: + { + RecordVal* m = new RecordVal(hdrType(ip6_mob_brr_type, "ip6_mobility_bu")); + m->Assign(0, new Val(ntohs(*((uint16*)msg_data)), TYPE_COUNT)); + m->Assign(1, new Val(ntohs(*((uint16*)(msg_data + sizeof(uint16)))) & 0x8000, TYPE_BOOL)); + m->Assign(2, new Val(ntohs(*((uint16*)(msg_data + sizeof(uint16)))) & 0x4000, TYPE_BOOL)); + m->Assign(3, new Val(ntohs(*((uint16*)(msg_data + sizeof(uint16)))) & 0x2000, TYPE_BOOL)); + m->Assign(4, new Val(ntohs(*((uint16*)(msg_data + sizeof(uint16)))) & 0x1000, TYPE_BOOL)); + m->Assign(5, new Val(ntohs(*((uint16*)(msg_data + 2*sizeof(uint16)))), TYPE_COUNT)); + off += 3 * sizeof(uint16); + m->Assign(6, BuildOptionsVal(data + off, Length() - off)); + msg->Assign(6, m); + break; + } + + case 6: + { + RecordVal* m = new RecordVal(hdrType(ip6_mob_brr_type, "ip6_mobility_back")); + m->Assign(0, new Val(*((uint8*)msg_data), TYPE_COUNT)); + m->Assign(1, new Val(*((uint8*)(msg_data + sizeof(uint8))) & 0x80, TYPE_BOOL)); + m->Assign(2, new Val(ntohs(*((uint16*)(msg_data + sizeof(uint16)))), TYPE_COUNT)); + m->Assign(3, new Val(ntohs(*((uint16*)(msg_data + 2*sizeof(uint16)))), TYPE_COUNT)); + off += 3 * sizeof(uint16); + m->Assign(4, BuildOptionsVal(data + off, Length() - off)); + msg->Assign(7, m); + break; + } + + case 7: + { + RecordVal* m = new RecordVal(hdrType(ip6_mob_brr_type, "ip6_mobility_be")); + m->Assign(0, new Val(*((uint8*)msg_data), TYPE_COUNT)); + const in6_addr* hoa = (const in6_addr*)(msg_data + sizeof(uint16)); + m->Assign(1, new AddrVal(IPAddr(*hoa))); + off += sizeof(uint16) + sizeof(in6_addr); + m->Assign(2, BuildOptionsVal(data + off, Length() - off)); + msg->Assign(8, m); + break; + } + + default: + reporter->Weird(fmt("unknown_mobility_type_%d", mob->ip6mob_type)); + break; + } + + rv->Assign(5, msg); + } + break; +#endif //ENABLE_MOBILE_IPV6 + + default: + break; + } + + return rv; + } + +RecordVal* IP_Hdr::BuildIPHdrVal() const + { + RecordVal* rval = 0; + + if ( ip4 ) + { + rval = new RecordVal(hdrType(ip4_hdr_type, "ip4_hdr")); + rval->Assign(0, new Val(ip4->ip_hl * 4, TYPE_COUNT)); + rval->Assign(1, new Val(ip4->ip_tos, TYPE_COUNT)); + rval->Assign(2, new Val(ntohs(ip4->ip_len), TYPE_COUNT)); + rval->Assign(3, new Val(ntohs(ip4->ip_id), TYPE_COUNT)); + rval->Assign(4, new Val(ip4->ip_ttl, TYPE_COUNT)); + rval->Assign(5, new Val(ip4->ip_p, TYPE_COUNT)); + rval->Assign(6, new AddrVal(ip4->ip_src.s_addr)); + rval->Assign(7, new AddrVal(ip4->ip_dst.s_addr)); + } + else + { + rval = ((*ip6_hdrs)[0])->BuildRecordVal(ip6_hdrs->BuildVal()); + } + + return rval; + } + +RecordVal* IP_Hdr::BuildPktHdrVal() const + { + static RecordType* pkt_hdr_type = 0; + static RecordType* tcp_hdr_type = 0; + static RecordType* udp_hdr_type = 0; + static RecordType* icmp_hdr_type = 0; + + if ( ! pkt_hdr_type ) + { + pkt_hdr_type = internal_type("pkt_hdr")->AsRecordType(); + tcp_hdr_type = internal_type("tcp_hdr")->AsRecordType(); + udp_hdr_type = internal_type("udp_hdr")->AsRecordType(); + icmp_hdr_type = internal_type("icmp_hdr")->AsRecordType(); + } + + RecordVal* pkt_hdr = new RecordVal(pkt_hdr_type); + + if ( ip4 ) + pkt_hdr->Assign(0, BuildIPHdrVal()); + else + pkt_hdr->Assign(1, BuildIPHdrVal()); + + // L4 header. + const u_char* data = Payload(); + + int proto = NextProto(); + switch ( proto ) { + case IPPROTO_TCP: + { + const struct tcphdr* tp = (const struct tcphdr*) data; + RecordVal* tcp_hdr = new RecordVal(tcp_hdr_type); + + int tcp_hdr_len = tp->th_off * 4; + int data_len = PayloadLen() - tcp_hdr_len; + + tcp_hdr->Assign(0, new PortVal(ntohs(tp->th_sport), TRANSPORT_TCP)); + tcp_hdr->Assign(1, new PortVal(ntohs(tp->th_dport), TRANSPORT_TCP)); + tcp_hdr->Assign(2, new Val(uint32(ntohl(tp->th_seq)), TYPE_COUNT)); + tcp_hdr->Assign(3, new Val(uint32(ntohl(tp->th_ack)), TYPE_COUNT)); + tcp_hdr->Assign(4, new Val(tcp_hdr_len, TYPE_COUNT)); + tcp_hdr->Assign(5, new Val(data_len, TYPE_COUNT)); + tcp_hdr->Assign(6, new Val(tp->th_flags, TYPE_COUNT)); + tcp_hdr->Assign(7, new Val(ntohs(tp->th_win), TYPE_COUNT)); + + pkt_hdr->Assign(2, tcp_hdr); + break; + } + + case IPPROTO_UDP: + { + const struct udphdr* up = (const struct udphdr*) data; + RecordVal* udp_hdr = new RecordVal(udp_hdr_type); + + udp_hdr->Assign(0, new PortVal(ntohs(up->uh_sport), TRANSPORT_UDP)); + udp_hdr->Assign(1, new PortVal(ntohs(up->uh_dport), TRANSPORT_UDP)); + udp_hdr->Assign(2, new Val(ntohs(up->uh_ulen), TYPE_COUNT)); + + pkt_hdr->Assign(3, udp_hdr); + break; + } + + case IPPROTO_ICMP: + { + const struct icmp* icmpp = (const struct icmp *) data; + RecordVal* icmp_hdr = new RecordVal(icmp_hdr_type); + + icmp_hdr->Assign(0, new Val(icmpp->icmp_type, TYPE_COUNT)); + + pkt_hdr->Assign(4, icmp_hdr); + break; + } + + default: + { + // This is not a protocol we understand. + break; + } + } + + return pkt_hdr; + } + +static inline bool isIPv6ExtHeader(uint8 type) + { + switch (type) { + case IPPROTO_HOPOPTS: + case IPPROTO_ROUTING: + case IPPROTO_DSTOPTS: + case IPPROTO_FRAGMENT: + case IPPROTO_AH: + case IPPROTO_ESP: +#ifdef ENABLE_MOBILE_IPV6 + case IPPROTO_MOBILITY: +#endif + return true; + default: + return false; + } + } + +void IPv6_Hdr_Chain::Init(const struct ip6_hdr* ip6, int total_len, + bool set_next, uint16 next) + { + length = 0; + uint8 current_type, next_type; + next_type = IPPROTO_IPV6; + const u_char* hdrs = (const u_char*) ip6; + + if ( total_len < (int)sizeof(struct ip6_hdr) ) + reporter->InternalError("IPv6_HdrChain::Init with truncated IP header"); + + do + { + // We can't determine a given header's length if there's less than + // two bytes of data available (2nd byte of extension headers is length) + if ( total_len < 2 ) + return; + + current_type = next_type; + IPv6_Hdr* p = new IPv6_Hdr(current_type, hdrs); + + next_type = p->NextHdr(); + uint16 cur_len = p->Length(); + + // If this header is truncated, don't add it to chain, don't go further. + if ( cur_len > total_len ) + { + delete p; + return; + } + + if ( set_next && next_type == IPPROTO_FRAGMENT ) + { + p->ChangeNext(next); + next_type = next; + } + + chain.push_back(p); + + // Check for routing headers and remember final destination address. + if ( current_type == IPPROTO_ROUTING ) + ProcessRoutingHeader((const struct ip6_rthdr*) hdrs, cur_len); + +#ifdef ENABLE_MOBILE_IPV6 + // Only Mobile IPv6 has a destination option we care about right now. + if ( current_type == IPPROTO_DSTOPTS ) + ProcessDstOpts((const struct ip6_dest*) hdrs, cur_len); +#endif + + hdrs += cur_len; + length += cur_len; + total_len -= cur_len; + + } while ( current_type != IPPROTO_FRAGMENT && + current_type != IPPROTO_ESP && +#ifdef ENABLE_MOBILE_IPV6 + current_type != IPPROTO_MOBILITY && +#endif + isIPv6ExtHeader(next_type) ); + } + +void IPv6_Hdr_Chain::ProcessRoutingHeader(const struct ip6_rthdr* r, uint16 len) + { + if ( finalDst ) + { + // RFC 2460 section 4.1 says Routing should occur at most once. + reporter->Weird(SrcAddr(), DstAddr(), "multiple_routing_headers"); + return; + } + + // Last 16 bytes of header (for all known types) is the address we want. + const in6_addr* addr = (const in6_addr*)(((const u_char*)r) + len - 16); + + switch ( r->ip6r_type ) { + case 0: // Defined by RFC 2460, deprecated by RFC 5095 + { + if ( r->ip6r_segleft > 0 && r->ip6r_len >= 2 ) + { + if ( r->ip6r_len % 2 == 0 ) + finalDst = new IPAddr(*addr); + else + reporter->Weird(SrcAddr(), DstAddr(), "odd_routing0_len"); + } + + // Always raise a weird since this type is deprecated. + reporter->Weird(SrcAddr(), DstAddr(), "routing0_hdr"); + } + break; + +#ifdef ENABLE_MOBILE_IPV6 + case 2: // Defined by Mobile IPv6 RFC 6275. + { + if ( r->ip6r_segleft > 0 ) + { + if ( r->ip6r_len == 2 ) + finalDst = new IPAddr(*addr); + else + reporter->Weird(SrcAddr(), DstAddr(), "bad_routing2_len"); + } + } + break; +#endif + + default: + reporter->Weird(fmt("unknown_routing_type_%d", r->ip6r_type)); + break; + } + } + +#ifdef ENABLE_MOBILE_IPV6 +void IPv6_Hdr_Chain::ProcessDstOpts(const struct ip6_dest* d, uint16 len) + { + const u_char* data = (const u_char*) d; + len -= 2 * sizeof(uint8); + data += 2* sizeof(uint8); + + while ( len > 0 ) + { + const struct ip6_opt* opt = (const struct ip6_opt*) data; + switch ( opt->ip6o_type ) { + case 201: // Home Address Option, Mobile IPv6 RFC 6275 section 6.3 + { + if ( opt->ip6o_len == 16 ) + if ( homeAddr ) + reporter->Weird(SrcAddr(), DstAddr(), "multiple_home_addr_opts"); + else + homeAddr = new IPAddr(*((const in6_addr*)(data + 2))); + else + reporter->Weird(SrcAddr(), DstAddr(), "bad_home_addr_len"); + } + break; + + default: + break; + } + + if ( opt->ip6o_type == 0 ) + { + data += sizeof(uint8); + len -= sizeof(uint8); + } + else + { + data += 2 * sizeof(uint8) + opt->ip6o_len; + len -= 2 * sizeof(uint8) + opt->ip6o_len; + } + } + } +#endif + +VectorVal* IPv6_Hdr_Chain::BuildVal() const + { + if ( ! ip6_ext_hdr_type ) + { + ip6_ext_hdr_type = internal_type("ip6_ext_hdr")->AsRecordType(); + ip6_hopopts_type = internal_type("ip6_hopopts")->AsRecordType(); + ip6_dstopts_type = internal_type("ip6_dstopts")->AsRecordType(); + ip6_routing_type = internal_type("ip6_routing")->AsRecordType(); + ip6_fragment_type = internal_type("ip6_fragment")->AsRecordType(); + ip6_ah_type = internal_type("ip6_ah")->AsRecordType(); + ip6_esp_type = internal_type("ip6_esp")->AsRecordType(); + ip6_mob_type = internal_type("ip6_mobility_hdr")->AsRecordType(); + } + + VectorVal* rval = new VectorVal(new VectorType(ip6_ext_hdr_type->Ref())); + + for ( size_t i = 1; i < chain.size(); ++i ) + { + RecordVal* v = chain[i]->BuildRecordVal(); + RecordVal* ext_hdr = new RecordVal(ip6_ext_hdr_type); + uint8 type = chain[i]->Type(); + ext_hdr->Assign(0, new Val(type, TYPE_COUNT)); + + switch (type) { + case IPPROTO_HOPOPTS: + ext_hdr->Assign(1, v); + break; + case IPPROTO_DSTOPTS: + ext_hdr->Assign(2, v); + break; + case IPPROTO_ROUTING: + ext_hdr->Assign(3, v); + break; + case IPPROTO_FRAGMENT: + ext_hdr->Assign(4, v); + break; + case IPPROTO_AH: + ext_hdr->Assign(5, v); + break; + case IPPROTO_ESP: + ext_hdr->Assign(6, v); + break; +#ifdef ENABLE_MOBILE_IPV6 + case IPPROTO_MOBILITY: + ext_hdr->Assign(7, v); + break; +#endif + default: + reporter->InternalError("IPv6_Hdr_Chain bad header %d", type); + break; + } + rval->Assign(rval->Size(), ext_hdr, 0); + } + + return rval; + } diff --git a/src/IP.h b/src/IP.h index 73ac4ee5c7..502ae857c0 100644 --- a/src/IP.h +++ b/src/IP.h @@ -4,67 +4,370 @@ #define ip_h #include "config.h" +#include "net_util.h" +#include "IPAddr.h" +#include "Reporter.h" +#include "Val.h" +#include "Type.h" +#include +#include +#include -#include +#ifdef ENABLE_MOBILE_IPV6 +#ifndef IPPROTO_MOBILITY +#define IPPROTO_MOBILITY 135 +#endif + +struct ip6_mobility { + uint8 ip6mob_payload; + uint8 ip6mob_len; + uint8 ip6mob_type; + uint8 ip6mob_rsv; + uint16 ip6mob_chksum; +}; + +#endif //ENABLE_MOBILE_IPV6 + +/** + * Base class for IPv6 header/extensions. + */ +class IPv6_Hdr { +public: + /** + * Construct an IPv6 header or extension header from assigned type number. + */ + IPv6_Hdr(uint8 t, const u_char* d) : type(t), data(d) {} + + /** + * Replace the value of the next protocol field. + */ + void ChangeNext(uint8 next_type) + { + switch ( type ) { + case IPPROTO_IPV6: + ((ip6_hdr*)data)->ip6_nxt = next_type; + break; + case IPPROTO_HOPOPTS: + case IPPROTO_DSTOPTS: + case IPPROTO_ROUTING: + case IPPROTO_FRAGMENT: + case IPPROTO_AH: +#ifdef ENABLE_MOBILE_IPV6 + case IPPROTO_MOBILITY: +#endif + ((ip6_ext*)data)->ip6e_nxt = next_type; + break; + case IPPROTO_ESP: + default: + break; + } + } + + ~IPv6_Hdr() {} + + /** + * Returns the assigned IPv6 extension header type number of the header + * that immediately follows this one. + */ + uint8 NextHdr() const + { + switch ( type ) { + case IPPROTO_IPV6: + return ((ip6_hdr*)data)->ip6_nxt; + case IPPROTO_HOPOPTS: + case IPPROTO_DSTOPTS: + case IPPROTO_ROUTING: + case IPPROTO_FRAGMENT: + case IPPROTO_AH: +#ifdef ENABLE_MOBILE_IPV6 + case IPPROTO_MOBILITY: +#endif + return ((ip6_ext*)data)->ip6e_nxt; + case IPPROTO_ESP: + default: + return IPPROTO_NONE; + } + } + + /** + * Returns the length of the header in bytes. + */ + uint16 Length() const + { + switch ( type ) { + case IPPROTO_IPV6: + return 40; + case IPPROTO_HOPOPTS: + case IPPROTO_DSTOPTS: + case IPPROTO_ROUTING: +#ifdef ENABLE_MOBILE_IPV6 + case IPPROTO_MOBILITY: +#endif + return 8 + 8 * ((ip6_ext*)data)->ip6e_len; + case IPPROTO_FRAGMENT: + return 8; + case IPPROTO_AH: + return 8 + 4 * ((ip6_ext*)data)->ip6e_len; + case IPPROTO_ESP: + return 8; //encrypted payload begins after 8 bytes + default: + return 0; + } + } + + /** + * Returns the RFC 1700 et seq. IANA assigned number for the header. + */ + uint8 Type() const { return type; } + + /** + * Returns pointer to the start of where header structure resides in memory. + */ + const u_char* Data() const { return data; } + + /** + * Returns the script-layer record representation of the header. + */ + RecordVal* BuildRecordVal(VectorVal* chain = 0) const; + +protected: + uint8 type; + const u_char* data; +}; + +class IPv6_Hdr_Chain { +public: + /** + * Initializes the header chain from an IPv6 header structure. + */ + IPv6_Hdr_Chain(const struct ip6_hdr* ip6, int len) : +#ifdef ENABLE_MOBILE_IPV6 + homeAddr(0), +#endif + finalDst(0) + { Init(ip6, len, false); } + + ~IPv6_Hdr_Chain() + { + for ( size_t i = 0; i < chain.size(); ++i ) delete chain[i]; +#ifdef ENABLE_MOBILE_IPV6 + delete homeAddr; +#endif + delete finalDst; + } + + /** + * Returns the number of headers in the chain. + */ + size_t Size() const { return chain.size(); } + + /** + * Returns the sum of the length of all headers in the chain in bytes. + */ + uint16 TotalLength() const { return length; } + + /** + * Accesses the header at the given location in the chain. + */ + const IPv6_Hdr* operator[](const size_t i) const { return chain[i]; } + + /** + * Returns whether the header chain indicates a fragmented packet. + */ + bool IsFragment() const + { return chain[chain.size()-1]->Type() == IPPROTO_FRAGMENT; } + + /** + * Returns pointer to fragment header structure if the chain contains one. + */ + const struct ip6_frag* GetFragHdr() const + { return IsFragment() ? + (const struct ip6_frag*)chain[chain.size()-1]->Data(): 0; } + + /** + * If the header chain is a fragment, returns the offset in number of bytes + * relative to the start of the Fragmentable Part of the original packet. + */ + uint16 FragOffset() const + { return IsFragment() ? + (ntohs(GetFragHdr()->ip6f_offlg) & 0xfff8) : 0; } + + /** + * If the header chain is a fragment, returns the identification field. + */ + uint32 ID() const + { return IsFragment() ? ntohl(GetFragHdr()->ip6f_ident) : 0; } + + /** + * If the header chain is a fragment, returns the M (more fragments) flag. + */ + int MF() const + { return IsFragment() ? + (ntohs(GetFragHdr()->ip6f_offlg) & 0x0001) != 0 : 0; } + + /** + * If the chain contains a Destination Options header with a Home Address + * option as defined by Mobile IPv6 (RFC 6275), then return it, else + * return the source address in the main IPv6 header. + */ + IPAddr SrcAddr() const + { +#ifdef ENABLE_MOBILE_IPV6 + if ( homeAddr ) + return IPAddr(*homeAddr); + else +#endif + return IPAddr(((const struct ip6_hdr*)(chain[0]->Data()))->ip6_src); + } + + /** + * If the chain contains a Routing header with non-zero segments left, + * then return the last address of the first such header, else return + * the destination address of the main IPv6 header. + */ + IPAddr DstAddr() const + { + if ( finalDst ) + return IPAddr(*finalDst); + else + return IPAddr(((const struct ip6_hdr*)(chain[0]->Data()))->ip6_dst); + } + + /** + * Returns a vector of ip6_ext_hdr RecordVals that includes script-layer + * representation of all extension headers in the chain. + */ + VectorVal* BuildVal() const; + +protected: + // for access to protected ctor that changes next header values that + // point to a fragment + friend class FragReassembler; + + /** + * Initializes the header chain from an IPv6 header structure, and replaces + * the first next protocol pointer field that points to a fragment header. + */ + IPv6_Hdr_Chain(const struct ip6_hdr* ip6, uint16 next, int len) : +#ifdef ENABLE_MOBILE_IPV6 + homeAddr(0), +#endif + finalDst(0) + { Init(ip6, len, true, next); } + + /** + * Initializes the header chain from an IPv6 header structure of a given + * length, possibly setting the first next protocol pointer field that + * points to a fragment header. + */ + void Init(const struct ip6_hdr* ip6, int total_len, bool set_next, + uint16 next = 0); + + /** + * Process a routing header and allocate/remember the final destination + * address if it has segments left and is a valid routing header. + */ + void ProcessRoutingHeader(const struct ip6_rthdr* r, uint16 len); + +#ifdef ENABLE_MOBILE_IPV6 + /** + * Inspect a Destination Option header's options for things we need to + * remember, such as the Home Address option from Mobile IPv6. + */ + void ProcessDstOpts(const struct ip6_dest* d, uint16 len); +#endif + + vector chain; + + /** + * The summation of all header lengths in the chain in bytes. + */ + uint16 length; + +#ifdef ENABLE_MOBILE_IPV6 + /** + * Home Address of the packet's source as defined by Mobile IPv6 (RFC 6275). + */ + IPAddr* homeAddr; +#endif + + /** + * The final destination address in chain's first Routing header that has + * non-zero segments left. + */ + IPAddr* finalDst; +}; + +/** + * A class that wraps either an IPv4 or IPv6 packet and abstracts methods + * for inquiring about common features between the two. + */ class IP_Hdr { public: - IP_Hdr(struct ip* arg_ip4) + /** + * Attempts to construct the header from some blob of data based on IP + * version number. Caller must have already checked that the header + * is not truncated. + * @param p pointer to memory containing an IPv4 or IPv6 packet. + * @param arg_del whether to take ownership of \a p pointer's memory. + * @param len the length of data, in bytes, pointed to by \a p. + */ + IP_Hdr(const u_char* p, bool arg_del, int len) + : ip4(0), ip6(0), del(arg_del), ip6_hdrs(0) { - ip4 = arg_ip4; - ip6 = 0; - del = 1; - -#ifdef BROv6 - src_addr[0] = src_addr[1] = src_addr[2] = 0; - dst_addr[0] = dst_addr[1] = dst_addr[2] = 0; - - src_addr[3] = ip4->ip_src.s_addr; - dst_addr[3] = ip4->ip_dst.s_addr; -#endif + if ( ((const struct ip*)p)->ip_v == 4 ) + ip4 = (const struct ip*)p; + else if ( ((const struct ip*)p)->ip_v == 6 ) + { + ip6 = (const struct ip6_hdr*)p; + ip6_hdrs = new IPv6_Hdr_Chain(ip6, len); + } + else + { + if ( arg_del ) + delete [] p; + reporter->InternalError("bad IP version in IP_Hdr ctor"); + } } - IP_Hdr(const struct ip* arg_ip4) + /** + * Construct the header wrapper from an IPv4 packet. Caller must have + * already checked that the header is not truncated. + * @param arg_ip4 pointer to memory containing an IPv4 packet. + * @param arg_del whether to take ownership of \a arg_ip4 pointer's memory. + */ + IP_Hdr(const struct ip* arg_ip4, bool arg_del) + : ip4(arg_ip4), ip6(0), del(arg_del), ip6_hdrs(0) { - ip4 = arg_ip4; - ip6 = 0; - del = 0; - -#ifdef BROv6 - src_addr[0] = src_addr[1] = src_addr[2] = 0; - dst_addr[0] = dst_addr[1] = dst_addr[2] = 0; - - src_addr[3] = ip4->ip_src.s_addr; - dst_addr[3] = ip4->ip_dst.s_addr; -#endif } - IP_Hdr(struct ip6_hdr* arg_ip6) + /** + * Construct the header wrapper from an IPv6 packet. Caller must have + * already checked that the static IPv6 header is not truncated. If + * the packet contains extension headers and they are truncated, that can + * be checked afterwards by comparing \a len with \a TotalLen. E.g. + * NetSessions::DoNextPacket does this to skip truncated packets. + * @param arg_ip6 pointer to memory containing an IPv6 packet. + * @param arg_del whether to take ownership of \a arg_ip6 pointer's memory. + * @param len the packet's length in bytes. + * @param c an already-constructed header chain to take ownership of. + */ + IP_Hdr(const struct ip6_hdr* arg_ip6, bool arg_del, int len, + const IPv6_Hdr_Chain* c = 0) + : ip4(0), ip6(arg_ip6), del(arg_del), + ip6_hdrs(c ? c : new IPv6_Hdr_Chain(ip6, len)) { - ip4 = 0; - ip6 = arg_ip6; - del = 1; - -#ifdef BROv6 - memcpy(src_addr, ip6->ip6_src.s6_addr, 16); - memcpy(dst_addr, ip6->ip6_dst.s6_addr, 16); -#endif - } - - IP_Hdr(const struct ip6_hdr* arg_ip6) - { - ip4 = 0; - ip6 = arg_ip6; - del = 0; - -#ifdef BROv6 - memcpy(src_addr, ip6->ip6_src.s6_addr, 16); - memcpy(dst_addr, ip6->ip6_dst.s6_addr, 16); -#endif } + /** + * Destructor. + */ ~IP_Hdr() { + if ( ip6 ) + delete ip6_hdrs; + if ( del ) { if ( ip4 ) @@ -74,68 +377,175 @@ public: } } + /** + * If an IPv4 packet is wrapped, return a pointer to it, else null. + */ const struct ip* IP4_Hdr() const { return ip4; } + + /** + * If an IPv6 packet is wrapped, return a pointer to it, else null. + */ const struct ip6_hdr* IP6_Hdr() const { return ip6; } -#ifdef BROv6 - const uint32* SrcAddr() const { return src_addr; } - const uint32* DstAddr() const { return dst_addr; } -#else - const uint32* SrcAddr() const - { return ip4 ? &(ip4->ip_src.s_addr) : 0; } - const uint32* DstAddr() const - { return ip4 ? &(ip4->ip_dst.s_addr) : 0; } -#endif + /** + * Returns the source address held in the IP header. + */ + IPAddr IPHeaderSrcAddr() const + { return ip4 ? IPAddr(ip4->ip_src) : IPAddr(ip6->ip6_src); } - uint32 SrcAddr4() const { return ip4->ip_src.s_addr; } - uint32 DstAddr4() const { return ip4->ip_dst.s_addr; } + /** + * Returns the destination address held in the IP header. + */ + IPAddr IPHeaderDstAddr() const + { return ip4 ? IPAddr(ip4->ip_dst) : IPAddr(ip6->ip6_dst); } - uint16 ID4() const { return ip4 ? ip4->ip_id : 0; } + /** + * For IPv4 or IPv6 headers that don't contain a Home Address option + * (Mobile IPv6, RFC 6275), return source address held in the IP header. + * For IPv6 headers that contain a Home Address option, return that address. + */ + IPAddr SrcAddr() const + { return ip4 ? IPAddr(ip4->ip_src) : ip6_hdrs->SrcAddr(); } + /** + * For IPv4 or IPv6 headers that don't contain a Routing header with + * non-zero segments left, return destination address held in the IP header. + * For IPv6 headers with a Routing header that has non-zero segments left, + * return the last address in the first such Routing header. + */ + IPAddr DstAddr() const + { return ip4 ? IPAddr(ip4->ip_dst) : ip6_hdrs->DstAddr(); } + + /** + * Returns a pointer to the payload of the IP packet, usually an + * upper-layer protocol. + */ const u_char* Payload() const { if ( ip4 ) return ((const u_char*) ip4) + ip4->ip_hl * 4; else - return ((const u_char*) ip6) + 40; + return ((const u_char*) ip6) + ip6_hdrs->TotalLength(); } +#ifdef ENABLE_MOBILE_IPV6 + /** + * Returns a pointer to the mobility header of the IP packet, if present, + * else a null pointer. + */ + const ip6_mobility* MobilityHeader() const + { + if ( ip4 ) + return 0; + else if ( (*ip6_hdrs)[ip6_hdrs->Size()-1]->Type() != IPPROTO_MOBILITY ) + return 0; + else + return (const ip6_mobility*)(*ip6_hdrs)[ip6_hdrs->Size()-1]->Data(); + } +#endif + + /** + * Returns the length of the IP packet's payload (length of packet minus + * header length or, for IPv6, also minus length of all extension headers). + */ uint16 PayloadLen() const { if ( ip4 ) return ntohs(ip4->ip_len) - ip4->ip_hl * 4; else - return ntohs(ip6->ip6_plen); + return ntohs(ip6->ip6_plen) + 40 - ip6_hdrs->TotalLength(); } - uint16 TotalLen() const - { - if ( ip4 ) - return ntohs(ip4->ip_len); - else - return ntohs(ip6->ip6_plen) + 40; - } + /** + * Returns the length of the IP packet (length of headers and payload). + */ + uint32 TotalLen() const + { return ip4 ? ntohs(ip4->ip_len) : ntohs(ip6->ip6_plen) + 40; } - uint16 HdrLen() const { return ip4 ? ip4->ip_hl * 4 : 40; } + /** + * Returns length of IP packet header (includes extension headers for IPv6). + */ + uint16 HdrLen() const + { return ip4 ? ip4->ip_hl * 4 : ip6_hdrs->TotalLength(); } + + /** + * For IPv6 header chains, returns the type of the last header in the chain. + */ + uint8 LastHeader() const + { return ip4 ? IPPROTO_RAW : + ((*ip6_hdrs)[ip6_hdrs->Size()-1])->Type(); } + + /** + * Returns the protocol type of the IP packet's payload, usually an + * upper-layer protocol. For IPv6, this returns the last (extension) + * header's Next Header value. + */ unsigned char NextProto() const - { return ip4 ? ip4->ip_p : ip6->ip6_nxt; } + { return ip4 ? ip4->ip_p : + ((*ip6_hdrs)[ip6_hdrs->Size()-1])->NextHdr(); } + + /** + * Returns the IPv4 Time to Live or IPv6 Hop Limit field. + */ unsigned char TTL() const { return ip4 ? ip4->ip_ttl : ip6->ip6_hlim; } - uint16 FragField() const - { return ntohs(ip4 ? ip4->ip_off : 0); } + + /** + * Returns whether the IP header indicates this packet is a fragment. + */ + bool IsFragment() const + { return ip4 ? (ntohs(ip4->ip_off) & 0x3fff) != 0 : + ip6_hdrs->IsFragment(); } + + /** + * Returns the fragment packet's offset in relation to the original + * packet in bytes. + */ + uint16 FragOffset() const + { return ip4 ? (ntohs(ip4->ip_off) & 0x1fff) * 8 : + ip6_hdrs->FragOffset(); } + + /** + * Returns the fragment packet's identification field. + */ + uint32 ID() const + { return ip4 ? ntohs(ip4->ip_id) : ip6_hdrs->ID(); } + + /** + * Returns whether a fragment packet's "More Fragments" field is set. + */ + int MF() const + { return ip4 ? (ntohs(ip4->ip_off) & 0x2000) != 0 : ip6_hdrs->MF(); } + + /** + * Returns whether a fragment packet's "Don't Fragment" field is set. + * Note that IPv6 has no such field. + */ int DF() const - { return ip4 ? ((ntohs(ip4->ip_off) & IP_DF) != 0) : 0; } - uint16 IP_ID() const - { return ip4 ? (ntohs(ip4->ip_id)) : 0; } + { return ip4 ? ((ntohs(ip4->ip_off) & 0x4000) != 0) : 0; } + + /** + * Returns number of IP headers in packet (includes IPv6 extension headers). + */ + size_t NumHeaders() const + { return ip4 ? 1 : ip6_hdrs->Size(); } + + /** + * Returns an ip_hdr or ip6_hdr_chain RecordVal. + */ + RecordVal* BuildIPHdrVal() const; + + /** + * Returns a pkt_hdr RecordVal, which includes not only the IP header, but + * also upper-layer (tcp/udp/icmp) headers. + */ + RecordVal* BuildPktHdrVal() const; private: const struct ip* ip4; const struct ip6_hdr* ip6; -#ifdef BROv6 - uint32 src_addr[NUM_ADDR_WORDS]; - uint32 dst_addr[NUM_ADDR_WORDS]; -#endif - int del; + bool del; + const IPv6_Hdr_Chain* ip6_hdrs; }; #endif diff --git a/src/IPAddr.cc b/src/IPAddr.cc new file mode 100644 index 0000000000..0ba5589fff --- /dev/null +++ b/src/IPAddr.cc @@ -0,0 +1,286 @@ +// See the file "COPYING" in the main distribution directory for copyright. + +#include +#include +#include "IPAddr.h" +#include "Reporter.h" +#include "Conn.h" +#include "DPM.h" +#include "bro_inet_ntop.h" + +const uint8_t IPAddr::v4_mapped_prefix[12] = { 0, 0, 0, 0, + 0, 0, 0, 0, + 0, 0, 0xff, 0xff }; + +HashKey* BuildConnIDHashKey(const ConnID& id) + { + struct { + in6_addr ip1; + in6_addr ip2; + uint16 port1; + uint16 port2; + } key; + + // Lookup up connection based on canonical ordering, which is + // the smaller of and + // followed by the other. + if ( id.is_one_way || + addr_port_canon_lt(id.src_addr, id.src_port, id.dst_addr, id.dst_port) + ) + { + key.ip1 = id.src_addr.in6; + key.ip2 = id.dst_addr.in6; + key.port1 = id.src_port; + key.port2 = id.dst_port; + } + else + { + key.ip1 = id.dst_addr.in6; + key.ip2 = id.src_addr.in6; + key.port1 = id.dst_port; + key.port2 = id.src_port; + } + + return new HashKey(&key, sizeof(key)); + } + +HashKey* BuildExpectedConnHashKey(const ExpectedConn& c) + { + struct { + in6_addr orig; + in6_addr resp; + uint16 resp_p; + uint16 proto; + } key; + + key.orig = c.orig.in6; + key.resp = c.resp.in6; + key.resp_p = c.resp_p; + key.proto = c.proto; + + return new HashKey(&key, sizeof(key)); + } + +void IPAddr::Mask(int top_bits_to_keep) + { + if ( top_bits_to_keep < 0 || top_bits_to_keep > 128 ) + { + reporter->Error("Bad IPAddr::Mask value %d", top_bits_to_keep); + return; + } + + uint32_t tmp[4]; + memcpy(tmp, in6.s6_addr, sizeof(in6.s6_addr)); + + int word = 3; + int bits_to_chop = 128 - top_bits_to_keep; + + while ( bits_to_chop >= 32 ) + { + tmp[word] = 0; + --word; + bits_to_chop -= 32; + } + + uint32_t w = ntohl(tmp[word]); + w >>= bits_to_chop; + w <<= bits_to_chop; + tmp[word] = htonl(w); + + memcpy(in6.s6_addr, tmp, sizeof(in6.s6_addr)); + } + +void IPAddr::ReverseMask(int top_bits_to_chop) + { + if ( top_bits_to_chop < 0 || top_bits_to_chop > 128 ) + { + reporter->Error("Bad IPAddr::ReverseMask value %d", top_bits_to_chop); + return; + } + + uint32_t tmp[4]; + memcpy(tmp, in6.s6_addr, sizeof(in6.s6_addr)); + + int word = 0; + int bits_to_chop = top_bits_to_chop; + + while ( bits_to_chop >= 32 ) + { + tmp[word] = 0; + ++word; + bits_to_chop -= 32; + } + + uint32_t w = ntohl(tmp[word]); + w <<= bits_to_chop; + w >>= bits_to_chop; + tmp[word] = htonl(w); + + memcpy(in6.s6_addr, tmp, sizeof(in6.s6_addr)); + } + +void IPAddr::Init(const std::string& s) + { + if ( s.find(':') == std::string::npos ) // IPv4. + { + memcpy(in6.s6_addr, v4_mapped_prefix, sizeof(v4_mapped_prefix)); + + // Parse the address directly instead of using inet_pton since + // some platforms have more sensitive implementations than others + // that can't e.g. handle leading zeroes. + int a[4]; + int n = sscanf(s.c_str(), "%d.%d.%d.%d", a+0, a+1, a+2, a+3); + + if ( n != 4 || a[0] < 0 || a[1] < 0 || a[2] < 0 || a[3] < 0 || + a[0] > 255 || a[1] > 255 || a[2] > 255 || a[3] > 255 ) + { + reporter->Error("Bad IP address: %s", s.c_str()); + memset(in6.s6_addr, 0, sizeof(in6.s6_addr)); + return; + } + + uint32_t addr = (a[0] << 24) | (a[1] << 16) | (a[2] << 8) | a[3]; + addr = htonl(addr); + memcpy(&in6.s6_addr[12], &addr, sizeof(uint32_t)); + } + + else + { + if ( inet_pton(AF_INET6, s.c_str(), in6.s6_addr) <=0 ) + { + reporter->Error("Bad IP address: %s", s.c_str()); + memset(in6.s6_addr, 0, sizeof(in6.s6_addr)); + } + } + } + +string IPAddr::AsString() const + { + if ( GetFamily() == IPv4 ) + { + char s[INET_ADDRSTRLEN]; + + if ( ! bro_inet_ntop(AF_INET, &in6.s6_addr[12], s, INET_ADDRSTRLEN) ) + return "> 24) & 0xff; + uint32_t a2 = (a >> 16) & 0xff; + uint32_t a1 = (a >> 8) & 0xff; + uint32_t a0 = a & 0xff; + snprintf(buf, sizeof(buf), "%u.%u.%u.%u.in-addr.arpa", a0, a1, a2, a3); + return buf; + } + else + { + static const char hex_digit[] = "0123456789abcdef"; + string ptr_name("ip6.arpa"); + uint32_t* p = (uint32_t*) in6.s6_addr; + + for ( unsigned int i = 0; i < 4; ++i ) + { + uint32 a = ntohl(p[i]); + for ( unsigned int j = 1; j <=8; ++j ) + { + ptr_name.insert(0, 1, '.'); + ptr_name.insert(0, 1, hex_digit[(a >> (32-j*4)) & 0x0f]); + } + } + + return ptr_name; + } + } + +IPPrefix::IPPrefix(const in4_addr& in4, uint8_t length) + : prefix(in4), length(96 + length) + { + if ( length > 32 ) + reporter->InternalError("Bad in4_addr IPPrefix length : %d", length); + + prefix.Mask(this->length); + } + +IPPrefix::IPPrefix(const in6_addr& in6, uint8_t length) + : prefix(in6), length(length) + { + if ( length > 128 ) + reporter->InternalError("Bad in6_addr IPPrefix length : %d", length); + + prefix.Mask(this->length); + } + +IPPrefix::IPPrefix(const IPAddr& addr, uint8_t length) + : prefix(addr) + { + if ( prefix.GetFamily() == IPv4 ) + { + if ( length > 32 ) + reporter->InternalError("Bad IPAddr(v4) IPPrefix length : %d", + length); + + this->length = length + 96; + } + + else + { + if ( length > 128 ) + reporter->InternalError("Bad IPAddr(v6) IPPrefix length : %d", + length); + + this->length = length; + } + + prefix.Mask(this->length); + } + +string IPPrefix::AsString() const + { + char l[16]; + + if ( prefix.GetFamily() == IPv4 ) + modp_uitoa10(length - 96, l); + else + modp_uitoa10(length, l); + + return prefix.AsString() +"/" + l; + } + diff --git a/src/IPAddr.h b/src/IPAddr.h new file mode 100644 index 0000000000..8e1921e07b --- /dev/null +++ b/src/IPAddr.h @@ -0,0 +1,586 @@ +// See the file "COPYING" in the main distribution directory for copyright. + +#ifndef IPADDR_H +#define IPADDR_H + +#include +#include +#include + +#include "BroString.h" +#include "Hash.h" +#include "util.h" +#include "Type.h" +#include "threading/SerialTypes.h" + +struct ConnID; +class ExpectedConn; + +typedef in_addr in4_addr; + +/** + * Class storing both IPv4 and IPv6 addresses. + */ +class IPAddr +{ +public: + /** + * Address family. + */ + typedef IPFamily Family; + + /** + * Byte order. + */ + enum ByteOrder { Host, Network }; + + /** + * Constructs the unspecified IPv6 address (all 128 bits zeroed). + */ + IPAddr() + { + memset(in6.s6_addr, 0, sizeof(in6.s6_addr)); + } + + /** + * Constructs an address instance from an IPv4 address. + * + * @param in6 The IPv6 address. + */ + explicit IPAddr(const in4_addr& in4) + { + memcpy(in6.s6_addr, v4_mapped_prefix, sizeof(v4_mapped_prefix)); + memcpy(&in6.s6_addr[12], &in4.s_addr, sizeof(in4.s_addr)); + } + + /** + * Constructs an address instance from an IPv6 address. + * + * @param in6 The IPv6 address. + */ + explicit IPAddr(const in6_addr& arg_in6) : in6(arg_in6) { } + + /** + * Constructs an address instance from a string representation. + * + * @param s String containing an IP address as either a dotted IPv4 + * address or a hex IPv6 address. + */ + IPAddr(const std::string& s) + { + Init(s); + } + + /** + * Constructs an address instance from a string representation. + * + * @param s ASCIIZ string containing an IP address as either a + * dotted IPv4 address or a hex IPv6 address. + */ + IPAddr(const char* s) + { + Init(s); + } + + /** + * Constructs an address instance from a string representation. + * + * @param s String containing an IP address as either a dotted IPv4 + * address or a hex IPv6 address. + */ + IPAddr(const BroString& s) + { + Init(s.CheckString()); + } + + /** + * Constructs an address instance from a raw byte representation. + * + * @param family The address family. + * + * @param bytes A pointer to the raw byte representation. This must point + * to 4 bytes if \a family is IPv4, and to 16 bytes if \a family is + * IPv6. + * + * @param order Indicates whether the raw representation pointed to + * by \a bytes is stored in network or host order. + */ + IPAddr(Family family, const uint32_t* bytes, ByteOrder order); + + /** + * Copy constructor. + */ + IPAddr(const IPAddr& other) : in6(other.in6) { }; + + /** + * Destructor. + */ + ~IPAddr() { }; + + /** + * Returns the address' family. + */ + Family GetFamily() const + { + if ( memcmp(in6.s6_addr, v4_mapped_prefix, 12) == 0 ) + return IPv4; + else + return IPv6; + } + + /** + * Returns true if the address represents a loopback device. + */ + bool IsLoopback() const; + + /** + * Returns true if the address represents a multicast address. + */ + bool IsMulticast() const + { + if ( GetFamily() == IPv4 ) + return in6.s6_addr[12] == 224; + else + return in6.s6_addr[0] == 0xff; + } + + /** + * Returns true if the address represents a broadcast address. + */ + bool IsBroadcast() const + { + if ( GetFamily() == IPv4 ) + return ((in6.s6_addr[12] == 0xff) && (in6.s6_addr[13] == 0xff) + && (in6.s6_addr[14] == 0xff) && (in6.s6_addr[15] == 0xff)); + else + return false; + } + + /** + * Retrieves the raw byte representation of the address. + * + * @param bytes The pointer to which \a bytes points will be set to + * the address of the raw representation in network-byte order. + * The return value indicates how many 32-bit words are valid starting at + * that address. The pointer will be valid as long as the address instance + * exists. + * + * @return The number of 32-bit words the raw representation uses. This + * will be 1 for an IPv4 address and 4 for an IPv6 address. + */ + int GetBytes(const uint32_t** bytes) const + { + if ( GetFamily() == IPv4 ) + { + *bytes = (uint32_t*) &in6.s6_addr[12]; + return 1; + } + else + { + *bytes = (uint32_t*) in6.s6_addr; + return 4; + } + } + + /** + * Retrieves a copy of the IPv6 raw byte representation of the address. + * If the internal address is IPv4, then the copied bytes use the + * IPv4 to IPv6 address mapping to return a full 16 bytes. + * + * @param bytes The pointer to a memory location in which the + * raw bytes of the address are to be copied in network byte-order. + */ + void CopyIPv6(uint32_t* bytes) const + { + memcpy(bytes, in6.s6_addr, sizeof(in6.s6_addr)); + } + + /** + * Retrieves a copy of the IPv6 raw byte representation of the address. + * @see CopyIPv6(uint32_t) + */ + void CopyIPv6(in6_addr* arg_in6) const + { + memcpy(arg_in6->s6_addr, in6.s6_addr, sizeof(in6.s6_addr)); + } + + /** + * Retrieves a copy of the IPv4 raw byte representation of the address. + * The caller should verify the address is of the IPv4 family type + * beforehand. @see GetFamily(). + * + * @param in4 The pointer to a memory location in which the raw bytes + * of the address are to be copied in network byte-order. + */ + void CopyIPv4(in4_addr* in4) const + { + memcpy(&in4->s_addr, &in6.s6_addr[12], sizeof(in4->s_addr)); + } + + /** + * Returns a key that can be used to lookup the IP Address in a hash + * table. Passes ownership to caller. + */ + HashKey* GetHashKey() const + { + return new HashKey((void*)in6.s6_addr, sizeof(in6.s6_addr)); + } + + /** + * Masks out lower bits of the address. + * + * @param top_bits_to_keep The number of bits \a not to mask out, + * counting from the highest order bit. The value is always + * interpreted relative to the IPv6 bit width, even if the address + * is IPv4. That means if compute ``192.168.1.2/16``, you need to + * pass in 112 (i.e., 96 + 16). The value must be in the range from + * 0 to 128. + */ + void Mask(int top_bits_to_keep); + + /** + * Masks out top bits of the address. + * + * @param top_bits_to_chop The number of bits to mask out, counting + * from the highest order bit. The value is always interpreted relative + * to the IPv6 bit width, even if the address is IPv4. So to mask out + * the first 16 bits of an IPv4 address, pass in 112 (i.e., 96 + 16). + * The value must be in the range from 0 to 128. + */ + void ReverseMask(int top_bits_to_chop); + + /** + * Assignment operator. + */ + IPAddr& operator=(const IPAddr& other) + { + // No self-assignment check here because it's correct without it and + // makes the common case faster. + in6 = other.in6; + return *this; + } + + /** + * Bitwise OR operator returns the IP address resulting from the bitwise + * OR operation on the raw bytes of this address with another. + */ + IPAddr operator|(const IPAddr& other) + { + in6_addr result; + for ( int i = 0; i < 16; ++i ) + result.s6_addr[i] = this->in6.s6_addr[i] | other.in6.s6_addr[i]; + + return IPAddr(result); + } + + /** + * Returns a string representation of the address. IPv4 addresses + * will be returned in dotted representation, IPv6 addresses in + * compressed hex. + */ + string AsString() const; + + /** + * Returns a host-order, plain hex string representation of the address. + */ + string AsHexString() const; + + /** + * Returns a string representation of the address. This returns the + * same as AsString(). + */ + operator std::string() const { return AsString(); } + + /** + * Returns a reverse pointer name associated with the IP address. + * For example, 192.168.0.1's reverse pointer is 1.0.168.192.in-addr.arpa. + */ + string PtrName() const; + + /** + * Comparison operator for IP address. + */ + friend bool operator==(const IPAddr& addr1, const IPAddr& addr2) + { + return memcmp(&addr1.in6, &addr2.in6, sizeof(in6_addr)) == 0; + } + + friend bool operator!=(const IPAddr& addr1, const IPAddr& addr2) + { + return ! (addr1 == addr2); + } + + /** + * Comparison operator IP addresses. This defines a well-defined order for + * IP addresses. However, the order does not necessarily correspond to + * their numerical values. + */ + friend bool operator<(const IPAddr& addr1, const IPAddr& addr2) + { + return memcmp(&addr1.in6, &addr2.in6, sizeof(in6_addr)) < 0; + } + + /** Converts the address into the type used internally by the + * inter-thread communication. + */ + void ConvertToThreadingValue(threading::Value::addr_t* v) const; + + friend HashKey* BuildConnIDHashKey(const ConnID& id); + friend HashKey* BuildExpectedConnHashKey(const ExpectedConn& c); + + unsigned int MemoryAllocation() const { return padded_sizeof(*this); } + +private: + friend class IPPrefix; + + /** + * Initializes an address instance from a string representation. + * + * @param s String containing an IP address as either a dotted IPv4 + * address or a hex IPv6 address. + */ + void Init(const std::string& s); + + in6_addr in6; // IPv6 or v4-to-v6-mapped address + + static const uint8_t v4_mapped_prefix[12]; // top 96 bits of v4-mapped-addr +}; + +inline IPAddr::IPAddr(Family family, const uint32_t* bytes, ByteOrder order) + { + if ( family == IPv4 ) + { + memcpy(in6.s6_addr, v4_mapped_prefix, sizeof(v4_mapped_prefix)); + memcpy(&in6.s6_addr[12], bytes, sizeof(uint32_t)); + + if ( order == Host ) + { + uint32_t* p = (uint32_t*) &in6.s6_addr[12]; + *p = htonl(*p); + } + } + + else + { + memcpy(in6.s6_addr, bytes, sizeof(in6.s6_addr)); + + if ( order == Host ) + { + for ( unsigned int i = 0; i < 4; ++ i) + { + uint32_t* p = (uint32_t*) &in6.s6_addr[i*4]; + *p = htonl(*p); + } + } + } + } + +inline bool IPAddr::IsLoopback() const + { + if ( GetFamily() == IPv4 ) + return in6.s6_addr[12] == 127; + + else + return ((in6.s6_addr[0] == 0) && (in6.s6_addr[1] == 0) + && (in6.s6_addr[2] == 0) && (in6.s6_addr[3] == 0) + && (in6.s6_addr[4] == 0) && (in6.s6_addr[5] == 0) + && (in6.s6_addr[6] == 0) && (in6.s6_addr[7] == 0) + && (in6.s6_addr[8] == 0) && (in6.s6_addr[9] == 0) + && (in6.s6_addr[10] == 0) && (in6.s6_addr[11] == 0) + && (in6.s6_addr[12] == 0) && (in6.s6_addr[13] == 0) + && (in6.s6_addr[14] == 0) && (in6.s6_addr[15] == 1)); + } + +inline void IPAddr::ConvertToThreadingValue(threading::Value::addr_t* v) const + { + v->family = GetFamily(); + + switch ( v->family ) { + + case IPv4: + CopyIPv4(&v->in.in4); + return; + + case IPv6: + CopyIPv6(&v->in.in6); + return; + + // Can't be reached. + abort(); + } + } + +/** + * Returns a hash key for a given ConnID. Passes ownership to caller. + */ +HashKey* BuildConnIDHashKey(const ConnID& id); + +/** + * Returns a hash key for a given ExpectedConn instance. Passes ownership to caller. + */ +HashKey* BuildExpectedConnHashKey(const ExpectedConn& c); + +/** + * Class storing both IPv4 and IPv6 prefixes + * (i.e., \c 192.168.1.1/16 and \c FD00::/8. + */ +class IPPrefix +{ +public: + + /** + * Constructs a prefix 0/0. + */ + IPPrefix() : length(0) {} + + /** + * Constructs a prefix instance from an IPv4 address and a prefix + * length. + * + * @param in4 The IPv4 address. + * + * @param length The prefix length in the range from 0 to 32. + */ + IPPrefix(const in4_addr& in4, uint8_t length); + + /** + * Constructs a prefix instance from an IPv6 address and a prefix + * length. + * + * @param in6 The IPv6 address. + * + * @param length The prefix length in the range from 0 to 128. + */ + IPPrefix(const in6_addr& in6, uint8_t length); + + /** + * Constructs a prefix instance from an IPAddr object and prefix length. + * + * @param addr The IP address. + * + * @param length The prefix length in the range from 0 to 128 + */ + IPPrefix(const IPAddr& addr, uint8_t length); + + /** + * Copy constructor. + */ + IPPrefix(const IPPrefix& other) + : prefix(other.prefix), length(other.length) { } + + /** + * Destructor. + */ + ~IPPrefix() { } + + /** + * Returns the prefix in the form of an IP address. The address will + * have all bits not part of the prefixed set to zero. + */ + const IPAddr& Prefix() const { return prefix; } + + /** + * Returns the bit length of the prefix, relative to the 32 bits + * of an IPv4 prefix or relative to the 128 bits of an IPv6 prefix. + */ + uint8_t Length() const + { + return prefix.GetFamily() == IPv4 ? length - 96 : length; + } + + /** + * Returns the bit length of the prefix always relative to a full + * 128 bits of an IPv6 prefix (or IPv4 mapped to IPv6). + */ + uint8_t LengthIPv6() const { return length; } + + /** Returns true if the given address is part of the prefix. + * + * @param addr The address to test. + */ + bool Contains(const IPAddr& addr) const + { + IPAddr p(addr); + p.Mask(length); + return p == prefix; + } + /** + * Assignment operator. + */ + IPPrefix& operator=(const IPPrefix& other) + { + // No self-assignment check here because it's correct without it and + // makes the common case faster. + prefix = other.prefix; + length = other.length; + return *this; + } + + /** + * Returns a string representation of the prefix. IPv4 addresses + * will be returned in dotted representation, IPv6 addresses in + * compressed hex. + */ + string AsString() const; + + operator std::string() const { return AsString(); } + + /** + * Returns a key that can be used to lookup the IP Prefix in a hash + * table. Passes ownership to caller. + */ + HashKey* GetHashKey() const + { + struct { + in6_addr ip; + uint32 len; + } key; + + key.ip = prefix.in6; + key.len = Length(); + + return new HashKey(&key, sizeof(key)); + } + + /** Converts the prefix into the type used internally by the + * inter-thread communication. + */ + void ConvertToThreadingValue(threading::Value::subnet_t* v) const + { + v->length = length; + prefix.ConvertToThreadingValue(&v->prefix); + } + + unsigned int MemoryAllocation() const { return padded_sizeof(*this); } + + /** + * Comparison operator for IP prefix. + */ + friend bool operator==(const IPPrefix& net1, const IPPrefix& net2) + { + return net1.Prefix() == net2.Prefix() && net1.Length() == net2.Length(); + } + + /** + * Comparison operator IP prefixes. This defines a well-defined order for + * IP prefix. However, the order does not necessarily corresponding to their + * numerical values. + */ + friend bool operator<(const IPPrefix& net1, const IPPrefix& net2) + { + if ( net1.Prefix() < net2.Prefix() ) + return true; + + else if ( net1.Prefix() == net2.Prefix() ) + return net1.Length() < net2.Length(); + + else + return false; + } + +private: + IPAddr prefix; // We store it as an address with the non-prefix bits masked out via Mask(). + uint8_t length; // The bit length of the prefix relative to full IPv6 addr. +}; + +#endif diff --git a/src/LogMgr.h b/src/LogMgr.h deleted file mode 100644 index 3eaba360d5..0000000000 --- a/src/LogMgr.h +++ /dev/null @@ -1,142 +0,0 @@ -// See the file "COPYING" in the main distribution directory for copyright. -// -// A class managing log writers and filters. - -#ifndef LOGMGR_H -#define LOGMGR_H - -#include "Val.h" -#include "EventHandler.h" -#include "RemoteSerializer.h" - -class SerializationFormat; - -// Description of a log field. -struct LogField { - string name; - TypeTag type; - // inner type of sets - TypeTag subtype; - - LogField() { subtype = TYPE_VOID; } - LogField(const LogField& other) - : name(other.name), type(other.type), subtype(other.subtype) { } - - // (Un-)serialize. - bool Read(SerializationFormat* fmt); - bool Write(SerializationFormat* fmt) const; -}; - -// Values as logged by a writer. -struct LogVal { - TypeTag type; - bool present; // False for unset fields. - - // The following union is a subset of BroValUnion, including only the - // types we can log directly. - struct set_t { bro_int_t size; LogVal** vals; }; - typedef set_t vec_t; - - union _val { - bro_int_t int_val; - bro_uint_t uint_val; - uint32 addr_val[NUM_ADDR_WORDS]; - subnet_type subnet_val; - double double_val; - string* string_val; - set_t set_val; - vec_t vector_val; - } val; - - LogVal(TypeTag arg_type = TYPE_ERROR, bool arg_present = true) - : type(arg_type), present(arg_present) {} - ~LogVal(); - - // (Un-)serialize. - bool Read(SerializationFormat* fmt); - bool Write(SerializationFormat* fmt) const; - - // Returns true if the type can be logged the framework. If - // `atomic_only` is true, will not permit composite types. - static bool IsCompatibleType(BroType* t, bool atomic_only=false); - -private: - LogVal(const LogVal& other) { } -}; - -class LogWriter; -class RemoteSerializer; -class RotationTimer; - -class LogMgr { -public: - LogMgr(); - ~LogMgr(); - - // These correspond to the BiFs visible on the scripting layer. The - // actual BiFs just forward here. - bool CreateStream(EnumVal* id, RecordVal* stream); - bool EnableStream(EnumVal* id); - bool DisableStream(EnumVal* id); - bool AddFilter(EnumVal* id, RecordVal* filter); - bool RemoveFilter(EnumVal* id, StringVal* name); - bool RemoveFilter(EnumVal* id, string name); - bool Write(EnumVal* id, RecordVal* columns); - bool SetBuf(EnumVal* id, bool enabled); // Adjusts all writers. - bool Flush(EnumVal* id); // Flushes all writers.. - -protected: - friend class LogWriter; - friend class RemoteSerializer; - friend class RotationTimer; - - //// Function also used by the RemoteSerializer. - - // Takes ownership of fields. - LogWriter* CreateWriter(EnumVal* id, EnumVal* writer, string path, - int num_fields, LogField** fields); - - // Takes ownership of values.. - bool Write(EnumVal* id, EnumVal* writer, string path, - int num_fields, LogVal** vals); - - // Announces all instantiated writers to peer. - void SendAllWritersTo(RemoteSerializer::PeerID peer); - - //// Functions safe to use by writers. - - // Signals that a file has been rotated. - bool FinishedRotation(LogWriter* writer, string new_name, string old_name, - double open, double close, bool terminating); - - // Reports an error for the given writer. - void Error(LogWriter* writer, const char* msg); - - // Deletes the values as passed into Write(). - void DeleteVals(int num_fields, LogVal** vals); - -private: - struct Filter; - struct Stream; - struct WriterInfo; - - bool TraverseRecord(Stream* stream, Filter* filter, RecordType* rt, - TableVal* include, TableVal* exclude, string path, list indices); - - LogVal** RecordToFilterVals(Stream* stream, Filter* filter, - RecordVal* columns); - - LogVal* ValToLogVal(Val* val, BroType* ty = 0); - Stream* FindStream(EnumVal* id); - void RemoveDisabledWriters(Stream* stream); - void InstallRotationTimer(WriterInfo* winfo); - void Rotate(WriterInfo* info); - Filter* FindFilter(EnumVal* id, StringVal* filter); - WriterInfo* FindWriter(LogWriter* writer); - - vector streams; // Indexed by stream enum. -}; - -extern LogMgr* log_mgr; - -#endif diff --git a/src/LogWriter.cc b/src/LogWriter.cc deleted file mode 100644 index 8584a0b0b5..0000000000 --- a/src/LogWriter.cc +++ /dev/null @@ -1,158 +0,0 @@ -// See the file "COPYING" in the main distribution directory for copyright. - -#include "util.h" -#include "LogWriter.h" - -LogWriter::LogWriter() - { - buf = 0; - buf_len = 1024; - buffering = true; - disabled = false; - } - -LogWriter::~LogWriter() - { - if ( buf ) - free(buf); - - for(int i = 0; i < num_fields; ++i) - delete fields[i]; - - delete [] fields; - } - -bool LogWriter::Init(string arg_path, int arg_num_fields, - const LogField* const * arg_fields) - { - path = arg_path; - num_fields = arg_num_fields; - fields = arg_fields; - - if ( ! DoInit(arg_path, arg_num_fields, arg_fields) ) - { - disabled = true; - return false; - } - - return true; - } - -bool LogWriter::Write(int arg_num_fields, LogVal** vals) - { - // Double-check that the arguments match. If we get this from remote, - // something might be mixed up. - if ( num_fields != arg_num_fields ) - { - DBG_LOG(DBG_LOGGING, "Number of fields don't match in LogWriter::Write() (%d vs. %d)", - arg_num_fields, num_fields); - - DeleteVals(vals); - return false; - } - - for ( int i = 0; i < num_fields; ++i ) - { - if ( vals[i]->type != fields[i]->type ) - { - DBG_LOG(DBG_LOGGING, "Field type doesn't match in LogWriter::Write() (%d vs. %d)", - vals[i]->type, fields[i]->type); - DeleteVals(vals); - return false; - } - } - - bool result = DoWrite(num_fields, fields, vals); - - DeleteVals(vals); - - if ( ! result ) - disabled = true; - - return result; - } - -bool LogWriter::SetBuf(bool enabled) - { - if ( enabled == buffering ) - // No change. - return true; - - buffering = enabled; - - if ( ! DoSetBuf(enabled) ) - { - disabled = true; - return false; - } - - return true; - } - -bool LogWriter::Rotate(string rotated_path, double open, - double close, bool terminating) - { - if ( ! DoRotate(rotated_path, open, close, terminating) ) - { - disabled = true; - return false; - } - - return true; - } - -bool LogWriter::Flush() - { - if ( ! DoFlush() ) - { - disabled = true; - return false; - } - - return true; - } - -void LogWriter::Finish() - { - DoFinish(); - } - -const char* LogWriter::Fmt(const char* format, ...) - { - if ( ! buf ) - buf = (char*) malloc(buf_len); - - va_list al; - va_start(al, format); - int n = safe_vsnprintf(buf, buf_len, format, al); - va_end(al); - - if ( (unsigned int) n >= buf_len ) - { // Not enough room, grow the buffer. - buf_len = n + 32; - buf = (char*) realloc(buf, buf_len); - - // Is it portable to restart? - va_start(al, format); - n = safe_vsnprintf(buf, buf_len, format, al); - va_end(al); - } - - return buf; - } - -void LogWriter::Error(const char *msg) - { - log_mgr->Error(this, msg); - } - -void LogWriter::DeleteVals(LogVal** vals) - { - log_mgr->DeleteVals(num_fields, vals); - } - -bool LogWriter::FinishedRotation(string new_name, string old_name, double open, - double close, bool terminating) - { - return log_mgr->FinishedRotation(this, new_name, old_name, open, close, terminating); - } diff --git a/src/LogWriter.h b/src/LogWriter.h deleted file mode 100644 index 1d2f9fa4b2..0000000000 --- a/src/LogWriter.h +++ /dev/null @@ -1,192 +0,0 @@ -// See the file "COPYING" in the main distribution directory for copyright. -// -// Interface API for a log writer backend. The LogMgr creates a separate -// writer instance of pair of (writer type, output path). -// -// Note thay classes derived from LogWriter must be fully thread-safe and not -// use any non-thread-safe Bro functionality (which includes almost -// everything ...). In particular, do not use fmt() but LogWriter::Fmt()!. -// -// The one exception to this rule is the constructor: it is guaranteed to be -// executed inside the main thread and can thus in particular access global -// script variables. - -#ifndef LOGWRITER_H -#define LOGWRITER_H - -#include "LogMgr.h" -#include "BroString.h" - -class LogWriter { -public: - LogWriter(); - virtual ~LogWriter(); - - //// Interface methods to interact with the writer. Note that these - //// methods are not necessarily thread-safe and must be called only - //// from the main thread (which will typically mean only from the - //// LogMgr). In particular, they must not be called from the - //// writer's derived implementation. - - // One-time initialization of the writer to define the logged fields. - // Interpretation of "path" is left to the writer, and will be - // corresponding the value configured on the script-level. - // - // Returns false if an error occured, in which case the writer must - // not be used further. - // - // The new instance takes ownership of "fields", and will delete them - // when done. - bool Init(string path, int num_fields, const LogField* const * fields); - - // Writes one log entry. The method takes ownership of "vals" and - // will return immediately after queueing the write request, which is - // potentially before output has actually been written out. - // - // num_fields and the types of the LogVals must match what was passed - // to Init(). - // - // Returns false if an error occured, in which case the writer must - // not be used any further. - bool Write(int num_fields, LogVal** vals); - - // Sets the buffering status for the writer, if the writer supports - // that. (If not, it will be ignored). - bool SetBuf(bool enabled); - - // Flushes any currently buffered output, if the writer supports - // that. (If not, it will be ignored). - bool Flush(); - - // Triggers rotation, if the writer supports that. (If not, it will - // be ignored). - bool Rotate(string rotated_path, double open, double close, bool terminating); - - // Finishes writing to this logger regularly. Must not be called if - // an error has been indicated earlier. After calling this, no - // further writing must be performed. - void Finish(); - - //// Thread-safe methods that may be called from the writer - //// implementation. - - // The following methods return the information as passed to Init(). - const string Path() const { return path; } - int NumFields() const { return num_fields; } - const LogField* const * Fields() const { return fields; } - -protected: - // Methods for writers to override. If any of these returs false, it - // will be assumed that a fatal error has occured that prevents the - // writer from further operation. It will then be disabled and - // deleted. When return false, the writer should also report the - // error via Error(). Note that even if a writer does not support the - // functionality for one these methods (like rotation), it must still - // return true if that is not to be considered a fatal error. - // - // Called once for initialization of the writer. - virtual bool DoInit(string path, int num_fields, - const LogField* const * fields) = 0; - - // Called once per log entry to record. - virtual bool DoWrite(int num_fields, const LogField* const * fields, - LogVal** vals) = 0; - - // Called when the buffering status for this writer is changed. If - // buffering is disabled, the writer should attempt to write out - // information as quickly as possible even if doing so may have a - // performance impact. If enabled (which is the default), it may - // buffer data as helpful and write it out later in a way optimized - // for performance. The current buffering state can be queried via - // IsBuf(). - // - // A writer may ignore buffering changes if it doesn't fit with its - // semantics (but must still return true in that case). - virtual bool DoSetBuf(bool enabled) = 0; - - // Called to flush any currently buffered output. - // - // A writer may ignore flush requests if it doesn't fit with its - // semantics (but must still return true in that case). - virtual bool DoFlush() = 0; - - // Called when a log output is to be rotated. Most directly this only - // applies to writers writing into files, which should then close the - // current file and open a new one. However, a writer may also - // trigger other apppropiate actions if semantics are similar. - // - // Once rotation has finished, the implementation should call - // RotationDone() to signal the log manager that potential - // postprocessors can now run. - // - // "rotate_path" reflects the path to where the rotated output is to - // be moved, with specifics depending on the writer. It should - // generally be interpreted in a way consistent with that of "path" - // as passed into DoInit(). As an example, for file-based output, - // "rotate_path" could be the original filename extended with a - // timestamp indicating the time of the rotation. - // - // "open" and "close" are the network time's when the *current* file - // was opened and closed, respectively. - // - // "terminating" indicated whether the rotation request occurs due - // the main Bro prcoess terminating (and not because we've reach a - // regularly scheduled time for rotation). - // - // A writer may ignore rotation requests if it doesn't fit with its - // semantics (but must still return true in that case). - virtual bool DoRotate(string rotated_path, double open, double close, - bool terminating) = 0; - - // Called once on termination. Not called when any of the other - // methods has previously signaled an error, i.e., executing this - // method signals a regular shutdown of the writer. - virtual void DoFinish() = 0; - - //// Methods for writers to use. These are thread-safe. - - // A thread-safe version of fmt(). - const char* Fmt(const char* format, ...); - - // Returns the current buffering state. - bool IsBuf() { return buffering; } - - // Reports an error to the user. - void Error(const char *msg); - - // Signals to the log manager that a file has been rotated. - // - // new_name: The filename of the rotated file. old_name: The filename - // of the origina file. - // - // open/close: The timestamps when the original file was opened and - // closed, respectively. - // - // terminating: True if rotation request occured due to the main Bro - // process shutting down. - bool FinishedRotation(string new_name, string old_name, double open, - double close, bool terminating); - -private: - friend class LogMgr; - - // When an error occurs, we call this method to set a flag marking - // the writer as disabled. The LogMgr will check the flag later and - // remove the writer. - bool Disabled() { return disabled; } - - // Deletes the values as passed into Write(). - void DeleteVals(LogVal** vals); - - string path; - int num_fields; - const LogField* const * fields; - bool buffering; - bool disabled; - - // For implementing Fmt(). - char* buf; - unsigned int buf_len; -}; - -#endif diff --git a/src/LogWriterNone.h b/src/LogWriterNone.h deleted file mode 100644 index 3811a19469..0000000000 --- a/src/LogWriterNone.h +++ /dev/null @@ -1,30 +0,0 @@ -// See the file "COPYING" in the main distribution directory for copyright. -// -// Dummy log writer that just discards everything (but still pretends to rotate). - -#ifndef LOGWRITERNONE_H -#define LOGWRITERNONE_H - -#include "LogWriter.h" - -class LogWriterNone : public LogWriter { -public: - LogWriterNone() {} - ~LogWriterNone() {}; - - static LogWriter* Instantiate() { return new LogWriterNone; } - -protected: - virtual bool DoInit(string path, int num_fields, - const LogField* const * fields) { return true; } - - virtual bool DoWrite(int num_fields, const LogField* const * fields, - LogVal** vals) { return true; } - virtual bool DoSetBuf(bool enabled) { return true; } - virtual bool DoRotate(string rotated_path, double open, double close, - bool terminating); - virtual bool DoFlush() { return true; } - virtual void DoFinish() {} -}; - -#endif diff --git a/src/Login.cc b/src/Login.cc index 56efd12f53..e626fb3a0a 100644 --- a/src/Login.cc +++ b/src/Login.cc @@ -38,7 +38,7 @@ Login_Analyzer::Login_Analyzer(AnalyzerTag::Tag tag, Connection* conn) if ( ! re_skip_authentication ) { -#ifdef USE_PERFTOOLS +#ifdef USE_PERFTOOLS_DEBUG HeapLeakChecker::Disabler disabler; #endif re_skip_authentication = init_RE(skip_authentication); diff --git a/src/NCP.cc b/src/NCP.cc index 83378a09a7..edd882747c 100644 --- a/src/NCP.cc +++ b/src/NCP.cc @@ -225,5 +225,7 @@ NCP_Analyzer::NCP_Analyzer(Connection* conn) NCP_Analyzer::~NCP_Analyzer() { delete session; + delete o_ncp; + delete r_ncp; } diff --git a/src/Net.cc b/src/Net.cc index 2d8ee85353..5bfae2275b 100644 --- a/src/Net.cc +++ b/src/Net.cc @@ -42,7 +42,6 @@ extern int select(int, fd_set *, fd_set *, fd_set *, struct timeval *); PList(PktSrc) pkt_srcs; // FIXME: We should really merge PktDumper and PacketDumper. -// It's on my to-do [Robin]. PktDumper* pkt_dumper = 0; int reading_live = 0; @@ -143,7 +142,7 @@ RETSIGTYPE watchdog(int /* signo */) return RETSIGVAL; } -void net_init(name_list& interfaces, name_list& readfiles, +void net_init(name_list& interfaces, name_list& readfiles, name_list& netflows, name_list& flowfiles, const char* writefile, const char* filter, const char* secondary_filter, int do_watchdog) @@ -248,12 +247,14 @@ void net_init(name_list& interfaces, name_list& readfiles, FlowSocketSrc* fs = new FlowSocketSrc(netflows[i]); if ( ! fs->IsOpen() ) + { reporter->Error("%s: problem with netflow socket %s - %s\n", prog, netflows[i], fs->ErrorMsg()); - else - { - io_sources.Register(fs); + delete fs; } + + else + io_sources.Register(fs); } } @@ -454,6 +455,7 @@ void net_run() // date on timers and events. network_time = ct; expire_timers(); + usleep(1); // Just yield. } } @@ -483,6 +485,8 @@ void net_run() // since Bro timers are not high-precision anyway.) if ( ! using_communication ) usleep(100000); + else + usleep(1000); // Flawfinder says about usleep: // diff --git a/src/NetVar.cc b/src/NetVar.cc index 5aed213508..59cc1cc633 100644 --- a/src/NetVar.cc +++ b/src/NetVar.cc @@ -47,15 +47,6 @@ int tcp_max_initial_window; int tcp_max_above_hole_without_any_acks; int tcp_excessive_data_without_further_acks; -int ssl_compare_cipherspecs; -int ssl_analyze_certificates; -int ssl_store_certificates; -int ssl_verify_certificates; -int ssl_store_key_material; -int ssl_max_cipherspec_size; -StringVal* ssl_store_cert_path; -StringVal* x509_trusted_cert_path; -TableType* cipher_suites_list; RecordType* x509_type; double non_analyzed_lifetime; @@ -192,8 +183,6 @@ StringVal* ssl_ca_certificate; StringVal* ssl_private_key; StringVal* ssl_passphrase; -StringVal* x509_crl_file; - Val* profiling_file; double profiling_interval; int expensive_profiling_multiple; @@ -213,11 +202,6 @@ int sig_max_group_size; int enable_syslog; -int use_connection_compressor; -int cc_handle_resets; -int cc_handle_only_syns; -int cc_instantiate_on_data; - TableType* irc_join_list; RecordType* irc_join_info; TableVal* irc_servers; @@ -358,17 +342,7 @@ void init_net_var() tcp_excessive_data_without_further_acks = opt_internal_int("tcp_excessive_data_without_further_acks"); - ssl_compare_cipherspecs = opt_internal_int("ssl_compare_cipherspecs"); - ssl_analyze_certificates = opt_internal_int("ssl_analyze_certificates"); - ssl_store_certificates = opt_internal_int("ssl_store_certificates"); - ssl_verify_certificates = opt_internal_int("ssl_verify_certificates"); - ssl_store_key_material = opt_internal_int("ssl_store_key_material"); - ssl_max_cipherspec_size = opt_internal_int("ssl_max_cipherspec_size"); - - x509_trusted_cert_path = opt_internal_string("X509_trusted_cert_path"); - ssl_store_cert_path = opt_internal_string("ssl_store_cert_path"); x509_type = internal_type("X509")->AsRecordType(); - x509_crl_file = opt_internal_string("X509_crl_file"); non_analyzed_lifetime = opt_internal_double("non_analyzed_lifetime"); tcp_inactivity_timeout = opt_internal_double("tcp_inactivity_timeout"); @@ -525,12 +499,6 @@ void init_net_var() gap_report_freq = opt_internal_double("gap_report_freq"); - use_connection_compressor = - opt_internal_int("use_connection_compressor"); - cc_handle_resets = opt_internal_int("cc_handle_resets"); - cc_handle_only_syns = opt_internal_int("cc_handle_only_syns"); - cc_instantiate_on_data = opt_internal_int("cc_instantiate_on_data"); - irc_join_info = internal_type("irc_join_info")->AsRecordType(); irc_join_list = internal_type("irc_join_list")->AsTableType(); irc_servers = internal_val("irc_servers")->AsTableVal(); diff --git a/src/NetVar.h b/src/NetVar.h index 4a513a8a53..425ea93e09 100644 --- a/src/NetVar.h +++ b/src/NetVar.h @@ -50,17 +50,7 @@ extern int tcp_max_initial_window; extern int tcp_max_above_hole_without_any_acks; extern int tcp_excessive_data_without_further_acks; -// see policy/ssl.bro for details -extern int ssl_compare_cipherspecs; -extern int ssl_analyze_certificates; -extern int ssl_store_certificates; -extern int ssl_verify_certificates; -extern int ssl_store_key_material; -extern int ssl_max_cipherspec_size; -extern StringVal* ssl_store_cert_path; -extern StringVal* x509_trusted_cert_path; extern RecordType* x509_type; -extern StringVal* x509_crl_file; extern double non_analyzed_lifetime; extern double tcp_inactivity_timeout; @@ -216,11 +206,6 @@ extern int sig_max_group_size; extern int enable_syslog; -extern int use_connection_compressor; -extern int cc_handle_resets; -extern int cc_handle_only_syns; -extern int cc_instantiate_on_data; - extern TableType* irc_join_list; extern RecordType* irc_join_info; extern TableVal* irc_servers; diff --git a/src/NetbiosSSN.cc b/src/NetbiosSSN.cc index 274e76f137..362d974956 100644 --- a/src/NetbiosSSN.cc +++ b/src/NetbiosSSN.cc @@ -110,7 +110,7 @@ int NetbiosSSN_Interpreter::ParseDatagram(const u_char* data, int len, return 0; } - + int NetbiosSSN_Interpreter::ParseBroadcast(const u_char* data, int len, int is_query) { @@ -131,6 +131,9 @@ int NetbiosSSN_Interpreter::ParseBroadcast(const u_char* data, int len, return 0; } + delete srcname; + delete dstname; + return 0; } diff --git a/src/OSFinger.cc b/src/OSFinger.cc index f8032d60ee..3368a8e40c 100644 --- a/src/OSFinger.cc +++ b/src/OSFinger.cc @@ -63,15 +63,16 @@ OSFingerprint::OSFingerprint(FingerprintMode arg_mode) } } -bool OSFingerprint::CacheMatch(uint32 addr, int id) +bool OSFingerprint::CacheMatch(const IPAddr& addr, int id) { - HashKey key = HashKey(&addr, 1); + HashKey* key = addr.GetHashKey(); int* pid = new int; *pid=id; - int* prev = os_matches.Insert(&key, pid); + int* prev = os_matches.Insert(key, pid); bool ret = (prev ? *prev != id : 1); if (prev) delete prev; + delete key; return ret; } diff --git a/src/OSFinger.h b/src/OSFinger.h index e5a0829640..0968fb5fd3 100644 --- a/src/OSFinger.h +++ b/src/OSFinger.h @@ -14,6 +14,7 @@ #include "util.h" #include "Dict.h" #include "Reporter.h" +#include "IPAddr.h" // Size limit for size wildcards. #define PACKET_BIG 100 @@ -88,7 +89,7 @@ public: int FindMatch(struct os_type* retval, uint16 tot, uint8 DF_flag, uint8 TTL, uint16 WSS, uint8 ocnt, uint8* op, uint16 MSS, uint8 win_scale, uint32 tstamp, uint32 quirks, uint8 ECN) const; - bool CacheMatch(uint32 addr, int id); + bool CacheMatch(const IPAddr& addr, int id); int Get_OS_From_SYN(struct os_type* retval, uint16 tot, uint8 DF_flag, uint8 TTL, uint16 WSS, diff --git a/src/PIA.cc b/src/PIA.cc index 6a00e7e1d0..9adb4ccab3 100644 --- a/src/PIA.cc +++ b/src/PIA.cc @@ -196,20 +196,20 @@ void PIA_TCP::FirstPacket(bool is_orig, const IP_Hdr* ip) ip4->ip_p = IPPROTO_TCP; // Cast to const so that it doesn't delete it. - ip4_hdr = new IP_Hdr((const struct ip*) ip4); + ip4_hdr = new IP_Hdr(ip4, false); } if ( is_orig ) { - copy_addr(Conn()->OrigAddr(), &ip4->ip_src.s_addr); - copy_addr(Conn()->RespAddr(), &ip4->ip_dst.s_addr); + Conn()->OrigAddr().CopyIPv4(&ip4->ip_src); + Conn()->RespAddr().CopyIPv4(&ip4->ip_dst); tcp4->th_sport = htons(Conn()->OrigPort()); tcp4->th_dport = htons(Conn()->RespPort()); } else { - copy_addr(Conn()->RespAddr(), &ip4->ip_src.s_addr); - copy_addr(Conn()->OrigAddr(), &ip4->ip_dst.s_addr); + Conn()->RespAddr().CopyIPv4(&ip4->ip_src); + Conn()->OrigAddr().CopyIPv4(&ip4->ip_dst); tcp4->th_sport = htons(Conn()->RespPort()); tcp4->th_dport = htons(Conn()->OrigPort()); } diff --git a/src/POP3.cc b/src/POP3.cc index 4ffe67ef48..3075e76507 100644 --- a/src/POP3.cc +++ b/src/POP3.cc @@ -158,6 +158,7 @@ void POP3_Analyzer::ProcessRequest(int length, const char* line) if ( e >= end ) { Weird("pop3_malformed_auth_plain"); + delete decoded; return; } @@ -167,6 +168,7 @@ void POP3_Analyzer::ProcessRequest(int length, const char* line) if ( s >= end ) { Weird("pop3_malformed_auth_plain"); + delete decoded; return; } diff --git a/src/PacketFilter.cc b/src/PacketFilter.cc index 9b6b881ce5..4fb3b1c8f7 100644 --- a/src/PacketFilter.cc +++ b/src/PacketFilter.cc @@ -1,11 +1,11 @@ #include "PacketFilter.h" -void PacketFilter::AddSrc(addr_type src, uint32 tcp_flags, double probability) +void PacketFilter::AddSrc(const IPAddr& src, uint32 tcp_flags, double probability) { Filter* f = new Filter; f->tcp_flags = tcp_flags; f->probability = uint32(probability * RAND_MAX); - src_filter.Insert(src, NUM_ADDR_WORDS * 32, f); + src_filter.Insert(src, 128, f); } void PacketFilter::AddSrc(Val* src, uint32 tcp_flags, double probability) @@ -16,12 +16,12 @@ void PacketFilter::AddSrc(Val* src, uint32 tcp_flags, double probability) src_filter.Insert(src, f); } -void PacketFilter::AddDst(addr_type dst, uint32 tcp_flags, double probability) +void PacketFilter::AddDst(const IPAddr& dst, uint32 tcp_flags, double probability) { Filter* f = new Filter; f->tcp_flags = tcp_flags; f->probability = uint32(probability * RAND_MAX); - dst_filter.Insert(dst, NUM_ADDR_WORDS * 32, f); + dst_filter.Insert(dst, 128, f); } void PacketFilter::AddDst(Val* dst, uint32 tcp_flags, double probability) @@ -32,9 +32,9 @@ void PacketFilter::AddDst(Val* dst, uint32 tcp_flags, double probability) dst_filter.Insert(dst, f); } -bool PacketFilter::RemoveSrc(addr_type src) +bool PacketFilter::RemoveSrc(const IPAddr& src) { - return src_filter.Remove(src, NUM_ADDR_WORDS * 32) != 0; + return src_filter.Remove(src, 128) != 0; } bool PacketFilter::RemoveSrc(Val* src) @@ -42,9 +42,9 @@ bool PacketFilter::RemoveSrc(Val* src) return src_filter.Remove(src) != NULL; } -bool PacketFilter::RemoveDst(addr_type dst) +bool PacketFilter::RemoveDst(const IPAddr& dst) { - return dst_filter.Remove(dst, NUM_ADDR_WORDS * 32) != NULL; + return dst_filter.Remove(dst, 128) != NULL; } bool PacketFilter::RemoveDst(Val* dst) @@ -54,21 +54,11 @@ bool PacketFilter::RemoveDst(Val* dst) bool PacketFilter::Match(const IP_Hdr* ip, int len, int caplen) { -#ifdef BROv6 - Filter* f = (Filter*) src_filter.Lookup(ip->SrcAddr(), - NUM_ADDR_WORDS * 32); -#else - Filter* f = (Filter*) src_filter.Lookup(*ip->SrcAddr(), - NUM_ADDR_WORDS * 32); -#endif + Filter* f = (Filter*) src_filter.Lookup(ip->SrcAddr(), 128); if ( f ) return MatchFilter(*f, *ip, len, caplen); -#ifdef BROv6 - f = (Filter*) dst_filter.Lookup(ip->DstAddr(), NUM_ADDR_WORDS * 32); -#else - f = (Filter*) dst_filter.Lookup(*ip->DstAddr(), NUM_ADDR_WORDS * 32); -#endif + f = (Filter*) dst_filter.Lookup(ip->DstAddr(), 128); if ( f ) return MatchFilter(*f, *ip, len, caplen); @@ -81,9 +71,7 @@ bool PacketFilter::MatchFilter(const Filter& f, const IP_Hdr& ip, if ( ip.NextProto() == IPPROTO_TCP && f.tcp_flags ) { // Caution! The packet sanity checks have not been performed yet - const struct ip* ip4 = ip.IP4_Hdr(); - - int ip_hdr_len = ip4->ip_hl * 4; + int ip_hdr_len = ip.HdrLen(); len -= ip_hdr_len; // remove IP header caplen -= ip_hdr_len; @@ -92,8 +80,7 @@ bool PacketFilter::MatchFilter(const Filter& f, const IP_Hdr& ip, // Packet too short, will be dropped anyway. return false; - const struct tcphdr* tp = - (const struct tcphdr*) ((u_char*) ip4 + ip_hdr_len); + const struct tcphdr* tp = (const struct tcphdr*) ip.Payload(); if ( tp->th_flags & f.tcp_flags ) // At least one of the flags is set, so don't drop diff --git a/src/PacketFilter.h b/src/PacketFilter.h index ed8000f40f..3d7a3aa3be 100644 --- a/src/PacketFilter.h +++ b/src/PacketFilter.h @@ -14,16 +14,16 @@ public: // Drops all packets from a particular source (which may be given // as an AddrVal or a SubnetVal) which hasn't any of TCP flags set // (TH_*) with the given probability (from 0..MAX_PROB). - void AddSrc(addr_type src, uint32 tcp_flags, double probability); + void AddSrc(const IPAddr& src, uint32 tcp_flags, double probability); void AddSrc(Val* src, uint32 tcp_flags, double probability); - void AddDst(addr_type src, uint32 tcp_flags, double probability); + void AddDst(const IPAddr& src, uint32 tcp_flags, double probability); void AddDst(Val* src, uint32 tcp_flags, double probability); // Removes the filter entry for the given src/dst // Returns false if filter doesn not exist. - bool RemoveSrc(addr_type src); + bool RemoveSrc(const IPAddr& src); bool RemoveSrc(Val* dst); - bool RemoveDst(addr_type dst); + bool RemoveDst(const IPAddr& dst); bool RemoveDst(Val* dst); // Returns true if packet matches a drop filter diff --git a/src/PacketSort.cc b/src/PacketSort.cc index 0ff08b3280..a7e2b04572 100644 --- a/src/PacketSort.cc +++ b/src/PacketSort.cc @@ -27,13 +27,16 @@ PacketSortElement::PacketSortElement(PktSrc* arg_src, { const struct ip* ip = (const struct ip*) (pkt + hdr_size); if ( ip->ip_v == 4 ) - ip_hdr = new IP_Hdr(ip); + ip_hdr = new IP_Hdr(ip, false); + else if ( ip->ip_v == 6 && (caplen >= sizeof(struct ip6_hdr) + hdr_size) ) + ip_hdr = new IP_Hdr((const struct ip6_hdr*) ip, false, caplen - hdr_size); else - ip_hdr = new IP_Hdr((const struct ip6_hdr*) ip); + // Weird will be generated later in NetSessions::NextPacket. + return; if ( ip_hdr->NextProto() == IPPROTO_TCP && // Note: can't sort fragmented packets - (ip_hdr->FragField() & 0x3fff) == 0 ) + ( ! ip_hdr->IsFragment() ) ) { tcp_offset = hdr_size + ip_hdr->HdrLen(); if ( caplen >= tcp_offset + sizeof(struct tcphdr) ) @@ -65,7 +68,7 @@ PacketSortElement::PacketSortElement(PktSrc* arg_src, payload_length = ip_hdr->PayloadLen() - tp->th_off * 4; - key = id.BuildConnKey(); + key = BuildConnIDHashKey(id); is_tcp = 1; } diff --git a/src/PersistenceSerializer.cc b/src/PersistenceSerializer.cc index c757467f90..d9baad05bb 100644 --- a/src/PersistenceSerializer.cc +++ b/src/PersistenceSerializer.cc @@ -137,7 +137,7 @@ bool PersistenceSerializer::CheckForFile(UnserialInfo* info, const char* file, bool PersistenceSerializer::ReadAll(bool is_init, bool delete_files) { -#ifdef USE_PERFTOOLS +#ifdef USE_PERFTOOLS_DEBUG HeapLeakChecker::Disabler disabler; #endif diff --git a/src/PktSrc.cc b/src/PktSrc.cc index 68b9785e6f..615815b41b 100644 --- a/src/PktSrc.cc +++ b/src/PktSrc.cc @@ -382,6 +382,7 @@ void PktSrc::AddSecondaryTablePrograms() { delete program; Close(); + return; } SecondaryProgram* sp = new SecondaryProgram(program, se); diff --git a/src/PrefixTable.cc b/src/PrefixTable.cc index e04c3f9294..58a488a9f1 100644 --- a/src/PrefixTable.cc +++ b/src/PrefixTable.cc @@ -1,34 +1,19 @@ #include "PrefixTable.h" #include "Reporter.h" -// IPv4 version. -inline static prefix_t* make_prefix(const uint32 addr, int width) +inline static prefix_t* make_prefix(const IPAddr& addr, int width) { prefix_t* prefix = (prefix_t*) safe_malloc(sizeof(prefix_t)); - memcpy(&prefix->add.sin, &addr, sizeof(prefix->add.sin)) ; - prefix->family = AF_INET; - prefix->bitlen = width; - prefix->ref_count = 1; - - return prefix; - } - -#ifdef BROv6 -inline static prefix_t* make_prefix(const uint32* addr, int width) - { - prefix_t* prefix = (prefix_t*) safe_malloc(sizeof(prefix_t)); - - memcpy(&prefix->add.sin6, addr, 4 * sizeof(uint32)); + addr.CopyIPv6(&prefix->add.sin6); prefix->family = AF_INET6; prefix->bitlen = width; prefix->ref_count = 1; return prefix; } -#endif -void* PrefixTable::Insert(const_addr_type addr, int width, void* data) +void* PrefixTable::Insert(const IPAddr& addr, int width, void* data) { prefix_t* prefix = make_prefix(addr, width); patricia_node_t* node = patricia_lookup(tree, prefix); @@ -55,12 +40,12 @@ void* PrefixTable::Insert(const Val* value, void* data) switch ( value->Type()->Tag() ) { case TYPE_ADDR: - return Insert(value->AsAddr(), NUM_ADDR_WORDS * 32, data); + return Insert(value->AsAddr(), 128, data); break; case TYPE_SUBNET: - return Insert(value->AsSubNet()->net, - value->AsSubNet()->width, data); + return Insert(value->AsSubNet().Prefix(), + value->AsSubNet().LengthIPv6(), data); break; default: @@ -69,7 +54,7 @@ void* PrefixTable::Insert(const Val* value, void* data) } } -void* PrefixTable::Lookup(const_addr_type addr, int width, bool exact) const +void* PrefixTable::Lookup(const IPAddr& addr, int width, bool exact) const { prefix_t* prefix = make_prefix(addr, width); patricia_node_t* node = @@ -89,12 +74,12 @@ void* PrefixTable::Lookup(const Val* value, bool exact) const switch ( value->Type()->Tag() ) { case TYPE_ADDR: - return Lookup(value->AsAddr(), NUM_ADDR_WORDS * 32, exact); + return Lookup(value->AsAddr(), 128, exact); break; case TYPE_SUBNET: - return Lookup(value->AsSubNet()->net, - value->AsSubNet()->width, exact); + return Lookup(value->AsSubNet().Prefix(), + value->AsSubNet().LengthIPv6(), exact); break; default: @@ -104,7 +89,7 @@ void* PrefixTable::Lookup(const Val* value, bool exact) const } } -void* PrefixTable::Remove(const_addr_type addr, int width) +void* PrefixTable::Remove(const IPAddr& addr, int width) { prefix_t* prefix = make_prefix(addr, width); patricia_node_t* node = patricia_search_exact(tree, prefix); @@ -128,11 +113,12 @@ void* PrefixTable::Remove(const Val* value) switch ( value->Type()->Tag() ) { case TYPE_ADDR: - return Remove(value->AsAddr(), NUM_ADDR_WORDS * 32); + return Remove(value->AsAddr(), 128); break; case TYPE_SUBNET: - return Remove(value->AsSubNet()->net, value->AsSubNet()->width); + return Remove(value->AsSubNet().Prefix(), + value->AsSubNet().LengthIPv6()); break; default: diff --git a/src/PrefixTable.h b/src/PrefixTable.h index 78596c7f35..2e5f43a0a8 100644 --- a/src/PrefixTable.h +++ b/src/PrefixTable.h @@ -3,6 +3,7 @@ #include "Val.h" #include "net_util.h" +#include "IPAddr.h" extern "C" { #include "patricia.h" @@ -24,7 +25,7 @@ public: // Addr in network byte order. If data is zero, acts like a set. // Returns ptr to old data if already existing. // For existing items without data, returns non-nil if found. - void* Insert(const_addr_type addr, int width, void* data = 0); + void* Insert(const IPAddr& addr, int width, void* data = 0); // Value may be addr or subnet. void* Insert(const Val* value, void* data = 0); @@ -32,11 +33,11 @@ public: // Returns nil if not found, pointer to data otherwise. // For items without data, returns non-nil if found. // If exact is false, performs exact rather than longest-prefix match. - void* Lookup(const_addr_type addr, int width, bool exact = false) const; + void* Lookup(const IPAddr& addr, int width, bool exact = false) const; void* Lookup(const Val* value, bool exact = false) const; // Returns pointer to data or nil if not found. - void* Remove(const_addr_type addr, int width); + void* Remove(const IPAddr& addr, int width); void* Remove(const Val* value); void Clear() { Clear_Patricia(tree, 0); } diff --git a/src/Reassem.cc b/src/Reassem.cc index 89fe29e7d4..c3c19ff0e6 100644 --- a/src/Reassem.cc +++ b/src/Reassem.cc @@ -43,8 +43,7 @@ DataBlock::DataBlock(const u_char* data, int size, int arg_seq, unsigned int Reassembler::total_size = 0; -Reassembler::Reassembler(int init_seq, const uint32* ip_addr, - ReassemblerType arg_type) +Reassembler::Reassembler(int init_seq, ReassemblerType arg_type) { blocks = last_block = 0; trim_seq = last_reassem_seq = init_seq; diff --git a/src/Reassem.h b/src/Reassem.h index 06d1e28f40..1f65059e02 100644 --- a/src/Reassem.h +++ b/src/Reassem.h @@ -4,6 +4,7 @@ #define reassem_h #include "Obj.h" +#include "IPAddr.h" class DataBlock { public: @@ -25,8 +26,7 @@ enum ReassemblerType { REASSEM_IP, REASSEM_TCP }; class Reassembler : public BroObj { public: - Reassembler(int init_seq, const uint32* ip_addr, - ReassemblerType arg_type); + Reassembler(int init_seq, ReassemblerType arg_type); virtual ~Reassembler(); void NewBlock(double t, int seq, int len, const u_char* data); diff --git a/src/RemoteSerializer.cc b/src/RemoteSerializer.cc index b72a6dcc1a..61be8a9e8f 100644 --- a/src/RemoteSerializer.cc +++ b/src/RemoteSerializer.cc @@ -183,8 +183,11 @@ #include "Sessions.h" #include "File.h" #include "Conn.h" -#include "LogMgr.h" #include "Reporter.h" +#include "threading/SerialTypes.h" +#include "logging/Manager.h" +#include "IPAddr.h" +#include "bro_inet_ntop.h" extern "C" { #include "setsignal.h" @@ -463,7 +466,7 @@ static inline const char* ip2a(uint32 ip) addr.s_addr = htonl(ip); - return inet_ntop(AF_INET, &addr, buffer, 32); + return bro_inet_ntop(AF_INET, &addr, buffer, 32); } static pid_t child_pid = 0; @@ -530,6 +533,7 @@ RemoteSerializer::RemoteSerializer() terminating = false; in_sync = 0; last_flush = 0; + received_logs = 0; } RemoteSerializer::~RemoteSerializer() @@ -670,8 +674,8 @@ void RemoteSerializer::Fork() } } -RemoteSerializer::PeerID RemoteSerializer::Connect(addr_type ip, uint16 port, - const char* our_class, double retry, bool use_ssl) +RemoteSerializer::PeerID RemoteSerializer::Connect(const IPAddr& ip, + uint16 port, const char* our_class, double retry, bool use_ssl) { if ( ! using_communication ) return true; @@ -679,16 +683,12 @@ RemoteSerializer::PeerID RemoteSerializer::Connect(addr_type ip, uint16 port, if ( ! initialized ) reporter->InternalError("remote serializer not initialized"); -#ifdef BROv6 - if ( ! is_v4_addr(ip) ) + if ( ip.GetFamily() == IPv6 ) Error("inter-Bro communication not supported over IPv6"); - uint32 ip4 = to_v4_addr(ip); -#else - uint32 ip4 = ip; -#endif - - ip4 = ntohl(ip4); + const uint32* bytes; + ip.GetBytes(&bytes); + uint32 ip4 = ntohl(*bytes); if ( ! child_pid ) Fork(); @@ -1232,7 +1232,7 @@ bool RemoteSerializer::SendCapabilities(Peer* peer) return caps ? SendToChild(MSG_CAPS, peer, 3, caps, 0, 0) : true; } -bool RemoteSerializer::Listen(addr_type ip, uint16 port, bool expect_ssl) +bool RemoteSerializer::Listen(const IPAddr& ip, uint16 port, bool expect_ssl) { if ( ! using_communication ) return true; @@ -1240,16 +1240,12 @@ bool RemoteSerializer::Listen(addr_type ip, uint16 port, bool expect_ssl) if ( ! initialized ) reporter->InternalError("remote serializer not initialized"); -#ifdef BROv6 - if ( ! is_v4_addr(ip) ) + if ( ip.GetFamily() == IPv6 ) Error("inter-Bro communication not supported over IPv6"); - uint32 ip4 = to_v4_addr(ip); -#else - uint32 ip4 = ip; -#endif - - ip4 = ntohl(ip4); + const uint32* bytes; + ip.GetBytes(&bytes); + uint32 ip4 = ntohl(*bytes); if ( ! SendToChild(MSG_LISTEN, 0, 3, ip4, port, expect_ssl) ) return false; @@ -1359,6 +1355,14 @@ double RemoteSerializer::NextTimestamp(double* local_network_time) { Poll(false); + if ( received_logs > 0 ) + { + // If we processed logs last time, assume there's more. + idle = false; + received_logs = 0; + return timer_mgr->Time(); + } + double et = events.length() ? events[0]->time : -1; double pt = packets.length() ? packets[0]->time : -1; @@ -2476,7 +2480,7 @@ bool RemoteSerializer::ProcessRemotePrint() return true; } -bool RemoteSerializer::SendLogCreateWriter(EnumVal* id, EnumVal* writer, string path, int num_fields, const LogField* const * fields) +bool RemoteSerializer::SendLogCreateWriter(EnumVal* id, EnumVal* writer, string path, int num_fields, const threading::Field* const * fields) { loop_over_list(peers, i) { @@ -2486,7 +2490,7 @@ bool RemoteSerializer::SendLogCreateWriter(EnumVal* id, EnumVal* writer, string return true; } -bool RemoteSerializer::SendLogCreateWriter(PeerID peer_id, EnumVal* id, EnumVal* writer, string path, int num_fields, const LogField* const * fields) +bool RemoteSerializer::SendLogCreateWriter(PeerID peer_id, EnumVal* id, EnumVal* writer, string path, int num_fields, const threading::Field* const * fields) { SetErrorDescr("logging"); @@ -2499,6 +2503,9 @@ bool RemoteSerializer::SendLogCreateWriter(PeerID peer_id, EnumVal* id, EnumVal* if ( peer->phase != Peer::HANDSHAKE && peer->phase != Peer::RUNNING ) return false; + if ( ! peer->logs_requested ) + return false; + BinarySerializationFormat fmt; fmt.StartWrite(); @@ -2540,7 +2547,7 @@ error: return false; } -bool RemoteSerializer::SendLogWrite(EnumVal* id, EnumVal* writer, string path, int num_fields, const LogVal* const * vals) +bool RemoteSerializer::SendLogWrite(EnumVal* id, EnumVal* writer, string path, int num_fields, const threading::Value* const * vals) { loop_over_list(peers, i) { @@ -2550,7 +2557,7 @@ bool RemoteSerializer::SendLogWrite(EnumVal* id, EnumVal* writer, string path, i return true; } -bool RemoteSerializer::SendLogWrite(Peer* peer, EnumVal* id, EnumVal* writer, string path, int num_fields, const LogVal* const * vals) +bool RemoteSerializer::SendLogWrite(Peer* peer, EnumVal* id, EnumVal* writer, string path, int num_fields, const threading::Value* const * vals) { if ( peer->phase != Peer::HANDSHAKE && peer->phase != Peer::RUNNING ) return false; @@ -2558,7 +2565,9 @@ bool RemoteSerializer::SendLogWrite(Peer* peer, EnumVal* id, EnumVal* writer, st if ( ! peer->logs_requested ) return false; - assert(peer->log_buffer); + if ( ! peer->log_buffer ) + // Peer shutting down. + return false; // Serialize the log record entry. @@ -2593,7 +2602,10 @@ bool RemoteSerializer::SendLogWrite(Peer* peer, EnumVal* id, EnumVal* writer, st if ( len > (LOG_BUFFER_SIZE - peer->log_buffer_used) || (network_time - last_flush > 1.0) ) { if ( ! FlushLogBuffer(peer) ) + { + delete [] data; return false; + } } // If the data is actually larger than our complete buffer, just send it out. @@ -2616,6 +2628,9 @@ error: bool RemoteSerializer::FlushLogBuffer(Peer* p) { + if ( ! p->logs_requested ) + return false; + last_flush = network_time; if ( p->state == Peer::CLOSING ) @@ -2637,11 +2652,17 @@ bool RemoteSerializer::ProcessLogCreateWriter() if ( current_peer->state == Peer::CLOSING ) return false; +#ifdef USE_PERFTOOLS_DEBUG + // Don't track allocations here, they'll be released only after the + // main loop exists. And it's just a tiny amount anyway. + HeapLeakChecker::Disabler disabler; +#endif + assert(current_args); EnumVal* id_val = 0; EnumVal* writer_val = 0; - LogField** fields = 0; + threading::Field** fields = 0; BinarySerializationFormat fmt; fmt.StartRead(current_args->data, current_args->len); @@ -2658,11 +2679,11 @@ bool RemoteSerializer::ProcessLogCreateWriter() if ( ! success ) goto error; - fields = new LogField* [num_fields]; + fields = new threading::Field* [num_fields]; for ( int i = 0; i < num_fields; i++ ) { - fields[i] = new LogField; + fields[i] = new threading::Field; if ( ! fields[i]->Read(&fmt) ) goto error; } @@ -2672,7 +2693,7 @@ bool RemoteSerializer::ProcessLogCreateWriter() id_val = new EnumVal(id, BifType::Enum::Log::ID); writer_val = new EnumVal(writer, BifType::Enum::Log::Writer); - if ( ! log_mgr->CreateWriter(id_val, writer_val, path, num_fields, fields) ) + if ( ! log_mgr->CreateWriter(id_val, writer_val, path, num_fields, fields, true, false) ) goto error; Unref(id_val); @@ -2703,7 +2724,7 @@ bool RemoteSerializer::ProcessLogWrite() // Unserialize one entry. EnumVal* id_val = 0; EnumVal* writer_val = 0; - LogVal** vals = 0; + threading::Value** vals = 0; int id, writer; string path; @@ -2717,11 +2738,11 @@ bool RemoteSerializer::ProcessLogWrite() if ( ! success ) goto error; - vals = new LogVal* [num_fields]; + vals = new threading::Value* [num_fields]; for ( int i = 0; i < num_fields; i++ ) { - vals[i] = new LogVal; + vals[i] = new threading::Value; if ( ! vals[i]->Read(&fmt) ) goto error; } @@ -2741,6 +2762,8 @@ bool RemoteSerializer::ProcessLogWrite() fmt.EndRead(); + ++received_logs; + return true; error: @@ -2850,7 +2873,7 @@ void RemoteSerializer::GotID(ID* id, Val* val) (desc && *desc) ? desc : "not set"), current_peer); -#ifdef USE_PERFTOOLS +#ifdef USE_PERFTOOLS_DEBUG // May still be cached, but we don't care. heap_checker->IgnoreObject(id); #endif @@ -3382,6 +3405,11 @@ void SocketComm::Run() small_timeout.tv_usec = io->CanWrite() || io->CanRead() ? 1 : 10; +#if 0 + if ( ! io->CanWrite() ) + usleep(10); +#endif + int a = select(max_fd + 1, &fd_read, &fd_write, &fd_except, &small_timeout); diff --git a/src/RemoteSerializer.h b/src/RemoteSerializer.h index b64fdcbe66..05d25ca525 100644 --- a/src/RemoteSerializer.h +++ b/src/RemoteSerializer.h @@ -14,8 +14,11 @@ // FIXME: Change this to network byte order class IncrementalSendTimer; -class LogField; -class LogVal; + +namespace threading { + class Field; + class Value; +} // This class handles the communication done in Bro's main loop. class RemoteSerializer : public Serializer, public IOSource { @@ -32,7 +35,7 @@ public: static const PeerID PEER_NONE = SOURCE_LOCAL; // Connect to host (returns PEER_NONE on error). - PeerID Connect(addr_type ip, uint16 port, const char* our_class, double retry, bool use_ssl); + PeerID Connect(const IPAddr& ip, uint16 port, const char* our_class, double retry, bool use_ssl); // Close connection to host. bool CloseConnection(PeerID peer); @@ -60,7 +63,7 @@ public: bool CompleteHandshake(PeerID peer); // Start to listen. - bool Listen(addr_type ip, uint16 port, bool expect_ssl); + bool Listen(const IPAddr& ip, uint16 port, bool expect_ssl); // Stop it. bool StopListening(); @@ -99,13 +102,13 @@ public: bool SendPrintHookEvent(BroFile* f, const char* txt, size_t len); // Send a request to create a writer on a remote side. - bool SendLogCreateWriter(PeerID peer, EnumVal* id, EnumVal* writer, string path, int num_fields, const LogField* const * fields); + bool SendLogCreateWriter(PeerID peer, EnumVal* id, EnumVal* writer, string path, int num_fields, const threading::Field* const * fields); // Broadcasts a request to create a writer. - bool SendLogCreateWriter(EnumVal* id, EnumVal* writer, string path, int num_fields, const LogField* const * fields); + bool SendLogCreateWriter(EnumVal* id, EnumVal* writer, string path, int num_fields, const threading::Field* const * fields); // Broadcast a log entry to everybody interested. - bool SendLogWrite(EnumVal* id, EnumVal* writer, string path, int num_fields, const LogVal* const * vals); + bool SendLogWrite(EnumVal* id, EnumVal* writer, string path, int num_fields, const threading::Value* const * vals); // Synchronzizes time with all connected peers. Returns number of // current sync-point, or -1 on error. @@ -300,7 +303,7 @@ protected: bool SendID(SerialInfo* info, Peer* peer, const ID& id); bool SendCapabilities(Peer* peer); bool SendPacket(SerialInfo* info, Peer* peer, const Packet& p); - bool SendLogWrite(Peer* peer, EnumVal* id, EnumVal* writer, string path, int num_fields, const LogVal* const * vals); + bool SendLogWrite(Peer* peer, EnumVal* id, EnumVal* writer, string path, int num_fields, const threading::Value* const * vals); void UnregisterHandlers(Peer* peer); void RaiseEvent(EventHandlerPtr event, Peer* peer, const char* arg = 0); @@ -335,6 +338,7 @@ private: int propagate_accesses; bool ignore_accesses; bool terminating; + int received_logs; Peer* source_peer; PeerID id_counter; // Keeps track of assigned IDs. uint32 current_sync_point; diff --git a/src/Reporter.cc b/src/Reporter.cc index fca9a6a233..37470cd690 100644 --- a/src/Reporter.cc +++ b/src/Reporter.cc @@ -155,7 +155,7 @@ void Reporter::WeirdHelper(EventHandlerPtr event, Val* conn_val, const char* add delete vl; } -void Reporter::WeirdFlowHelper(const uint32* orig, const uint32* resp, const char* fmt_name, ...) +void Reporter::WeirdFlowHelper(const IPAddr& orig, const IPAddr& resp, const char* fmt_name, ...) { val_list* vl = new val_list(2); vl->append(new AddrVal(orig)); @@ -184,7 +184,7 @@ void Reporter::Weird(Val* conn_val, const char* name, const char* addl) WeirdHelper(conn_weird, conn_val, addl, "%s", name); } -void Reporter::Weird(const uint32* orig, const uint32* resp, const char* name) +void Reporter::Weird(const IPAddr& orig, const IPAddr& resp, const char* name) { WeirdFlowHelper(orig, resp, "%s", name); } diff --git a/src/Reporter.h b/src/Reporter.h index 20d7b17e2c..e610e1519e 100644 --- a/src/Reporter.h +++ b/src/Reporter.h @@ -9,8 +9,8 @@ #include #include "util.h" -#include "net_util.h" #include "EventHandler.h" +#include "IPAddr.h" class Connection; class Location; @@ -74,7 +74,7 @@ public: void Weird(const char* name); // Raises net_weird(). void Weird(Connection* conn, const char* name, const char* addl = ""); // Raises conn_weird(). void Weird(Val* conn_val, const char* name, const char* addl = ""); // Raises conn_weird(). - void Weird(const uint32* orig, const uint32* resp, const char* name); // Raises flow_weird(). + void Weird(const IPAddr& orig, const IPAddr& resp, const char* name); // Raises flow_weird(). // Syslog a message. This methods does nothing if we're running // offline from a trace. @@ -121,7 +121,7 @@ private: // The order if addl, name needs to be like that since fmt_name can // contain format specifiers void WeirdHelper(EventHandlerPtr event, Val* conn_val, const char* addl, const char* fmt_name, ...); - void WeirdFlowHelper(const uint32* orig, const uint32* resp, const char* fmt_name, ...); + void WeirdFlowHelper(const IPAddr& orig, const IPAddr& resp, const char* fmt_name, ...); int errors; bool via_events; diff --git a/src/RuleMatcher.cc b/src/RuleMatcher.cc index ac8f8a867c..c9cf1f5c11 100644 --- a/src/RuleMatcher.cc +++ b/src/RuleMatcher.cc @@ -73,6 +73,9 @@ RuleHdrTest::RuleHdrTest(RuleHdrTest& h) copied_set->ids = orig_set->ids; loop_over_list(orig_set->patterns, l) copied_set->patterns.append(copy_string(orig_set->patterns[l])); + delete copied_set; + // TODO: Why do we create copied_set only to then + // never use it? } } @@ -188,7 +191,7 @@ void RuleMatcher::Delete(RuleHdrTest* node) bool RuleMatcher::ReadFiles(const name_list& files) { -#ifdef USE_PERFTOOLS +#ifdef USE_PERFTOOLS_DEBUG HeapLeakChecker::Disabler disabler; #endif @@ -1067,16 +1070,22 @@ static bool val_to_maskedval(Val* v, maskedvalue_list* append_to) break; case TYPE_SUBNET: -#ifdef BROv6 { - uint32* n = v->AsSubNet()->net; - uint32* m = v->AsSubNetVal()->Mask(); + const uint32* n; + uint32 m[4]; + v->AsSubNet().Prefix().GetBytes(&n); + v->AsSubNetVal()->Mask().CopyIPv6(m); + + for ( unsigned int i = 0; i < 4; ++i ) + m[i] = ntohl(m[i]); + bool is_v4_mask = m[0] == 0xffffffff && m[1] == m[0] && m[2] == m[0]; - if ( is_v4_addr(n) && is_v4_mask ) + if ( v->AsSubNet().Prefix().GetFamily() == IPv4 && + is_v4_mask ) { - mval->val = ntohl(to_v4_addr(n)); + mval->val = ntohl(*n); mval->mask = m[3]; } @@ -1087,10 +1096,6 @@ static bool val_to_maskedval(Val* v, maskedvalue_list* append_to) mval->mask = 0; } } -#else - mval->val = ntohl(v->AsSubNet()->net); - mval->mask = v->AsSubNetVal()->Mask(); -#endif break; default: @@ -1114,7 +1119,12 @@ void id_to_maskedvallist(const char* id, maskedvalue_list* append_to) val_list* vals = v->AsTableVal()->ConvertToPureList()->Vals(); loop_over_list(*vals, i ) if ( ! val_to_maskedval((*vals)[i], append_to) ) + { + delete_vals(vals); return; + } + + delete_vals(vals); } else diff --git a/src/SMTP.cc b/src/SMTP.cc index 3af8af3b7b..85a3bc79dc 100644 --- a/src/SMTP.cc +++ b/src/SMTP.cc @@ -353,7 +353,6 @@ void SMTP_Analyzer::ProcessLine(int length, const char* line, bool orig) int ext_len; get_word(end_of_line - line, line, ext_len, ext); - line = skip_whitespace(line + ext_len, end_of_line); ProcessExtension(ext_len, ext); } } diff --git a/src/SSH.cc b/src/SSH.cc index c07aad3dd1..3a8f468ae4 100644 --- a/src/SSH.cc +++ b/src/SSH.cc @@ -50,23 +50,24 @@ void SSH_Analyzer::DeliverStream(int length, const u_char* data, bool is_orig) // SSH-.-\n // // We're interested in the "version" part here. - + if ( length < 4 || memcmp(line, "SSH-", 4) != 0 ) { Weird("malformed_ssh_identification"); ProtocolViolation("malformed ssh identification", line, length); return; } - + int i; for ( i = 4; i < length && line[i] != '-'; ++i ) ; - + if ( TCP() ) { if ( length >= i ) { - const uint32* dst; + IPAddr dst; + if ( is_orig ) dst = TCP()->Orig()->dst_addr; else diff --git a/src/SerialTypes.h b/src/SerialTypes.h index 0ba48f89a9..c47ff19298 100644 --- a/src/SerialTypes.h +++ b/src/SerialTypes.h @@ -125,7 +125,7 @@ SERIAL_EXPR(FIELD_EXPR, 22) SERIAL_EXPR(HAS_FIELD_EXPR, 23) SERIAL_EXPR(RECORD_CONSTRUCTOR_EXPR, 24) SERIAL_EXPR(FIELD_ASSIGN_EXPR, 25) -SERIAL_EXPR(RECORD_MATCH_EXPR, 26) +// There used to be a SERIAL_EXPR(RECORD_MATCH_EXPR, 26) here SERIAL_EXPR(ARITH_COERCE_EXPR, 27) SERIAL_EXPR(RECORD_COERCE_EXPR, 28) SERIAL_EXPR(FLATTEN_EXPR, 29) diff --git a/src/SerializationFormat.cc b/src/SerializationFormat.cc index 5e3a68a42e..10dd4f29ea 100644 --- a/src/SerializationFormat.cc +++ b/src/SerializationFormat.cc @@ -230,6 +230,71 @@ bool BinarySerializationFormat::Read(string* v, const char* tag) return true; } +bool BinarySerializationFormat::Read(IPAddr* addr, const char* tag) + { + int n = 0; + if ( ! Read(&n, "addr-len") ) + return false; + + if ( n != 1 && n != 4 ) + return false; + + uint32_t raw[4]; + + for ( int i = 0; i < n; ++i ) + { + if ( ! Read(&raw[i], "addr-part") ) + return false; + + raw[i] = htonl(raw[i]); + } + + if ( n == 1 ) + *addr = IPAddr(IPv4, raw, IPAddr::Network); + else + *addr = IPAddr(IPv6, raw, IPAddr::Network); + + return true; + } + +bool BinarySerializationFormat::Read(IPPrefix* prefix, const char* tag) + { + IPAddr addr; + int len; + + if ( ! (Read(&addr, "prefix") && Read(&len, "width")) ) + return false; + + *prefix = IPPrefix(addr, len); + return true; + } + +bool BinarySerializationFormat::Read(struct in_addr* addr, const char* tag) + { + uint32_t* bytes = (uint32_t*) &addr->s_addr; + + if ( ! Read(&bytes[0], "addr4") ) + return false; + + bytes[0] = htonl(bytes[0]); + return true; + } + +bool BinarySerializationFormat::Read(struct in6_addr* addr, const char* tag) + { + uint32_t* bytes = (uint32_t*) &addr->s6_addr; + + for ( int i = 0; i < 4; ++i ) + { + if ( ! Read(&bytes[i], "addr6-part") ) + return false; + + bytes[i] = htonl(bytes[i]); + } + + return true; + } + bool BinarySerializationFormat::Write(char v, const char* tag) { DBG_LOG(DBG_SERIAL, "Write char %s [%s]", fmt_bytes(&v, 1), tag); @@ -299,6 +364,53 @@ bool BinarySerializationFormat::Write(const string& s, const char* tag) return Write(s.data(), s.size(), tag); } +bool BinarySerializationFormat::Write(const IPAddr& addr, const char* tag) + { + const uint32_t* raw; + int n = addr.GetBytes(&raw); + + assert(n == 1 || n == 4); + + if ( ! Write(n, "addr-len") ) + return false; + + for ( int i = 0; i < n; ++i ) + { + if ( ! Write(ntohl(raw[i]), "addr-part") ) + return false; + } + + return true; + } + +bool BinarySerializationFormat::Write(const IPPrefix& prefix, const char* tag) + { + return Write(prefix.Prefix(), "prefix") && Write(prefix.Length(), "width"); + } + +bool BinarySerializationFormat::Write(const struct in_addr& addr, const char* tag) + { + const uint32_t* bytes = (uint32_t*) &addr.s_addr; + + if ( ! Write(ntohl(bytes[0]), "addr4") ) + return false; + + return true; + } + +bool BinarySerializationFormat::Write(const struct in6_addr& addr, const char* tag) + { + const uint32_t* bytes = (uint32_t*) &addr.s6_addr; + + for ( int i = 0; i < 4; ++i ) + { + if ( ! Write(ntohl(bytes[i]), "addr6-part") ) + return false; + } + + return true; + } + bool BinarySerializationFormat::WriteOpenTag(const char* tag) { return true; @@ -389,6 +501,30 @@ bool XMLSerializationFormat::Read(string* s, const char* tag) return false; } +bool XMLSerializationFormat::Read(IPAddr* addr, const char* tag) + { + reporter->InternalError("no reading of xml"); + return false; + } + +bool XMLSerializationFormat::Read(IPPrefix* prefix, const char* tag) + { + reporter->InternalError("no reading of xml"); + return false; + } + +bool XMLSerializationFormat::Read(struct in_addr* addr, const char* tag) + { + reporter->InternalError("no reading of xml"); + return false; + } + +bool XMLSerializationFormat::Read(struct in6_addr* addr, const char* tag) + { + reporter->InternalError("no reading of xml"); + return false; + } + bool XMLSerializationFormat::Write(char v, const char* tag) { return WriteElem(tag, "char", &v, 1); @@ -469,6 +605,30 @@ bool XMLSerializationFormat::Write(const char* buf, int len, const char* tag) return WriteElem(tag, "string", buf, len); } +bool XMLSerializationFormat::Write(const IPAddr& addr, const char* tag) + { + reporter->InternalError("XML output of addresses not implemented"); + return false; + } + +bool XMLSerializationFormat::Write(const IPPrefix& prefix, const char* tag) + { + reporter->InternalError("XML output of prefixes not implemented"); + return false; + } + +bool XMLSerializationFormat::Write(const struct in_addr& addr, const char* tag) + { + reporter->InternalError("XML output of in_addr not implemented"); + return false; + } + +bool XMLSerializationFormat::Write(const struct in6_addr& addr, const char* tag) + { + reporter->InternalError("XML output of in6_addr not implemented"); + return false; + } + bool XMLSerializationFormat::WriteEncodedString(const char* s, int len) { while ( len-- ) diff --git a/src/SerializationFormat.h b/src/SerializationFormat.h index 2067456bf1..f270b61bae 100644 --- a/src/SerializationFormat.h +++ b/src/SerializationFormat.h @@ -9,6 +9,9 @@ using namespace std; #include "util.h" +class IPAddr; +class IPPrefix; + // Abstract base class. class SerializationFormat { public: @@ -28,6 +31,10 @@ public: virtual bool Read(bool* v, const char* tag) = 0; virtual bool Read(double* d, const char* tag) = 0; virtual bool Read(string* s, const char* tag) = 0; + virtual bool Read(IPAddr* addr, const char* tag) = 0; + virtual bool Read(IPPrefix* prefix, const char* tag) = 0; + virtual bool Read(struct in_addr* addr, const char* tag) = 0; + virtual bool Read(struct in6_addr* addr, const char* tag) = 0; // Returns number of raw bytes read since last call to StartRead(). int BytesRead() const { return bytes_read; } @@ -50,6 +57,10 @@ public: virtual bool Write(const char* s, const char* tag) = 0; virtual bool Write(const char* buf, int len, const char* tag) = 0; virtual bool Write(const string& s, const char* tag) = 0; + virtual bool Write(const IPAddr& addr, const char* tag) = 0; + virtual bool Write(const IPPrefix& prefix, const char* tag) = 0; + virtual bool Write(const struct in_addr& addr, const char* tag) = 0; + virtual bool Write(const struct in6_addr& addr, const char* tag) = 0; virtual bool WriteOpenTag(const char* tag) = 0; virtual bool WriteCloseTag(const char* tag) = 0; @@ -90,6 +101,10 @@ public: virtual bool Read(double* d, const char* tag); virtual bool Read(char** str, int* len, const char* tag); virtual bool Read(string* s, const char* tag); + virtual bool Read(IPAddr* addr, const char* tag); + virtual bool Read(IPPrefix* prefix, const char* tag); + virtual bool Read(struct in_addr* addr, const char* tag); + virtual bool Read(struct in6_addr* addr, const char* tag); virtual bool Write(int v, const char* tag); virtual bool Write(uint16 v, const char* tag); virtual bool Write(uint32 v, const char* tag); @@ -101,6 +116,10 @@ public: virtual bool Write(const char* s, const char* tag); virtual bool Write(const char* buf, int len, const char* tag); virtual bool Write(const string& s, const char* tag); + virtual bool Write(const IPAddr& addr, const char* tag); + virtual bool Write(const IPPrefix& prefix, const char* tag); + virtual bool Write(const struct in_addr& addr, const char* tag); + virtual bool Write(const struct in6_addr& addr, const char* tag); virtual bool WriteOpenTag(const char* tag); virtual bool WriteCloseTag(const char* tag); virtual bool WriteSeparator(); @@ -123,6 +142,10 @@ public: virtual bool Write(const char* s, const char* tag); virtual bool Write(const char* buf, int len, const char* tag); virtual bool Write(const string& s, const char* tag); + virtual bool Write(const IPAddr& addr, const char* tag); + virtual bool Write(const IPPrefix& prefix, const char* tag); + virtual bool Write(const struct in_addr& addr, const char* tag); + virtual bool Write(const struct in6_addr& addr, const char* tag); virtual bool WriteOpenTag(const char* tag); virtual bool WriteCloseTag(const char* tag); virtual bool WriteSeparator(); @@ -138,6 +161,10 @@ public: virtual bool Read(double* d, const char* tag); virtual bool Read(char** str, int* len, const char* tag); virtual bool Read(string* s, const char* tag); + virtual bool Read(IPAddr* addr, const char* tag); + virtual bool Read(IPPrefix* prefix, const char* tag); + virtual bool Read(struct in_addr* addr, const char* tag); + virtual bool Read(struct in6_addr* addr, const char* tag); private: // Encodes non-printable characters. diff --git a/src/Serializer.cc b/src/Serializer.cc index b82da0aaf3..06bbf73f48 100644 --- a/src/Serializer.cc +++ b/src/Serializer.cc @@ -1103,9 +1103,9 @@ void EventPlayer::Process() void Packet::Describe(ODesc* d) const { const IP_Hdr ip = IP(); - d->Add(dotted_addr(ip.SrcAddr())); + d->Add(ip.SrcAddr()); d->Add("->"); - d->Add(dotted_addr(ip.DstAddr())); + d->Add(ip.DstAddr()); } bool Packet::Serialize(SerialInfo* info) const diff --git a/src/Serializer.h b/src/Serializer.h index 93581d83ce..e7396cb7f8 100644 --- a/src/Serializer.h +++ b/src/Serializer.h @@ -69,6 +69,8 @@ public: { return format->Read(const_cast(str), len, tag); } bool Read(string* s, const char* tag); + bool Read(IPAddr* a, const char* tag) { return format->Read(a, tag); } + bool Read(IPPrefix* p, const char* tag) { return format->Read(p, tag); } bool Write(const char* s, const char* tag) { return format->Write(s, tag); } @@ -76,6 +78,8 @@ public: { return format->Write(buf, len, tag); } bool Write(const string& s, const char* tag) { return format->Write(s.data(), s.size(), tag); } + bool Write(const IPAddr& a, const char* tag) { return format->Write(a, tag); } + bool Write(const IPPrefix& p, const char* tag) { return format->Write(p, tag); } bool WriteOpenTag(const char* tag) { return format->WriteOpenTag(tag); } @@ -121,7 +125,7 @@ protected: // This will be increased whenever there is an incompatible change // in the data format. - static const uint32 DATA_FORMAT_VERSION = 21; + static const uint32 DATA_FORMAT_VERSION = 22; ChunkedIO* io; @@ -411,7 +415,7 @@ public: } const IP_Hdr IP() const - { return IP_Hdr((struct ip *) (pkt + hdr_size)); } + { return IP_Hdr((struct ip *) (pkt + hdr_size), true); } void Describe(ODesc* d) const; diff --git a/src/Sessions.cc b/src/Sessions.cc index 572e8dea59..7da1f088de 100644 --- a/src/Sessions.cc +++ b/src/Sessions.cc @@ -27,7 +27,6 @@ #include "InterConn.h" #include "Discard.h" #include "RuleMatcher.h" -#include "ConnCompressor.h" #include "DPM.h" #include "PacketSort.h" @@ -71,8 +70,8 @@ void TimerMgrExpireTimer::Dispatch(double t, int is_expire) NetSessions::NetSessions() { TypeList* t = new TypeList(); - t->Append(base_type(TYPE_COUNT)); // source IP address - t->Append(base_type(TYPE_COUNT)); // dest IP address + t->Append(base_type(TYPE_ADDR)); // source IP address + t->Append(base_type(TYPE_ADDR)); // dest IP address t->Append(base_type(TYPE_COUNT)); // source and dest ports ch = new CompositeHash(t); @@ -135,12 +134,12 @@ NetSessions::~NetSessions() delete SYN_OS_Fingerprinter; delete pkt_profiler; Unref(arp_analyzer); + delete discarder; + delete stp_manager; } void NetSessions::Done() { - delete stp_manager; - delete discarder; } namespace // private namespace @@ -273,24 +272,35 @@ void NetSessions::NextPacket(double t, const struct pcap_pkthdr* hdr, } const struct ip* ip = (const struct ip*) (pkt + hdr_size); + if ( ip->ip_v == 4 ) { - IP_Hdr ip_hdr(ip); + IP_Hdr ip_hdr(ip, false); DoNextPacket(t, hdr, &ip_hdr, pkt, hdr_size); } - else if ( arp_analyzer && arp_analyzer->IsARP(pkt, hdr_size) ) - arp_analyzer->NextPacket(t, hdr, pkt, hdr_size); + else if ( ip->ip_v == 6 ) + { + if ( caplen < sizeof(struct ip6_hdr) ) + { + Weird("truncated_IP", hdr, pkt); + return; + } + + IP_Hdr ip_hdr((const struct ip6_hdr*) (pkt + hdr_size), false, caplen); + DoNextPacket(t, hdr, &ip_hdr, pkt, hdr_size); + } + + else if ( ARP_Analyzer::IsARP(pkt, hdr_size) ) + { + if ( arp_analyzer ) + arp_analyzer->NextPacket(t, hdr, pkt, hdr_size); + } else { -#ifdef BROv6 - IP_Hdr ip_hdr((const struct ip6_hdr*) (pkt + hdr_size)); - DoNextPacket(t, hdr, &ip_hdr, pkt, hdr_size); -#else - Weird("non_IPv4_packet", hdr, pkt); + Weird("unknown_packet_type", hdr, pkt); return; -#endif } } @@ -329,7 +339,8 @@ void NetSessions::NextPacketSecondary(double /* t */, const struct pcap_pkthdr* StringVal* cmd_val = new StringVal(sp->Event()->Filter()); args->append(cmd_val); - args->append(BuildHeader(ip)); + IP_Hdr ip_hdr(ip, false); + args->append(ip_hdr.BuildPktHdrVal()); // ### Need to queue event here. try { @@ -397,18 +408,6 @@ int NetSessions::CheckConnectionTag(Connection* conn) return 1; } - -static bool looks_like_IPv4_packet(int len, const struct ip* ip_hdr) - { - if ( (unsigned int) len < sizeof(struct ip) ) - return false; - - if ( ip_hdr->ip_v == 4 && ntohs(ip_hdr->ip_len) == len ) - return true; - else - return false; - } - void NetSessions::DoNextPacket(double t, const struct pcap_pkthdr* hdr, const IP_Hdr* ip_hdr, const u_char* const pkt, int hdr_size) @@ -438,18 +437,9 @@ void NetSessions::DoNextPacket(double t, const struct pcap_pkthdr* hdr, if ( discarder && discarder->NextPacket(ip_hdr, len, caplen) ) return; - int proto = ip_hdr->NextProto(); - if ( proto != IPPROTO_TCP && proto != IPPROTO_UDP && - proto != IPPROTO_ICMP ) - { - dump_this_packet = 1; - return; - } - FragReassembler* f = 0; - uint32 frag_field = ip_hdr->FragField(); - if ( (frag_field & 0x3fff) != 0 ) + if ( ip_hdr->IsFragment() ) { dump_this_packet = 1; // always record fragments @@ -460,12 +450,12 @@ void NetSessions::DoNextPacket(double t, const struct pcap_pkthdr* hdr, // Don't try to reassemble, that's doomed. // Discard all except the first fragment (which // is useful in analyzing header-only traces) - if ( (frag_field & 0x1fff) != 0 ) + if ( ip_hdr->FragOffset() != 0 ) return; } else { - f = NextFragment(t, ip_hdr, pkt + hdr_size, frag_field); + f = NextFragment(t, ip_hdr, pkt + hdr_size); const IP_Hdr* ih = f->ReassembledPkt(); if ( ! ih ) // It didn't reassemble into anything yet. @@ -482,21 +472,56 @@ void NetSessions::DoNextPacket(double t, const struct pcap_pkthdr* hdr, len -= ip_hdr_len; // remove IP header caplen -= ip_hdr_len; - uint32 min_hdr_len = (proto == IPPROTO_TCP) ? sizeof(struct tcphdr) : - (proto == IPPROTO_UDP ? sizeof(struct udphdr) : ICMP_MINLEN); - - if ( len < min_hdr_len ) + // We stop building the chain when seeing IPPROTO_ESP so if it's + // there, it's always the last. + if ( ip_hdr->LastHeader() == IPPROTO_ESP ) { - Weird("truncated_header", hdr, pkt); - if ( f ) - Remove(f); // ### + dump_this_packet = 1; + if ( esp_packet ) + { + val_list* vl = new val_list(); + vl->append(ip_hdr->BuildPktHdrVal()); + mgr.QueueEvent(esp_packet, vl); + } + Remove(f); + // Can't do more since upper-layer payloads are going to be encrypted. return; } - if ( caplen < min_hdr_len ) + +#ifdef ENABLE_MOBILE_IPV6 + // We stop building the chain when seeing IPPROTO_MOBILITY so it's always + // last if present. + if ( ip_hdr->LastHeader() == IPPROTO_MOBILITY ) { - Weird("internally_truncated_header", hdr, pkt); - if ( f ) - Remove(f); // ### + dump_this_packet = 1; + + if ( ! ignore_checksums && mobility_header_checksum(ip_hdr) != 0xffff ) + { + Weird("bad_MH_checksum", hdr, pkt); + Remove(f); + return; + } + + if ( mobile_ipv6_message ) + { + val_list* vl = new val_list(); + vl->append(ip_hdr->BuildPktHdrVal()); + mgr.QueueEvent(mobile_ipv6_message, vl); + } + + if ( ip_hdr->NextProto() != IPPROTO_NONE ) + Weird("mobility_piggyback", hdr, pkt); + + Remove(f); + return; + } +#endif + + int proto = ip_hdr->NextProto(); + + if ( CheckHeaderTrunc(proto, len, caplen, hdr, pkt) ) + { + Remove(f); return; } @@ -506,7 +531,6 @@ void NetSessions::DoNextPacket(double t, const struct pcap_pkthdr* hdr, id.src_addr = ip_hdr->SrcAddr(); id.dst_addr = ip_hdr->DstAddr(); Dictionary* d = 0; - bool pass_to_conn_compressor = false; switch ( proto ) { case IPPROTO_TCP: @@ -516,7 +540,6 @@ void NetSessions::DoNextPacket(double t, const struct pcap_pkthdr* hdr, id.dst_port = tp->th_dport; id.is_one_way = 0; d = &tcp_conns; - pass_to_conn_compressor = ip4 && use_connection_compressor; break; } @@ -535,7 +558,23 @@ void NetSessions::DoNextPacket(double t, const struct pcap_pkthdr* hdr, const struct icmp* icmpp = (const struct icmp *) data; id.src_port = icmpp->icmp_type; - id.dst_port = ICMP_counterpart(icmpp->icmp_type, + id.dst_port = ICMP4_counterpart(icmpp->icmp_type, + icmpp->icmp_code, + id.is_one_way); + + id.src_port = htons(id.src_port); + id.dst_port = htons(id.dst_port); + + d = &icmp_conns; + break; + } + + case IPPROTO_ICMPV6: + { + const struct icmp* icmpp = (const struct icmp *) data; + + id.src_port = icmpp->icmp_type; + id.dst_port = ICMP6_counterpart(icmpp->icmp_type, icmpp->icmp_code, id.is_one_way); @@ -547,11 +586,12 @@ void NetSessions::DoNextPacket(double t, const struct pcap_pkthdr* hdr, } default: - Weird(fmt("unknown_protocol %d", proto), hdr, pkt); + Weird(fmt("unknown_protocol_%d", proto), hdr, pkt); + Remove(f); return; } - HashKey* h = id.BuildConnKey(); + HashKey* h = BuildConnIDHashKey(id); if ( ! h ) reporter->InternalError("hash computation failed"); @@ -559,56 +599,62 @@ void NetSessions::DoNextPacket(double t, const struct pcap_pkthdr* hdr, // FIXME: The following is getting pretty complex. Need to split up // into separate functions. - if ( pass_to_conn_compressor ) - conn = conn_compressor->NextPacket(t, h, ip_hdr, hdr, pkt); + conn = (Connection*) d->Lookup(h); + if ( ! conn ) + { + conn = NewConn(h, t, &id, data, proto); + if ( conn ) + d->Insert(h, conn); + } else { - conn = (Connection*) d->Lookup(h); - if ( ! conn ) + // We already know that connection. + int consistent = CheckConnectionTag(conn); + if ( consistent < 0 ) { + delete h; + Remove(f); + return; + } + + if ( ! consistent || conn->IsReuse(t, data) ) + { + if ( consistent ) + conn->Event(connection_reused, 0); + + Remove(conn); conn = NewConn(h, t, &id, data, proto); if ( conn ) d->Insert(h, conn); } else - { - // We already know that connection. - int consistent = CheckConnectionTag(conn); - if ( consistent < 0 ) - { - delete h; - return; - } - - if ( ! consistent || conn->IsReuse(t, data) ) - { - if ( consistent ) - conn->Event(connection_reused, 0); - - Remove(conn); - conn = NewConn(h, t, &id, data, proto); - if ( conn ) - d->Insert(h, conn); - } - else - delete h; - } - - if ( ! conn ) delete h; } if ( ! conn ) + { + delete h; + Remove(f); return; + } int record_packet = 1; // whether to record the packet at all int record_content = 1; // whether to record its data - int is_orig = addr_eq(id.src_addr, conn->OrigAddr()) && - id.src_port == conn->OrigPort(); + int is_orig = (id.src_addr == conn->OrigAddr()) && + (id.src_port == conn->OrigPort()); - if ( new_packet && ip4 ) - conn->Event(new_packet, 0, BuildHeader(ip4)); + Val* pkt_hdr_val = 0; + + if ( ipv6_ext_headers && ip_hdr->NumHeaders() > 1 ) + { + pkt_hdr_val = ip_hdr->BuildPktHdrVal(); + conn->Event(ipv6_ext_headers, 0, pkt_hdr_val); + } + + if ( new_packet ) + conn->Event(new_packet, 0, + pkt_hdr_val ? pkt_hdr_val->Ref() : ip_hdr->BuildPktHdrVal()); conn->NextPacket(t, is_orig, ip_hdr, len, caplen, data, record_packet, record_content, @@ -618,7 +664,7 @@ void NetSessions::DoNextPacket(double t, const struct pcap_pkthdr* hdr, { // Above we already recorded the fragment in its entirety. f->DeleteTimer(); - Remove(f); // ### + Remove(f); } else if ( record_packet ) @@ -634,110 +680,47 @@ void NetSessions::DoNextPacket(double t, const struct pcap_pkthdr* hdr, } } -Val* NetSessions::BuildHeader(const struct ip* ip) +bool NetSessions::CheckHeaderTrunc(int proto, uint32 len, uint32 caplen, + const struct pcap_pkthdr* h, const u_char* p) { - static RecordType* pkt_hdr_type = 0; - static RecordType* ip_hdr_type = 0; - static RecordType* tcp_hdr_type = 0; - static RecordType* udp_hdr_type = 0; - static RecordType* icmp_hdr_type; - - if ( ! pkt_hdr_type ) - { - pkt_hdr_type = internal_type("pkt_hdr")->AsRecordType(); - ip_hdr_type = internal_type("ip_hdr")->AsRecordType(); - tcp_hdr_type = internal_type("tcp_hdr")->AsRecordType(); - udp_hdr_type = internal_type("udp_hdr")->AsRecordType(); - icmp_hdr_type = internal_type("icmp_hdr")->AsRecordType(); - } - - RecordVal* pkt_hdr = new RecordVal(pkt_hdr_type); - - RecordVal* ip_hdr = new RecordVal(ip_hdr_type); - - int ip_hdr_len = ip->ip_hl * 4; - int ip_pkt_len = ntohs(ip->ip_len); - - ip_hdr->Assign(0, new Val(ip->ip_hl * 4, TYPE_COUNT)); - ip_hdr->Assign(1, new Val(ip->ip_tos, TYPE_COUNT)); - ip_hdr->Assign(2, new Val(ip_pkt_len, TYPE_COUNT)); - ip_hdr->Assign(3, new Val(ntohs(ip->ip_id), TYPE_COUNT)); - ip_hdr->Assign(4, new Val(ip->ip_ttl, TYPE_COUNT)); - ip_hdr->Assign(5, new Val(ip->ip_p, TYPE_COUNT)); - ip_hdr->Assign(6, new AddrVal(ip->ip_src.s_addr)); - ip_hdr->Assign(7, new AddrVal(ip->ip_dst.s_addr)); - - pkt_hdr->Assign(0, ip_hdr); - - // L4 header. - const u_char* data = ((const u_char*) ip) + ip_hdr_len; - - int proto = ip->ip_p; + uint32 min_hdr_len = 0; switch ( proto ) { case IPPROTO_TCP: - { - const struct tcphdr* tp = (const struct tcphdr*) data; - RecordVal* tcp_hdr = new RecordVal(tcp_hdr_type); - - int tcp_hdr_len = tp->th_off * 4; - int data_len = ip_pkt_len - ip_hdr_len - tcp_hdr_len; - - tcp_hdr->Assign(0, new PortVal(ntohs(tp->th_sport), TRANSPORT_TCP)); - tcp_hdr->Assign(1, new PortVal(ntohs(tp->th_dport), TRANSPORT_TCP)); - tcp_hdr->Assign(2, new Val(uint32(ntohl(tp->th_seq)), TYPE_COUNT)); - tcp_hdr->Assign(3, new Val(uint32(ntohl(tp->th_ack)), TYPE_COUNT)); - tcp_hdr->Assign(4, new Val(tcp_hdr_len, TYPE_COUNT)); - tcp_hdr->Assign(5, new Val(data_len, TYPE_COUNT)); - tcp_hdr->Assign(6, new Val(tp->th_flags, TYPE_COUNT)); - tcp_hdr->Assign(7, new Val(ntohs(tp->th_win), TYPE_COUNT)); - - pkt_hdr->Assign(1, tcp_hdr); + min_hdr_len = sizeof(struct tcphdr); break; - } - case IPPROTO_UDP: - { - const struct udphdr* up = (const struct udphdr*) data; - RecordVal* udp_hdr = new RecordVal(udp_hdr_type); - - udp_hdr->Assign(0, new PortVal(ntohs(up->uh_sport), TRANSPORT_UDP)); - udp_hdr->Assign(1, new PortVal(ntohs(up->uh_dport), TRANSPORT_UDP)); - udp_hdr->Assign(2, new Val(ntohs(up->uh_ulen), TYPE_COUNT)); - - pkt_hdr->Assign(2, udp_hdr); + min_hdr_len = sizeof(struct udphdr); break; - } - case IPPROTO_ICMP: - { - const struct icmp* icmpp = (const struct icmp *) data; - RecordVal* icmp_hdr = new RecordVal(icmp_hdr_type); - - icmp_hdr->Assign(0, new Val(icmpp->icmp_type, TYPE_COUNT)); - - pkt_hdr->Assign(3, icmp_hdr); - break; - } - + case IPPROTO_ICMPV6: default: - { - // This is not a protocol we understand. - } + // Use for all other packets. + min_hdr_len = ICMP_MINLEN; } - return pkt_hdr; + if ( len < min_hdr_len ) + { + Weird("truncated_header", h, p); + return true; + } + + if ( caplen < min_hdr_len ) + { + Weird("internally_truncated_header", h, p); + return true; + } + + return false; } FragReassembler* NetSessions::NextFragment(double t, const IP_Hdr* ip, - const u_char* pkt, uint32 frag_field) + const u_char* pkt) { - uint32 src_addr = uint32(ip->SrcAddr4()); - uint32 dst_addr = uint32(ip->DstAddr4()); - uint32 frag_id = ntohs(ip->ID4()); // we actually could skip conv. + uint32 frag_id = ip->ID(); ListVal* key = new ListVal(TYPE_ANY); - key->Append(new Val(src_addr, TYPE_COUNT)); - key->Append(new Val(dst_addr, TYPE_COUNT)); + key->Append(new AddrVal(ip->SrcAddr())); + key->Append(new AddrVal(ip->DstAddr())); key->Append(new Val(frag_id, TYPE_COUNT)); HashKey* h = ch->ComputeHash(key, 1); @@ -747,7 +730,7 @@ FragReassembler* NetSessions::NextFragment(double t, const IP_Hdr* ip, FragReassembler* f = fragments.Lookup(h); if ( ! f ) { - f = new FragReassembler(this, ip, pkt, frag_field, h, t); + f = new FragReassembler(this, ip, pkt, h, t); fragments.Insert(h, f); Unref(key); return f; @@ -756,7 +739,7 @@ FragReassembler* NetSessions::NextFragment(double t, const IP_Hdr* ip, delete h; Unref(key); - f->AddFragment(t, ip, pkt, frag_field); + f->AddFragment(t, ip, pkt); return f; } @@ -772,7 +755,7 @@ int NetSessions::Get_OS_From_SYN(struct os_type* retval, quirks, ECN) : 0; } -bool NetSessions::CompareWithPreviousOSMatch(uint32 addr, int id) const +bool NetSessions::CompareWithPreviousOSMatch(const IPAddr& addr, int id) const { return SYN_OS_Fingerprinter ? SYN_OS_Fingerprinter->CacheMatch(addr, id) : 0; @@ -813,44 +796,30 @@ Connection* NetSessions::FindConnection(Val* v) // types, too. } - addr_type orig_addr = (*vl)[orig_h]->AsAddr(); - addr_type resp_addr = (*vl)[resp_h]->AsAddr(); + const IPAddr& orig_addr = (*vl)[orig_h]->AsAddr(); + const IPAddr& resp_addr = (*vl)[resp_h]->AsAddr(); PortVal* orig_portv = (*vl)[orig_p]->AsPortVal(); PortVal* resp_portv = (*vl)[resp_p]->AsPortVal(); ConnID id; -#ifdef BROv6 id.src_addr = orig_addr; id.dst_addr = resp_addr; -#else - id.src_addr = &orig_addr; - id.dst_addr = &resp_addr; -#endif id.src_port = htons((unsigned short) orig_portv->Port()); id.dst_port = htons((unsigned short) resp_portv->Port()); id.is_one_way = 0; // ### incorrect for ICMP connections - HashKey* h = id.BuildConnKey(); + HashKey* h = BuildConnIDHashKey(id); if ( ! h ) reporter->InternalError("hash computation failed"); Dictionary* d; if ( orig_portv->IsTCP() ) - { - if ( use_connection_compressor ) - { - Connection* conn = conn_compressor->Lookup(h); - delete h; - return conn; - } - else - d = &tcp_conns; - } + d = &tcp_conns; else if ( orig_portv->IsUDP() ) d = &udp_conns; else if ( orig_portv->IsICMP() ) @@ -903,17 +872,7 @@ void NetSessions::Remove(Connection* c) switch ( c->ConnTransport() ) { case TRANSPORT_TCP: - if ( use_connection_compressor && - conn_compressor->Remove(k) ) - // Note, if the Remove() returned false - // then the compressor doesn't know about - // this connection, which *should* mean that - // we never gave it the connection in the - // first place, and thus we should check - // the regular TCP table instead. - ; - - else if ( ! tcp_conns.RemoveEntry(k) ) + if ( ! tcp_conns.RemoveEntry(k) ) reporter->InternalError("connection missing"); break; @@ -939,6 +898,7 @@ void NetSessions::Remove(Connection* c) void NetSessions::Remove(FragReassembler* f) { + if ( ! f ) return; HashKey* k = f->Key(); if ( ! k ) reporter->InternalError("fragment block not in dictionary"); @@ -960,13 +920,8 @@ void NetSessions::Insert(Connection* c) // reference the old key for already existing connections. case TRANSPORT_TCP: - if ( use_connection_compressor ) - old = conn_compressor->Insert(c); - else - { - old = (Connection*) tcp_conns.Remove(c->Key()); - tcp_conns.Insert(c->Key(), c); - } + old = (Connection*) tcp_conns.Remove(c->Key()); + tcp_conns.Insert(c->Key(), c); break; case TRANSPORT_UDP: @@ -998,9 +953,6 @@ void NetSessions::Insert(Connection* c) void NetSessions::Drain() { - if ( use_connection_compressor ) - conn_compressor->Drain(); - IterCookie* cookie = tcp_conns.InitForIteration(); Connection* tc; @@ -1070,6 +1022,9 @@ Connection* NetSessions::NewConn(HashKey* k, double t, const ConnID* id, case IPPROTO_UDP: tproto = TRANSPORT_UDP; break; + case IPPROTO_ICMPV6: + tproto = TRANSPORT_ICMP; + break; default: reporter->InternalError("unknown transport protocol"); break; @@ -1092,7 +1047,7 @@ Connection* NetSessions::NewConn(HashKey* k, double t, const ConnID* id, // an analyzable connection. ConnID flip_id = *id; - const uint32* ta = flip_id.src_addr; + const IPAddr ta = flip_id.src_addr; flip_id.src_addr = flip_id.dst_addr; flip_id.dst_addr = ta; @@ -1113,10 +1068,7 @@ Connection* NetSessions::NewConn(HashKey* k, double t, const ConnID* id, conn->AppendAddl(fmt("tag=%s", conn->GetTimerMgr()->GetTag().c_str())); - // If the connection compressor is active, it takes care of the - // new_connection/connection_external events for TCP connections. - if ( new_connection && - (tproto != TRANSPORT_TCP || ! use_connection_compressor) ) + if ( new_connection ) { conn->Event(new_connection, 0); diff --git a/src/Sessions.h b/src/Sessions.h index 452de874db..06c6057dbf 100644 --- a/src/Sessions.h +++ b/src/Sessions.h @@ -79,7 +79,7 @@ public: // Returns a reassembled packet, or nil if there are still // some missing fragments. FragReassembler* NextFragment(double t, const IP_Hdr* ip, - const u_char* pkt, uint32 frag_field); + const u_char* pkt); int Get_OS_From_SYN(struct os_type* retval, uint16 tot, uint8 DF_flag, uint8 TTL, uint16 WSS, @@ -87,7 +87,7 @@ public: uint32 tstamp, /* uint8 TOS, */ uint32 quirks, uint8 ECN) const; - bool CompareWithPreviousOSMatch(uint32 addr, int id) const; + bool CompareWithPreviousOSMatch(const IPAddr& addr, int id) const; // Looks up the connection referred to by the given Val, // which should be a conn_id record. Returns nil if there's @@ -190,10 +190,11 @@ protected: void Internal(const char* msg, const struct pcap_pkthdr* hdr, const u_char* pkt); - // Builds a record encapsulating a packet. This should be more - // general, including the equivalent of a union of tcp/udp/icmp - // headers . - Val* BuildHeader(const struct ip* ip); + // For a given protocol, checks whether the header's length as derived + // from lower-level headers or the length actually captured is less + // than that protocol's minimum header size. + bool CheckHeaderTrunc(int proto, uint32 len, uint32 caplen, + const struct pcap_pkthdr* hdr, const u_char* pkt); CompositeHash* ch; PDict(Connection) tcp_conns; diff --git a/src/StateAccess.cc b/src/StateAccess.cc index 136a006c52..2d0a8dfc5a 100644 --- a/src/StateAccess.cc +++ b/src/StateAccess.cc @@ -231,7 +231,7 @@ bool StateAccess::CheckOldSet(const char* op, ID* id, Val* index, bool StateAccess::MergeTables(TableVal* dst, Val* src) { - if ( ! src->Type()->Tag() == TYPE_TABLE ) + if ( src->Type()->Tag() != TYPE_TABLE ) { reporter->Error("type mismatch while merging tables"); return false; @@ -678,7 +678,7 @@ bool StateAccess::DoUnserialize(UnserialInfo* info) target.id = new ID(name, SCOPE_GLOBAL, true); Ref(target.id); global_scope()->Insert(name, target.id); -#ifdef USE_PERFTOOLS +#ifdef USE_PERFTOOLS_DEBUG heap_checker->IgnoreObject(target.id); #endif } diff --git a/src/Stats.cc b/src/Stats.cc index 55835613e9..c3035231e9 100644 --- a/src/Stats.cc +++ b/src/Stats.cc @@ -6,10 +6,9 @@ #include "Stats.h" #include "Scope.h" #include "cq.h" -#include "ConnCompressor.h" #include "DNS_Mgr.h" #include "Trigger.h" - +#include "threading/Manager.h" int killed_by_inactivity = 0; @@ -129,19 +128,6 @@ void ProfileLogger::Log() expensive ? sessions->ConnectionMemoryUsageConnVals() / 1024 : 0 )); - const ConnCompressor::Sizes& cs = conn_compressor->Size(); - - file->Write(fmt("%.6f ConnCompressor: pending=%d pending_in_mem=%d full_conns=%d pending+real=%d mem=%dK avg=%.1f/%.1f\n", - network_time, - cs.pending_valid, - cs.pending_in_mem, - cs.connections, - cs.hash_table_size, - cs.memory / 1024, - cs.memory / double(cs.pending_valid), - cs.memory / double(cs.pending_in_mem) - )); - SessionStats s; sessions->GetStats(s); @@ -217,6 +203,25 @@ void ProfileLogger::Log() current_timers[i])); } + file->Write(fmt("%0.6f Threads: current=%d\n", network_time, thread_mgr->NumThreads())); + + const threading::Manager::msg_stats_list& thread_stats = thread_mgr->GetMsgThreadStats(); + for ( threading::Manager::msg_stats_list::const_iterator i = thread_stats.begin(); + i != thread_stats.end(); ++i ) + { + threading::MsgThread::Stats s = i->second; + file->Write(fmt("%0.6f %-25s in=%" PRIu64 " out=%" PRIu64 " pending=%" PRIu64 "/%" PRIu64 + " (#queue r/w: in=%" PRIu64 "/%" PRIu64 " out=%" PRIu64 "/%" PRIu64 ")" + "\n", + network_time, + i->first.c_str(), + s.sent_in, s.sent_out, + s.pending_in, s.pending_out, + s.queue_in_stats.num_reads, s.queue_in_stats.num_writes, + s.queue_out_stats.num_reads, s.queue_out_stats.num_writes + )); + } + // Script-level state. unsigned int size, mem = 0; PDict(ID)* globals = global_scope()->Vars(); diff --git a/src/TCP.cc b/src/TCP.cc index 0fae07a24d..57e4449bf8 100644 --- a/src/TCP.cc +++ b/src/TCP.cc @@ -276,7 +276,7 @@ void TCP_Analyzer::ProcessSYN(const IP_Hdr* ip, const struct tcphdr* tp, uint32 tcp_hdr_len, int& seq_len, TCP_Endpoint* endpoint, TCP_Endpoint* peer, uint32 base_seq, uint32 ack_seq, - const uint32* orig_addr, + const IPAddr& orig_addr, int is_orig, TCP_Flags flags) { int len = seq_len; @@ -346,7 +346,7 @@ void TCP_Analyzer::ProcessSYN(const IP_Hdr* ip, const struct tcphdr* tp, // is_orig will be removed once we can do SYN-ACK fingerprinting. if ( OS_version_found && is_orig ) { - Val src_addr_val(orig_addr, TYPE_ADDR); + AddrVal src_addr_val(orig_addr); if ( generate_OS_version_event->Size() == 0 || generate_OS_version_event->Lookup(&src_addr_val) ) { @@ -414,7 +414,7 @@ int TCP_Analyzer::ProcessFlags(double t, uint32 tcp_hdr_len, int len, int& seq_len, TCP_Endpoint* endpoint, TCP_Endpoint* peer, uint32 base_seq, uint32 ack_seq, - const uint32* orig_addr, + const IPAddr& orig_addr, int is_orig, TCP_Flags flags) { if ( flags.SYN() ) @@ -989,8 +989,7 @@ void TCP_Analyzer::DeliverPacket(int len, const u_char* data, bool is_orig, if ( ! orig->did_close || ! resp->did_close ) Conn()->SetLastTime(t); - const uint32* orig_addr = Conn()->OrigAddr(); - const uint32* resp_addr = Conn()->RespAddr(); + const IPAddr orig_addr = Conn()->OrigAddr(); uint32 tcp_hdr_len = data - (const u_char*) tp; @@ -1204,7 +1203,7 @@ RecordVal* TCP_Analyzer::BuildOSVal(int is_orig, const IP_Hdr* ip, if ( ip->HdrLen() > 20 ) quirks |= QUIRK_IPOPT; - if ( ip->IP_ID() == 0 ) + if ( ip->ID() == 0 ) quirks |= QUIRK_ZEROID; if ( tcp->th_seq == 0 ) @@ -1331,7 +1330,7 @@ RecordVal* TCP_Analyzer::BuildOSVal(int is_orig, const IP_Hdr* ip, tstamp, quirks, uint8(tcp->th_flags & (TH_ECE|TH_CWR))); - if ( sessions->CompareWithPreviousOSMatch(ip->SrcAddr4(), id) ) + if ( sessions->CompareWithPreviousOSMatch(ip->SrcAddr(), id) ) { RecordVal* os = new RecordVal(OS_version); @@ -1943,11 +1942,11 @@ int TCPStats_Endpoint::DataSent(double /* t */, int seq, int len, int caplen, { if ( ++num_pkts == 1 ) { // First packet. - last_id = ntohs(ip->ID4()); + last_id = ip->ID(); return 0; } - int id = ntohs(ip->ID4()); + int id = ip->ID(); if ( id == last_id ) { diff --git a/src/TCP.h b/src/TCP.h index 65f437856a..c84202fcf6 100644 --- a/src/TCP.h +++ b/src/TCP.h @@ -6,6 +6,7 @@ #include "Analyzer.h" #include "TCP.h" #include "PacketDumper.h" +#include "IPAddr.h" // We define two classes here: // - TCP_Analyzer is the analyzer for the TCP protocol itself. @@ -128,7 +129,7 @@ protected: uint32 tcp_hdr_len, int& seq_len, TCP_Endpoint* endpoint, TCP_Endpoint* peer, uint32 base_seq, uint32 ack_seq, - const uint32* orig_addr, + const IPAddr& orig_addr, int is_orig, TCP_Flags flags); void ProcessFIN(double t, TCP_Endpoint* endpoint, int& seq_len, @@ -144,7 +145,7 @@ protected: uint32 tcp_hdr_len, int len, int& seq_len, TCP_Endpoint* endpoint, TCP_Endpoint* peer, uint32 base_seq, uint32 ack_seq, - const uint32* orig_addr, + const IPAddr& orig_addr, int is_orig, TCP_Flags flags); void TransitionFromInactive(double t, TCP_Endpoint* endpoint, diff --git a/src/TCP_Endpoint.cc b/src/TCP_Endpoint.cc index 5a65a18d7c..69c08870d9 100644 --- a/src/TCP_Endpoint.cc +++ b/src/TCP_Endpoint.cc @@ -32,13 +32,8 @@ TCP_Endpoint::TCP_Endpoint(TCP_Analyzer* arg_analyzer, int arg_is_orig) dst_addr = is_orig ? tcp_analyzer->Conn()->OrigAddr() : tcp_analyzer->Conn()->RespAddr(); -#ifdef BROv6 - checksum_base = ones_complement_checksum((void*) src_addr, 16, 0); - checksum_base = ones_complement_checksum((void*) dst_addr, 16, checksum_base); -#else - checksum_base = ones_complement_checksum((void*) src_addr, 4, 0); - checksum_base = ones_complement_checksum((void*) dst_addr, 4, checksum_base); -#endif + checksum_base = ones_complement_checksum(src_addr, 0); + checksum_base = ones_complement_checksum(dst_addr, checksum_base); // Note, for IPv6, strictly speaking this field is 32 bits // rather than 16 bits. But because the upper bits are all zero, // we get the same checksum either way. The same applies to diff --git a/src/TCP_Endpoint.h b/src/TCP_Endpoint.h index 758a504ff5..52a757b256 100644 --- a/src/TCP_Endpoint.h +++ b/src/TCP_Endpoint.h @@ -3,6 +3,8 @@ #ifndef tcpendpoint_h #define tcpendpoint_h +#include "IPAddr.h" + typedef enum { TCP_ENDPOINT_INACTIVE, // no SYN (or other packets) seen for this side TCP_ENDPOINT_SYN_SENT, // SYN seen, but no ack @@ -128,8 +130,8 @@ public: uint32 checksum_base; double start_time, last_time; - const uint32* src_addr; // the other endpoint - const uint32* dst_addr; // this endpoint + IPAddr src_addr; // the other endpoint + IPAddr dst_addr; // this endpoint uint32 window; // current congestion window (*scaled*, not pre-scaling) int window_scale; // from the TCP option uint32 window_ack_seq; // at which ack_seq number did we record 'window' diff --git a/src/TCP_Reassembler.cc b/src/TCP_Reassembler.cc index ba31ab68d0..fb67dba7ee 100644 --- a/src/TCP_Reassembler.cc +++ b/src/TCP_Reassembler.cc @@ -29,7 +29,7 @@ TCP_Reassembler::TCP_Reassembler(Analyzer* arg_dst_analyzer, TCP_Analyzer* arg_tcp_analyzer, TCP_Reassembler::Type arg_type, bool arg_is_orig, TCP_Endpoint* arg_endp) -: Reassembler(1, arg_endp->dst_addr, REASSEM_TCP) + : Reassembler(1, REASSEM_TCP) { dst_analyzer = arg_dst_analyzer; tcp_analyzer = arg_tcp_analyzer; diff --git a/src/Trigger.cc b/src/Trigger.cc index 26f80c73d9..164f11b885 100644 --- a/src/Trigger.cc +++ b/src/Trigger.cc @@ -410,7 +410,7 @@ Val* Trigger::Lookup(const CallExpr* expr) return (i != cache.end()) ? i->second : 0; } -const char* Trigger::Name() +const char* Trigger::Name() const { assert(location); return fmt("%s:%d-%d", location->filename, diff --git a/src/Trigger.h b/src/Trigger.h index ffec50d7ef..8e04fb9189 100644 --- a/src/Trigger.h +++ b/src/Trigger.h @@ -60,7 +60,7 @@ public: virtual void Access(Val* val, const StateAccess& sa) { QueueTrigger(this); } - virtual const char* Name(); + virtual const char* Name() const; static void QueueTrigger(Trigger* trigger); diff --git a/src/Type.cc b/src/Type.cc index 4d80eda6f7..82221303af 100644 --- a/src/Type.cc +++ b/src/Type.cc @@ -1858,13 +1858,8 @@ BroType* merge_types(const BroType* t1, const BroType* t2) if ( t1->IsSet() ) return new SetType(tl3, 0); - else if ( tg1 == TYPE_TABLE ) - return new TableType(tl3, y3); else - { - reporter->InternalError("bad tag in merge_types"); - return 0; - } + return new TableType(tl3, y3); } case TYPE_FUNC: diff --git a/src/UDP.cc b/src/UDP.cc index 35e9f58388..d85cb39edd 100644 --- a/src/UDP.cc +++ b/src/UDP.cc @@ -57,15 +57,15 @@ void UDP_Analyzer::DeliverPacket(int len, const u_char* data, bool is_orig, { bool bad = false; - if ( ip->IP4_Hdr() && chksum && - udp_checksum(ip->IP4_Hdr(), up, len) != 0xffff ) - bad = true; + if ( ip->IP4_Hdr() ) + { + if ( chksum && ! ValidateChecksum(ip, up, len) ) + bad = true; + } -#ifdef BROv6 - if ( ip->IP6_Hdr() && /* checksum is not optional for IPv6 */ - udp6_checksum(ip->IP6_Hdr(), up, len) != 0xffff ) + /* checksum is not optional for IPv6 */ + else if ( ! ValidateChecksum(ip, up, len) ) bad = true; -#endif if ( bad ) { @@ -206,4 +206,24 @@ unsigned int UDP_Analyzer::MemoryAllocation() const return Analyzer::MemoryAllocation() + padded_sizeof(*this) - 24; } +bool UDP_Analyzer::ValidateChecksum(const IP_Hdr* ip, const udphdr* up, int len) + { + uint32 sum; + if ( len % 2 == 1 ) + // Add in pad byte. + sum = htons(((const u_char*) up)[len - 1] << 8); + else + sum = 0; + + sum = ones_complement_checksum(ip->SrcAddr(), sum); + sum = ones_complement_checksum(ip->DstAddr(), sum); + // Note, for IPv6, strictly speaking the protocol and length fields are + // 32 bits rather than 16 bits. But because the upper bits are all zero, + // we get the same checksum either way. + sum += htons(IPPROTO_UDP); + sum += htons((unsigned short) len); + sum = ones_complement_checksum((void*) up, len, sum); + + return sum == 0xffff; + } diff --git a/src/UDP.h b/src/UDP.h index 5124adf4cd..b93d4da97f 100644 --- a/src/UDP.h +++ b/src/UDP.h @@ -4,6 +4,7 @@ #define udp_h #include "Analyzer.h" +#include typedef enum { UDP_INACTIVE, // no packet seen @@ -31,6 +32,10 @@ protected: virtual bool IsReuse(double t, const u_char* pkt); virtual unsigned int MemoryAllocation() const; + // Returns true if the checksum is valid, false if not + static bool ValidateChecksum(const IP_Hdr* ip, const struct udphdr* up, + int len); + bro_int_t request_len, reply_len; private: diff --git a/src/Val.cc b/src/Val.cc index 1ec54d36cd..f3977789f2 100644 --- a/src/Val.cc +++ b/src/Val.cc @@ -25,7 +25,7 @@ #include "PrefixTable.h" #include "Conn.h" #include "Reporter.h" - +#include "IPAddr.h" Val::Val(Func* f) { @@ -205,29 +205,10 @@ bool Val::DoSerialize(SerialInfo* info) const val.string_val->Len()); case TYPE_INTERNAL_ADDR: - return SERIALIZE(NUM_ADDR_WORDS) -#ifdef BROv6 - && SERIALIZE(uint32(ntohl(val.addr_val[0]))) - && SERIALIZE(uint32(ntohl(val.addr_val[1]))) - && SERIALIZE(uint32(ntohl(val.addr_val[2]))) - && SERIALIZE(uint32(ntohl(val.addr_val[3]))); -#else - && SERIALIZE(uint32(ntohl(val.addr_val))); -#endif + return SERIALIZE(*val.addr_val); case TYPE_INTERNAL_SUBNET: - return info->s->WriteOpenTag("subnet") - && SERIALIZE(NUM_ADDR_WORDS) -#ifdef BROv6 - && SERIALIZE(uint32(ntohl(val.subnet_val.net[0]))) - && SERIALIZE(uint32(ntohl(val.subnet_val.net[1]))) - && SERIALIZE(uint32(ntohl(val.subnet_val.net[2]))) - && SERIALIZE(uint32(ntohl(val.subnet_val.net[3]))) -#else - && SERIALIZE(uint32(ntohl(val.subnet_val.net))) -#endif - && SERIALIZE(val.subnet_val.width) - && info->s->WriteCloseTag("subnet"); + return SERIALIZE(*val.subnet_val); case TYPE_INTERNAL_OTHER: // Derived classes are responsible for this. @@ -294,94 +275,15 @@ bool Val::DoUnserialize(UnserialInfo* info) case TYPE_INTERNAL_ADDR: { - int num_words; - if ( ! UNSERIALIZE(&num_words) ) - return false; - - if ( num_words != 1 && num_words != 4 ) - { - info->s->Error("bad address type"); - return false; - } - - uint32 a[4]; // big enough to hold either - - for ( int i = 0; i < num_words; ++i ) - { - if ( ! UNSERIALIZE(&a[i]) ) - return false; - - a[i] = htonl(a[i]); - } - -#ifndef BROv6 - if ( num_words == 4 ) - { - if ( a[0] || a[1] || a[2] ) - info->s->Warning("received IPv6 address, ignoring"); - ((AddrVal*) this)->Init(a[3]); - } - else - ((AddrVal*) this)->Init(a[0]); -#else - if ( num_words == 1 ) - ((AddrVal*) this)->Init(a[0]); - else - ((AddrVal*) this)->Init(a); -#endif + val.addr_val = new IPAddr(); + return UNSERIALIZE(val.addr_val); } - return true; case TYPE_INTERNAL_SUBNET: { - int num_words; - if ( ! UNSERIALIZE(&num_words) ) - return false; - - if ( num_words != 1 && num_words != 4 ) - { - info->s->Error("bad subnet type"); - return false; - } - - uint32 a[4]; // big enough to hold either - - for ( int i = 0; i < num_words; ++i ) - { - if ( ! UNSERIALIZE(&a[i]) ) - return false; - - a[i] = htonl(a[i]); - } - - int width; - if ( ! UNSERIALIZE(&width) ) - return false; - -#ifdef BROv6 - if ( num_words == 1 ) - { - a[3] = a[0]; - a[0] = a[1] = a[2] = 0; - } - - ((SubNetVal*) this)->Init(a, width); - -#else - if ( num_words == 4 ) - { - if ( a[0] || a[1] || a[2] ) - info->s->Warning("received IPv6 subnet, ignoring"); - a[0] = a[3]; - - if ( width > 32 ) - width -= 96; - } - - ((SubNetVal*) this)->Init(a[0], width); -#endif + val.subnet_val = new IPPrefix(); + return UNSERIALIZE(val.subnet_val); } - return true; case TYPE_INTERNAL_OTHER: // Derived classes are responsible for this. @@ -590,12 +492,10 @@ void Val::ValDescribe(ODesc* d) const case TYPE_INTERNAL_UNSIGNED: d->Add(val.uint_val); break; case TYPE_INTERNAL_DOUBLE: d->Add(val.double_val); break; case TYPE_INTERNAL_STRING: d->AddBytes(val.string_val); break; - case TYPE_INTERNAL_ADDR: d->Add(dotted_addr(val.addr_val)); break; + case TYPE_INTERNAL_ADDR: d->Add(val.addr_val->AsString().c_str()); break; case TYPE_INTERNAL_SUBNET: - d->Add(dotted_addr(val.subnet_val.net)); - d->Add("/"); - d->Add(val.subnet_val.width); + d->Add(val.subnet_val->AsString().c_str()); break; case TYPE_INTERNAL_ERROR: d->AddCS("error"); break; @@ -706,7 +606,8 @@ ID* MutableVal::Bind() const ip = htonl(0x7f000001); // 127.0.0.1 safe_snprintf(name, MAX_NAME_SIZE, "#%s#%d#", - dotted_addr(ip), getpid()); + IPAddr(IPv4, &ip, IPAddr::Network)->AsString().c_str(), + getpid()); #else safe_snprintf(name, MAX_NAME_SIZE, "#%s#%d#", host, getpid()); #endif @@ -957,92 +858,41 @@ bool PortVal::DoUnserialize(UnserialInfo* info) AddrVal::AddrVal(const char* text) : Val(TYPE_ADDR) { - const char* colon = strchr(text, ':'); - - if ( colon ) - { -#ifdef BROv6 - Init(dotted_to_addr6(text)); -#else - reporter->Error("bro wasn't compiled with IPv6 support"); - Init(uint32(0)); -#endif - } - - else - Init(dotted_to_addr(text)); + val.addr_val = new IPAddr(text); } AddrVal::AddrVal(uint32 addr) : Val(TYPE_ADDR) { // ### perhaps do gethostbyaddr here? - Init(addr); + val.addr_val = new IPAddr(IPv4, &addr, IPAddr::Network); } -AddrVal::AddrVal(const uint32* addr) : Val(TYPE_ADDR) +AddrVal::AddrVal(const uint32 addr[4]) : Val(TYPE_ADDR) { - Init(addr); + val.addr_val = new IPAddr(IPv6, addr, IPAddr::Network); + } + +AddrVal::AddrVal(const IPAddr& addr) : Val(TYPE_ADDR) + { + val.addr_val = new IPAddr(addr); } AddrVal::~AddrVal() { -#ifdef BROv6 - delete [] val.addr_val; -#endif - } - -Val* AddrVal::SizeVal() const - { - uint32 addr; - -#ifdef BROv6 - if ( ! is_v4_addr(val.addr_val) ) - { - Error("|addr| for IPv6 addresses not supported"); - return new Val(0, TYPE_COUNT); - } - - addr = to_v4_addr(val.addr_val); -#else - addr = val.addr_val; -#endif - - addr = ntohl(addr); - - return new Val(addr, TYPE_COUNT); - } - -void AddrVal::Init(uint32 addr) - { -#ifdef BROv6 - val.addr_val = new uint32[4]; - val.addr_val[0] = val.addr_val[1] = val.addr_val[2] = 0; - val.addr_val[3] = addr; -#else - val.addr_val = addr; -#endif - } - -void AddrVal::Init(const uint32* addr) - { -#ifdef BROv6 - val.addr_val = new uint32[4]; - val.addr_val[0] = addr[0]; - val.addr_val[1] = addr[1]; - val.addr_val[2] = addr[2]; - val.addr_val[3] = addr[3]; -#else - val.addr_val = addr[0]; -#endif + delete val.addr_val; } unsigned int AddrVal::MemoryAllocation() const { -#ifdef BROv6 - return padded_sizeof(*this) + pad_size(4 * sizeof(uint32)); -#else - return padded_sizeof(*this); -#endif + return padded_sizeof(*this) + val.addr_val->MemoryAllocation(); + } + +Val* AddrVal::SizeVal() const + { + if ( val.addr_val->GetFamily() == IPv4 ) + return new Val(32, TYPE_COUNT); + else + return new Val(128, TYPE_COUNT); } IMPLEMENT_SERIAL(AddrVal, SER_ADDR_VAL); @@ -1059,209 +909,109 @@ bool AddrVal::DoUnserialize(UnserialInfo* info) return true; } -static uint32 parse_dotted(const char* text, int& dots) - { - int addr[4]; - uint32 a = 0; - dots = 0; - - if ( sscanf(text, "%d.%d.%d.%d", addr+0, addr+1, addr+2, addr+3) == 4 ) - { - a = (addr[0] << 24) | (addr[1] << 16) | - (addr[2] << 8) | addr[3]; - dots = 3; - } - - else if ( sscanf(text, "%d.%d.%d", addr+0, addr+1, addr+2) == 3 ) - { - a = (addr[0] << 24) | (addr[1] << 16) | (addr[2] << 8); - dots = 2; - } - - else if ( sscanf(text, "%d.%d", addr+0, addr+1) == 2 ) - { - a = (addr[0] << 24) | (addr[1] << 16); - dots = 1; - } - - else - reporter->InternalError("scanf failed in parse_dotted()"); - - for ( int i = 0; i <= dots; ++i ) - { - if ( addr[i] < 0 || addr[i] > 255 ) - { - reporter->Error("bad dotted address %s", text); - break; - } - } - - return a; - } - SubNetVal::SubNetVal(const char* text) : Val(TYPE_SUBNET) { - const char* sep = strchr(text, '/'); - if ( ! sep ) - Internal("separator missing in SubNetVal::SubNetVal"); + string s(text); + size_t slash_loc = s.find('/'); - Init(text, atoi(sep+1)); + if ( slash_loc == string::npos ) + { + reporter->Error("Bad string in SubNetVal ctor: %s", text); + val.subnet_val = new IPPrefix(); + } + else + { + val.subnet_val = new IPPrefix(s.substr(0, slash_loc), + atoi(s.substr(slash_loc + 1).c_str())); + } } SubNetVal::SubNetVal(const char* text, int width) : Val(TYPE_SUBNET) { - Init(text, width); + val.subnet_val = new IPPrefix(text, width); } SubNetVal::SubNetVal(uint32 addr, int width) : Val(TYPE_SUBNET) { - Init(addr, width); + IPAddr a(IPv4, &addr, IPAddr::Network); + val.subnet_val = new IPPrefix(a, width); } -#ifdef BROv6 SubNetVal::SubNetVal(const uint32* addr, int width) : Val(TYPE_SUBNET) { - Init(addr, width); - } -#endif - -void SubNetVal::Init(const char* text, int width) - { -#ifdef BROv6 - if ( width <= 0 || width > 128 ) -#else - if ( width <= 0 || width > 32 ) -#endif - Error("bad subnet width"); - - int dots; - uint32 a = parse_dotted(text, dots); - - Init(uint32(htonl(a)), width); + IPAddr a(IPv6, addr, IPAddr::Network); + val.subnet_val = new IPPrefix(a, width); } - -void SubNetVal::Init(uint32 addr, int width) +SubNetVal::SubNetVal(const IPAddr& addr, int width) : Val(TYPE_SUBNET) { -#ifdef BROv6 - Internal("SubNetVal::Init called on 4-byte address w/ BROv6"); -#else - val.subnet_val.net = mask_addr(addr, uint32(width)); - val.subnet_val.width = width; -#endif + val.subnet_val = new IPPrefix(addr, width); } -void SubNetVal::Init(const uint32* addr, int width) +SubNetVal::~SubNetVal() { -#ifdef BROv6 - const uint32* a = mask_addr(addr, uint32(width)); + delete val.subnet_val; + } - val.subnet_val.net[0] = a[0]; - val.subnet_val.net[1] = a[1]; - val.subnet_val.net[2] = a[2]; - val.subnet_val.net[3] = a[3]; +const IPAddr& SubNetVal::Prefix() const + { + return val.subnet_val->Prefix(); + } - if ( is_v4_addr(addr) && width <= 32 ) - val.subnet_val.width = width + 96; - else - val.subnet_val.width = width; -#else - Internal("SubNetVal::Init called on 16-byte address w/o BROv6"); -#endif +int SubNetVal::Width() const + { + return val.subnet_val->Length(); + } + +unsigned int SubNetVal::MemoryAllocation() const + { + return padded_sizeof(*this) + val.subnet_val->MemoryAllocation(); } Val* SubNetVal::SizeVal() const { - int retained; -#ifdef BROv6 - retained = 128 - Width(); -#else - retained = 32 - Width(); -#endif - + int retained = 128 - val.subnet_val->LengthIPv6(); return new Val(pow(2.0, double(retained)), TYPE_DOUBLE); } void SubNetVal::ValDescribe(ODesc* d) const { - d->Add(dotted_addr(val.subnet_val.net, d->Style() == ALTERNATIVE_STYLE)); - d->Add("/"); -#ifdef BROv6 - if ( is_v4_addr(val.subnet_val.net) ) - d->Add(val.subnet_val.width - 96); - else -#endif - d->Add(val.subnet_val.width); + d->Add(string(*val.subnet_val).c_str()); } -addr_type SubNetVal::Mask() const +IPAddr SubNetVal::Mask() const { - if ( val.subnet_val.width == 0 ) + if ( val.subnet_val->Length() == 0 ) { // We need to special-case a mask width of zero, since // the compiler doesn't guarantee that 1 << 32 yields 0. -#ifdef BROv6 - uint32* m = new uint32[4]; - for ( int i = 0; i < 4; ++i ) + uint32 m[4]; + for ( unsigned int i = 0; i < 4; ++i ) m[i] = 0; - - return m; -#else - return 0; -#endif + IPAddr rval(IPv6, m, IPAddr::Host); + return rval; } -#ifdef BROv6 - uint32* m = new uint32[4]; + uint32 m[4]; uint32* mp = m; uint32 w; - for ( w = val.subnet_val.width; w >= 32; w -= 32 ) - *(mp++) = 0xffffffff; + for ( w = val.subnet_val->Length(); w >= 32; w -= 32 ) + *(mp++) = 0xffffffff; *mp = ~((1 << (32 - w)) - 1); while ( ++mp < m + 4 ) - *mp = 0; + *mp = 0; - return m; - -#else - return ~((1 << (32 - val.subnet_val.width)) - 1); -#endif + IPAddr rval(IPv6, m, IPAddr::Host); + return rval; } -bool SubNetVal::Contains(const uint32 addr) const +bool SubNetVal::Contains(const IPAddr& addr) const { -#ifdef BROv6 - Internal("SubNetVal::Contains called on 4-byte address w/ BROv6"); - return false; -#else - return ntohl(val.subnet_val.net) == (ntohl(addr) & Mask()); -#endif - } - -bool SubNetVal::Contains(const uint32* addr) const - { -#ifdef BROv6 - const uint32* net = val.subnet_val.net; - const uint32* a = addr; - uint32 m; - - for ( m = val.subnet_val.width; m > 32; m -= 32 ) - { - if ( *net != *a ) - return false; - - ++net; - ++a; - } - - uint32 mask = ~((1 << (32 - m)) - 1); - return ntohl(*net) == (ntohl(*a) & mask); -#else - return Contains(addr[3]); -#endif + IPAddr a(addr); + return val.subnet_val->Contains(a); } IMPLEMENT_SERIAL(SubNetVal, SER_SUBNET_VAL); @@ -2386,10 +2136,13 @@ void TableVal::DoExpire(double t) (v = tbl->NextEntry(k, expire_cookie)); ++i ) { if ( v->ExpireAccessTime() == 0 ) + { // This happens when we insert val while network_time - // hasn't been initialized yet (e.g. in bro_init()). - // We correct the timestamp now. - v->SetExpireAccess(network_time); + // hasn't been initialized yet (e.g. in bro_init()), and + // also when bro_start_network_time hasn't been initialized + // (e.g. before first packet). The expire_access_time is + // correct, so we just need to wait. + } else if ( v->ExpireAccessTime() + expire_time < t ) { @@ -3476,20 +3229,10 @@ int same_atomic_val(const Val* v1, const Val* v2) return v1->InternalDouble() == v2->InternalDouble(); case TYPE_INTERNAL_STRING: return Bstr_eq(v1->AsString(), v2->AsString()); - case TYPE_INTERNAL_ADDR: - { - const addr_type& a1 = v1->AsAddr(); - const addr_type& a2 = v2->AsAddr(); -#ifdef BROv6 - return addr_eq(a1, a2); -#else - return addr_eq(&a1, &a2); -#endif - } - + return v1->AsAddr() == v2->AsAddr(); case TYPE_INTERNAL_SUBNET: - return subnet_eq(v1->AsSubNet(), v2->AsSubNet()); + return v1->AsSubNet() == v2->AsSubNet(); default: reporter->InternalError("same_atomic_val called for non-atomic value"); diff --git a/src/Val.h b/src/Val.h index d851be311b..ed79f04671 100644 --- a/src/Val.h +++ b/src/Val.h @@ -18,6 +18,7 @@ #include "ID.h" #include "Scope.h" #include "StateAccess.h" +#include "IPAddr.h" class Val; class Func; @@ -53,11 +54,11 @@ typedef union { // Used for count, counter, port, subnet. bro_uint_t uint_val; - // Used for addr, net - addr_type addr_val; + // Used for addr + IPAddr* addr_val; // Used for subnet - subnet_type subnet_val; + IPPrefix* subnet_val; // Used for double, time, interval. double double_val; @@ -226,10 +227,10 @@ public: CONST_ACCESSOR(TYPE_PATTERN, RE_Matcher*, re_val, AsPattern) CONST_ACCESSOR(TYPE_VECTOR, vector*, vector_val, AsVector) - const subnet_type* AsSubNet() const + const IPPrefix& AsSubNet() const { CHECK_TAG(type->Tag(), TYPE_SUBNET, "Val::SubNet", type_name) - return &val.subnet_val; + return *val.subnet_val; } BroType* AsType() const @@ -238,12 +239,11 @@ public: return type; } - // ... in network byte order - const addr_type AsAddr() const + const IPAddr& AsAddr() const { if ( type->Tag() != TYPE_ADDR ) BadTag("Val::AsAddr", type_name(type->Tag())); - return val.addr_val; + return *val.addr_val; } #define ACCESSOR(tag, ctype, accessor, name) \ @@ -261,10 +261,17 @@ public: ACCESSOR(TYPE_PATTERN, RE_Matcher*, re_val, AsPattern) ACCESSOR(TYPE_VECTOR, vector*, vector_val, AsVector) - subnet_type* AsSubNet() + const IPPrefix& AsSubNet() { CHECK_TAG(type->Tag(), TYPE_SUBNET, "Val::SubNet", type_name) - return &val.subnet_val; + return *val.subnet_val; + } + + const IPAddr& AsAddr() + { + if ( type->Tag() != TYPE_ADDR ) + BadTag("Val::AsAddr", type_name(type->Tag())); + return *val.addr_val; } // Gives fast access to the bits of something that is one of @@ -282,6 +289,7 @@ public: CONVERTER(TYPE_PATTERN, PatternVal*, AsPatternVal) CONVERTER(TYPE_PORT, PortVal*, AsPortVal) CONVERTER(TYPE_SUBNET, SubNetVal*, AsSubNetVal) + CONVERTER(TYPE_ADDR, AddrVal*, AsAddrVal) CONVERTER(TYPE_TABLE, TableVal*, AsTableVal) CONVERTER(TYPE_RECORD, RecordVal*, AsRecordVal) CONVERTER(TYPE_LIST, ListVal*, AsListVal) @@ -299,6 +307,7 @@ public: CONST_CONVERTER(TYPE_PATTERN, PatternVal*, AsPatternVal) CONST_CONVERTER(TYPE_PORT, PortVal*, AsPortVal) CONST_CONVERTER(TYPE_SUBNET, SubNetVal*, AsSubNetVal) + CONST_CONVERTER(TYPE_ADDR, AddrVal*, AsAddrVal) CONST_CONVERTER(TYPE_TABLE, TableVal*, AsTableVal) CONST_CONVERTER(TYPE_RECORD, RecordVal*, AsRecordVal) CONST_CONVERTER(TYPE_LIST, ListVal*, AsListVal) @@ -500,13 +509,9 @@ protected: #define NUM_PORT_SPACES 4 #define PORT_SPACE_MASK 0x30000 -#define TCP_PORT_MASK 0x10000 -#define UDP_PORT_MASK 0x20000 -#define ICMP_PORT_MASK 0x30000 - -typedef enum { - TRANSPORT_UNKNOWN, TRANSPORT_TCP, TRANSPORT_UDP, TRANSPORT_ICMP, -} TransportProto; +#define TCP_PORT_MASK 0x10000 +#define UDP_PORT_MASK 0x20000 +#define ICMP_PORT_MASK 0x30000 class PortVal : public Val { public: @@ -553,8 +558,9 @@ public: Val* SizeVal() const; // Constructor for address already in network order. - AddrVal(uint32 addr); - AddrVal(const uint32* addr); + AddrVal(uint32 addr); // IPv4. + AddrVal(const uint32 addr[4]); // IPv6. + AddrVal(const IPAddr& addr); unsigned int MemoryAllocation() const; @@ -564,9 +570,6 @@ protected: AddrVal(TypeTag t) : Val(t) { } AddrVal(BroType* t) : Val(t) { } - void Init(uint32 addr); - void Init(const uint32* addr); - DECLARE_SERIAL(AddrVal); }; @@ -574,30 +577,25 @@ class SubNetVal : public Val { public: SubNetVal(const char* text); SubNetVal(const char* text, int width); - SubNetVal(uint32 addr, int width); // for address already massaged - SubNetVal(const uint32* addr, int width); // ditto + SubNetVal(uint32 addr, int width); // IPv4. + SubNetVal(const uint32 addr[4], int width); // IPv6. + SubNetVal(const IPAddr& addr, int width); + ~SubNetVal(); Val* SizeVal() const; - int Width() const { return val.subnet_val.width; } - addr_type Mask() const; // returns host byte order + const IPAddr& Prefix() const; + int Width() const; + IPAddr Mask() const; - bool Contains(const uint32 addr) const; - bool Contains(const uint32* addr) const; + bool Contains(const IPAddr& addr) const; - unsigned int MemoryAllocation() const - { - return Val::MemoryAllocation() + padded_sizeof(*this) - padded_sizeof(Val); - } + unsigned int MemoryAllocation() const; protected: friend class Val; SubNetVal() {} - void Init(const char* text, int width); - void Init(uint32 addr, int width); - void Init(const uint32 *addr, int width); - void ValDescribe(ODesc* d) const; DECLARE_SERIAL(SubNetVal); diff --git a/src/X509.cc b/src/X509.cc deleted file mode 100644 index 55b6b78f04..0000000000 --- a/src/X509.cc +++ /dev/null @@ -1,263 +0,0 @@ -#include - -#include "X509.h" -#include "config.h" - -// ### NOTE: while d2i_X509 does not take a const u_char** pointer, -// here we assume d2i_X509 does not write to , so it is safe to -// convert data to a non-const pointer. Could some X509 guru verify -// this? - -X509* d2i_X509_(X509** px, const u_char** in, int len) - { -#ifdef OPENSSL_D2I_X509_USES_CONST_CHAR - return d2i_X509(px, in, len); -#else - return d2i_X509(px, (u_char**)in, len); -#endif - } - -X509_STORE* X509_Cert::ctx = 0; -X509_LOOKUP* X509_Cert::lookup = 0; -X509_STORE_CTX X509_Cert::csc; -bool X509_Cert::bInited = false; - -// TODO: Check if Key < 768 Bits => Weakness! -// FIXME: Merge verify and verifyChain. - -void X509_Cert::sslCertificateEvent(Contents_SSL* e, X509* pCert) - { - EventHandlerPtr event = ssl_certificate; - if ( ! event ) - return; - - char tmp[256]; - RecordVal* pX509Cert = new RecordVal(x509_type); - - X509_NAME_oneline(X509_get_issuer_name(pCert), tmp, sizeof tmp); - pX509Cert->Assign(0, new StringVal(tmp)); - X509_NAME_oneline(X509_get_subject_name(pCert), tmp, sizeof tmp); - pX509Cert->Assign(1, new StringVal(tmp)); - pX509Cert->Assign(2, new AddrVal(e->Conn()->OrigAddr())); - - val_list* vl = new val_list; - vl->append(e->BuildConnVal()); - vl->append(pX509Cert); - vl->append(new Val(e->IsOrig(), TYPE_BOOL)); - - e->Conn()->ConnectionEvent(event, e, vl); - } - -void X509_Cert::sslCertificateError(Contents_SSL* e, int error_numbe) - { - Val* err_str = new StringVal(X509_verify_cert_error_string(csc.error)); - val_list* vl = new val_list; - - vl->append(e->BuildConnVal()); - vl->append(new Val(csc.error, TYPE_INT)); - vl->append(err_str); - - e->Conn()->ConnectionEvent(ssl_X509_error, e, vl); - } - -int X509_Cert::init() - { -#if 0 - OpenSSL_add_all_algorithms(); -#endif - - ctx = X509_STORE_new(); - int flag = 0; - int ret = 0; - - if ( x509_trusted_cert_path && - x509_trusted_cert_path->AsString()->Len() > 0 ) - { // add the path(s) for the local CA's certificates - const BroString* pString = x509_trusted_cert_path->AsString(); - - lookup = X509_STORE_add_lookup(ctx, X509_LOOKUP_hash_dir()); - if ( ! lookup ) - { - reporter->Error("X509_Cert::init(): initing lookup failed\n"); - flag = 1; - } - - int i = X509_LOOKUP_add_dir(lookup, - (const char*) pString->Bytes(), - X509_FILETYPE_PEM); - if ( ! i ) - { - reporter->Error("X509_Cert::init(): error adding lookup directory\n"); - ret = 0; - } - } - else - { - printf("X509: Using the default trusted cert path.\n"); - X509_STORE_set_default_paths(ctx); - } - - // Add crl functionality - will only add if defined and - // X509_STORE_add_lookup was successful. - if ( ! flag && x509_crl_file && x509_crl_file->AsString()->Len() > 0 ) - { - const BroString* rString = x509_crl_file->AsString(); - - if ( X509_load_crl_file(lookup, (const char*) rString->Bytes(), - X509_FILETYPE_PEM) != 1 ) - { - reporter->Error("X509_Cert::init(): error reading CRL file\n"); - ret = 1; - } - -#if 0 - // Note, openssl version must be > 0.9.7(a). - X509_STORE_set_flags(ctx, - X509_V_FLAG_CRL_CHECK | X509_V_FLAG_CRL_CHECK_ALL); -#endif - } - - bInited = true; - return ret; - } - -int X509_Cert::verify(Contents_SSL* e, const u_char* data, uint32 len) - { - if ( ! bInited ) - init(); - - X509* pCert = d2i_X509_(NULL, &data, len); - if ( ! pCert ) - { - // 5 = X509_V_ERR_UNABLE_TO_DECODE_ISSUER_PUBLIC_KEY - sslCertificateError(e, 5); - return -1; - } - - sslCertificateEvent(e, pCert); - - X509_STORE_CTX_init(&csc, ctx, pCert, 0); - X509_STORE_CTX_set_time(&csc, 0, (time_t) network_time); - int i = X509_verify_cert(&csc); - X509_STORE_CTX_cleanup(&csc); - int ret = 0; - - int ext = X509_get_ext_count(pCert); - - if ( ext > 0 ) - { - TableVal* x509ex = new TableVal(x509_extension); - val_list* vl = new val_list; - char buf[256]; - - for ( int k = 0; k < ext; ++k ) - { - X509_EXTENSION* ex = X509_get_ext(pCert, k); - ASN1_OBJECT* obj = X509_EXTENSION_get_object(ex); - i2t_ASN1_OBJECT(buf, sizeof(buf), obj); - - Val* index = new Val(k+1, TYPE_COUNT); - Val* value = new StringVal(strlen(buf), buf); - x509ex->Assign(index, value); - Unref(index); - // later we can do critical extensions like: - // X509_EXTENSION_get_critical(ex); - } - - vl->append(e->BuildConnVal()); - vl->append(x509ex); - e->Conn()->ConnectionEvent(process_X509_extensions, e, vl); - } - - if ( ! i ) - { - sslCertificateError(e, csc.error); - ret = csc.error; - } - else - ret = 0; - - delete pCert; - return ret; - } - -int X509_Cert::verifyChain(Contents_SSL* e, const u_char* data, uint32 len) - { - if ( ! bInited ) - init(); - - // Gets an ssl3x cert chain (could be one single cert, too, - // but in chain format). - - // Init the stack. - STACK_OF(X509)* untrustedCerts = sk_X509_new_null(); - if ( ! untrustedCerts ) - { - // Internal error allocating stack of untrusted certs. - // 11 = X509_V_ERR_OUT_OF_MEM - sslCertificateError(e, 11); - return -1; - } - - // NOT AGAIN!!! - // Extract certificates and put them into an OpenSSL Stack. - uint tempLength = 0; - int certCount = 0; - X509* pCert = 0; // base cert, this one is to be verified - - while ( tempLength < len ) - { - ++certCount; - uint32 certLength = - uint32((data[tempLength + 0] << 16) | - data[tempLength + 1] << 8) | - data[tempLength + 2]; - - // Points to current cert. - const u_char* pCurrentCert = &data[tempLength+3]; - - X509* pTemp = d2i_X509_(0, &pCurrentCert, certLength); - if ( ! pTemp ) - { // error is somewhat of a misnomer - // 5 = X509_V_ERR_UNABLE_TO_DECODE_ISSUER_PUBLIC_KEY - sslCertificateError(e, 5); - //FIXME: free ptrs - return -1; - } - - if ( certCount == 1 ) - // The first certificate goes directly into the ctx. - pCert = pTemp; - else - // The remaining certificates (if any) are put into - // the list of untrusted certificates - sk_X509_push(untrustedCerts, pTemp); - - tempLength += certLength + 3; - } - - sslCertificateEvent(e, pCert); - - X509_STORE_CTX_init(&csc, ctx, pCert, untrustedCerts); - X509_STORE_CTX_set_time(&csc, 0, (time_t) network_time); - int i = X509_verify_cert(&csc); - X509_STORE_CTX_cleanup(&csc); - //X509_STORE_CTX_free(&csc); - int ret = 0; - - if ( ! i ) - { - sslCertificateError(e, csc.error); - ret = csc.error; - } - else - ret = 0; - - delete pCert; - // Free the stack, incuding. contents. - - // FIXME: could this break Bro's memory tracking? - sk_X509_pop_free(untrustedCerts, X509_free); - - return ret; - } diff --git a/src/bif_type.def b/src/bif_type.def index 4e206ceea2..5d9963ffc9 100644 --- a/src/bif_type.def +++ b/src/bif_type.def @@ -1,6 +1,6 @@ // DEFINE_BIF_TYPE(id, bif_type, bro_type, c_type, accessor, constructor) -DEFINE_BIF_TYPE(TYPE_ADDR, "addr", "addr", "addr_type", "%s->AsAddr()", "new AddrVal(%s)") +DEFINE_BIF_TYPE(TYPE_ADDR, "addr", "addr", "AddrVal*", "%s->AsAddrVal()", "%s") DEFINE_BIF_TYPE(TYPE_ANY, "any", "any", "Val*", "%s", "%s") DEFINE_BIF_TYPE(TYPE_BOOL, "bool", "bool", "int", "%s->AsBool()", "new Val(%s, TYPE_BOOL)") DEFINE_BIF_TYPE(TYPE_CONN_ID, "conn_id", "conn_id", "Val*", "%s", "%s") diff --git a/src/bittorrent-analyzer.pac b/src/bittorrent-analyzer.pac index ee7a70ea21..3bc6d90230 100644 --- a/src/bittorrent-analyzer.pac +++ b/src/bittorrent-analyzer.pac @@ -10,25 +10,25 @@ flow BitTorrent_Flow(is_orig: bool) { %member{ bool handshake_ok; - uint64 _next_message_offset; + //uint64 _next_message_offset; %} %init{ handshake_ok = false; - _next_message_offset = 0; + //_next_message_offset = 0; %} - function next_message_offset(): uint64 - %{ - return &_next_message_offset; - %} + #function next_message_offset(): uint64 + # %{ + # return &_next_message_offset; + # %} - function increment_next_message_offset(go: bool, len: uint32): bool - %{ - if ( go ) - _next_message_offset += len; - return true; - %} + #function increment_next_message_offset(go: bool, len: uint32): bool + # %{ + # if ( go ) + # _next_message_offset += len; + # return true; + # %} function is_handshake_delivered(): bool %{ diff --git a/src/bittorrent-protocol.pac b/src/bittorrent-protocol.pac index d3a147f157..76bbafbf20 100644 --- a/src/bittorrent-protocol.pac +++ b/src/bittorrent-protocol.pac @@ -22,8 +22,8 @@ type BitTorrent_Handshake = record { } &length = 68, &let { validate: bool = $context.flow.validate_handshake(pstrlen, pstr); - incoffsetffset: bool = - $context.flow.increment_next_message_offset(true, 68); + #incoffsetffset: bool = + # $context.flow.increment_next_message_offset(true, 68); deliver: bool = $context.flow.deliver_handshake(reserved, info_hash, peer_id); }; @@ -72,8 +72,8 @@ type BitTorrent_PieceHeader(len: uint32) = record { index: uint32; begin: uint32; } &let { - incoffset: bool = - $context.flow.increment_next_message_offset(true, len + 5); + #incoffset: bool = + # $context.flow.increment_next_message_offset(true, len + 5); }; type BitTorrent_Piece(len: uint32) = record { @@ -134,9 +134,9 @@ type BitTorrent_Message = record { default -> message_id: BitTorrent_MessageID(len.len); }; } &length = 4 + len.len, &let { - incoffset: bool = $context.flow.increment_next_message_offset( - len.len == 0 || message_id.id != TYPE_PIECE, - 4 + len.len); + #incoffset: bool = $context.flow.increment_next_message_offset( + # len.len == 0 || message_id.id != TYPE_PIECE, + # 4 + len.len); }; type BitTorrent_PDU = case $context.flow.is_handshake_delivered() of { diff --git a/src/bro.bif b/src/bro.bif index 27d6216f1a..f76704cfe6 100644 --- a/src/bro.bif +++ b/src/bro.bif @@ -14,6 +14,7 @@ #include #include "Reporter.h" +#include "IPAddr.h" using namespace std; @@ -178,35 +179,8 @@ static void do_fmt(const char*& fmt, Val* v, ODesc* d) // This makes only a very slight difference, so not // clear it would e worth the hassle. - addr_type u = v->AsAddr(); -#ifdef BROv6 - // We explicitly convert the address to host order - // in a copy, because if we just call ntohl() for - // our invocation on snprintf() below, on some systems - // it turns a 32-bit value (Linux), whereas on - // others it returns a long (FreeBSD); the latter - // gets us in trouble if we have longs > 32 bits, - // because then the format specifier needs to be %lx - // rather than %x ....... what a pain! - // - // Also note that we don't change u in-place because - // that would alter the byte order of the underlying - // value. (Speaking of which, I'm not clear on why - // we're allowed to assign a const addr_type to an - // addr_type above, both g++ allows it.) - uint32 host_order_u[4]; - host_order_u[0] = ntohl(u[0]); - host_order_u[1] = ntohl(u[1]); - host_order_u[2] = ntohl(u[2]); - host_order_u[3] = ntohl(u[3]); - - snprintf(out_buf, sizeof(out_buf), "%08x%08x%08x%08x", - host_order_u[0], host_order_u[1], - host_order_u[2], host_order_u[3]); -#else - u = ntohl(u); - snprintf(out_buf, sizeof(out_buf), "%08x", u); -#endif + snprintf(out_buf, sizeof(out_buf), "%s", + v->AsAddr().AsHexString().c_str()); } else if ( ! check_fmt_type(t, ok_d_fmt) ) @@ -1959,44 +1933,41 @@ function do_profiling%(%) : any ## Returns: True if *ip* belongs to a local interface. function is_local_interface%(ip: addr%) : bool %{ - static uint32* addrs; - static int len = -1; + if ( ip->AsAddr().IsLoopback() ) + return new Val(1, TYPE_BOOL); - if ( len < 0 ) + list addrs; + + char host[MAXHOSTNAMELEN]; + + strcpy(host, "localhost"); + gethostname(host, MAXHOSTNAMELEN); + host[MAXHOSTNAMELEN-1] = '\0'; + + struct hostent* ent = gethostbyname2(host, AF_INET); + + if ( ent ) { - char host[MAXHOSTNAMELEN]; - - strcpy(host, "localhost"); - gethostname(host, MAXHOSTNAMELEN); - host[MAXHOSTNAMELEN-1] = '\0'; - - struct hostent* ent = gethostbyname(host); - - for ( len = 0; ent->h_addr_list[len]; ++len ) - ; - - addrs = new uint32[len + 1]; - for ( int i = 0; i < len; i++ ) - addrs[i] = *(uint32*) ent->h_addr_list[i]; - - addrs[len++] = 0x0100007f; // 127.0.0.1 + for ( unsigned int len = 0; ent->h_addr_list[len]; ++len ) + addrs.push_back(IPAddr(IPv4, (uint32*)ent->h_addr_list[len], + IPAddr::Network)); } -#ifdef BROv6 - if ( ! is_v4_addr(ip) ) + ent = gethostbyname2(host, AF_INET6); + + if ( ent ) { - builtin_error("is_local_interface() only supports IPv4 addresses"); - return new Val(0, TYPE_BOOL); + for ( unsigned int len = 0; ent->h_addr_list[len]; ++len ) + addrs.push_back(IPAddr(IPv6, (uint32*)ent->h_addr_list[len], + IPAddr::Network)); } - uint32 ip4 = to_v4_addr(ip); -#else - uint32 ip4 = ip; -#endif - - for ( int i = 0; i < len; i++ ) - if ( addrs[i] == ip4 ) + list::const_iterator it; + for ( it = addrs.begin(); it != addrs.end(); ++it ) + { + if ( *it == ip->AsAddr() ) return new Val(1, TYPE_BOOL); + } return new Val(0, TYPE_BOOL); %} @@ -2046,19 +2017,125 @@ function gethostname%(%) : string return new StringVal(buffer); %} +## Returns whether an address is IPv4 or not. +## +## a: the address to check. +## +## Returns: true if *a* is an IPv4 address, else false. +function is_v4_addr%(a: addr%): bool + %{ + if ( a->AsAddr().GetFamily() == IPv4 ) + return new Val(1, TYPE_BOOL); + else + return new Val(0, TYPE_BOOL); + %} + +## Returns whether an address is IPv6 or not. +## +## a: the address to check. +## +## Returns: true if *a* is an IPv6 address, else false. +function is_v6_addr%(a: addr%): bool + %{ + if ( a->AsAddr().GetFamily() == IPv6 ) + return new Val(1, TYPE_BOOL); + else + return new Val(0, TYPE_BOOL); + %} + # =========================================================================== # # Conversion # # =========================================================================== +## Converts the *data* field of :bro:type:`ip6_routing` records that have +## *rtype* of 0 into a vector of addresses. +## +## s: The *data* field of an :bro:type:`ip6_routing` record that has +## an *rtype* of 0. +## +## Returns: The vector of addresses contained in the routing header data. +function routing0_data_to_addrs%(s: string%): addr_vec + %{ + VectorVal* rval = new VectorVal(new VectorType(base_type(TYPE_ADDR))); + + int len = s->Len(); + const u_char* bytes = s->Bytes(); + bytes += 4; // go past 32-bit reserved field + len -= 4; + + if ( ( len % 16 ) != 0 ) + reporter->Warning("Bad ip6_routing data length: %d", s->Len()); + + while ( len > 0 ) + { + IPAddr a(IPv6, (const uint32*) bytes, IPAddr::Network); + rval->Assign(rval->Size(), new AddrVal(a), 0); + bytes += 16; + len -= 16; + } + + return rval; + %} + +## Converts a :bro:type:`addr` to a :bro:type:`index_vec`. +## +## a: The address to convert into a vector of counts. +## +## Returns: A vector containing the host-order address representation, +## four elements in size for IPv6 addresses, or one element for IPv4. +## +## .. bro:see:: counts_to_addr +function addr_to_counts%(a: addr%): index_vec + %{ + VectorVal* rval = new VectorVal(new VectorType(base_type(TYPE_COUNT))); + const uint32* bytes; + int len = a->AsAddr().GetBytes(&bytes); + + for ( int i = 0; i < len; ++i ) + rval->Assign(i, new Val(ntohl(bytes[i]), TYPE_COUNT), 0); + + return rval; + %} + +## Converts a :bro:type:`index_vec` to a :bro:type:`addr`. +## +## v: The vector containing host-order IP address representation, +## one element for IPv4 addresses, four elements for IPv6 addresses. +## +## Returns: An IP address. +## +## .. bro:see:: addr_to_counts +function counts_to_addr%(v: index_vec%): addr + %{ + if ( v->AsVector()->size() == 1 ) + { + return new AddrVal(htonl((*v->AsVector())[0]->AsCount())); + } + else if ( v->AsVector()->size() == 4 ) + { + uint32 bytes[4]; + for ( int i = 0; i < 4; ++i ) + bytes[i] = htonl((*v->AsVector())[i]->AsCount()); + return new AddrVal(bytes); + } + else + { + builtin_error("invalid vector size", @ARG@[0]); + uint32 bytes[4]; + memset(bytes, 0, sizeof(bytes)); + return new AddrVal(bytes); + } + %} + ## Converts a :bro:type:`string` to a :bro:type:`int`. ## ## str: The :bro:type:`string` to convert. ## ## Returns: The :bro:type:`string` *str* as :bro:type:`int`. ## -## .. bro:see:: to_addr to_port +## .. bro:see:: to_addr to_port to_subnet function to_int%(str: string%): int %{ const char* s = str->CheckString(); @@ -2115,7 +2192,7 @@ function double_to_count%(d: double%): count ## Returns: The :bro:type:`string` *str* as unsigned integer or if in invalid ## format. ## -## .. bro:see:: to_addr to_int to_port +## .. bro:see:: to_addr to_int to_port to_subnet function to_count%(str: string%): count %{ const char* s = str->CheckString(); @@ -2180,29 +2257,6 @@ function double_to_interval%(d: double%): interval return new Val(d, TYPE_INTERVAL); %} -## Converts a :bro:type:`addr` to a :bro:type:`count`. -## -## a: The :bro:type:`addr` to convert. -## -## Returns: The :bro:type:`addr` *a* as :bro:type:`count`. -## -## .. bro:see:: addr_to_ptr_name -function addr_to_count%(a: addr%): count - %{ -#ifdef BROv6 - if ( ! is_v4_addr(a) ) - { - builtin_error("conversion of non-IPv4 address to count", @ARG@[0]); - return new Val(0, TYPE_COUNT); - } - - uint32 addr = to_v4_addr(a); -#else - uint32 addr = a; -#endif - return new Val(ntohl(addr), TYPE_COUNT); - %} - ## Converts a :bro:type:`port` to a :bro:type:`count`. ## ## p: The :bro:type:`port` to convert. @@ -2233,9 +2287,11 @@ function count_to_port%(num: count, proto: transport_proto%): port ## ## ip: The :bro:type:`string` to convert. ## -## Returns: The :bro:type:`string` *ip* as :bro:type:`addr`. +## Returns: The :bro:type:`string` *ip* as :bro:type:`addr` or the unspecified +## address ``::`` if the input string does not parse correctly. ## ## .. bro:see:: to_count to_int to_port count_to_v4_addr raw_bytes_to_v4_addr +## to_subnet function to_addr%(ip: string%): addr %{ char* s = ip->AsString()->Render(); @@ -2244,13 +2300,30 @@ function to_addr%(ip: string%): addr return ret; %} +## Converts a :bro:type:`string` to a :bro:type:`subnet`. +## +## sn: The subnet to convert. +## +## Returns: The *sn* string as a :bro:type:`subnet` or the unspecified subnet +## ``::/0`` if the input string does not parse correctly. +## +## .. bro:see:: to_count to_int to_port count_to_v4_addr raw_bytes_to_v4_addr +## to_addr +function to_subnet%(sn: string%): subnet + %{ + char* s = sn->AsString()->Render(); + Val* ret = new SubNetVal(s); + delete [] s; + return ret; + %} + ## Converts a :bro:type:`count` to an :bro:type:`addr`. ## ## ip: The :bro:type:`count` to convert. ## ## Returns: The :bro:type:`count` *ip* as :bro:type:`addr`. ## -## .. bro:see:: raw_bytes_to_v4_addr to_addr +## .. bro:see:: raw_bytes_to_v4_addr to_addr to_subnet function count_to_v4_addr%(ip: count%): addr %{ if ( ip > 4294967295LU ) @@ -2270,7 +2343,7 @@ function count_to_v4_addr%(ip: count%): addr ## ## Returns: The byte :bro:type:`string` *ip* as :bro:type:`addr`. ## -## .. bro:see:: raw_bytes_to_v4_addr to_addr +## .. bro:see:: raw_bytes_to_v4_addr to_addr to_subnet function raw_bytes_to_v4_addr%(b: string%): addr %{ uint32 a = 0; @@ -2293,7 +2366,7 @@ function raw_bytes_to_v4_addr%(b: string%): addr ## ## Returns: A :bro:type:`port` converted from *s*. ## -## .. bro:see:: to_addr to_count to_int +## .. bro:see:: to_addr to_count to_int to_subnet function to_port%(s: string%): port %{ int port = 0; @@ -2327,20 +2400,58 @@ function to_port%(s: string%): port ## .. bro:see:: addr_to_ptr_name parse_dotted_addr function ptr_name_to_addr%(s: string%): addr %{ - int a[4]; - uint32 addr; - - if ( sscanf(s->CheckString(), - "%d.%d.%d.%d.in-addr.arpa", - a, a+1, a+2, a+3) != 4 ) + if ( s->Len() != 72 ) { - builtin_error("bad PTR name", @ARG@[0]); - addr = 0; + int a[4]; + uint32 addr; + char ss[13]; // this will contain "in-addr.arpa" + + if ( sscanf(s->CheckString(), + "%d.%d.%d.%d.%12s", + a, a+1, a+2, a+3, ss) != 5 + || strcmp(ss, "in-addr.arpa") != 0 ) + { + builtin_error("bad PTR name", @ARG@[0]); + addr = 0; + } + else + addr = (a[3] << 24) | (a[2] << 16) | (a[1] << 8) | a[0]; + + return new AddrVal(htonl(addr)); } else - addr = (a[3] << 24) | (a[2] << 16) | (a[1] << 8) | a[0]; + { + uint32 addr6[4]; + uint32 b[32]; + char ss[9]; // this will contain "ip6.arpa" + if ( sscanf(s->CheckString(), + "%1x.%1x.%1x.%1x.%1x.%1x.%1x.%1x." + "%1x.%1x.%1x.%1x.%1x.%1x.%1x.%1x." + "%1x.%1x.%1x.%1x.%1x.%1x.%1x.%1x." + "%1x.%1x.%1x.%1x.%1x.%1x.%1x.%1x.%8s", + b+31, b+30, b+29, b+28, b+27, b+26, b+25, b+24, + b+23, b+22, b+21, b+20, b+19, b+18, b+17, b+16, + b+15, b+14, b+13, b+12, b+11, b+10, b+9, b+8, + b+7, b+6, b+5, b+4, b+3, b+2, b+1, b, ss) != 33 + || strcmp(ss, "ip6.arpa") != 0 ) + { + builtin_error("bad PTR name", @ARG@[0]); + memset(addr6, 0, sizeof addr6); + } + else + { + for ( unsigned int i = 0; i < 4; ++i ) + { + uint32 a = 0; + for ( unsigned int j = 1; j <= 8; ++j ) + a |= b[8*i+j-1] << (32-j*4); - return new AddrVal(htonl(addr)); + addr6[i] = htonl(a); + } + } + + return new AddrVal(addr6); + } %} ## Converts an IP address to a reverse pointer name. For example, @@ -2350,34 +2461,10 @@ function ptr_name_to_addr%(s: string%): addr ## ## Returns: The reverse pointer representation of *a*. ## -## .. bro:see:: addr_to_count ptr_name_to_addr parse_dotted_addr +## .. bro:see:: ptr_name_to_addr parse_dotted_addr function addr_to_ptr_name%(a: addr%): string %{ - // ## Question: - // uint32 addr = ntohl((*args)[0]->InternalUnsigned()); - uint32 addr; -#ifdef BROv6 - if ( is_v4_addr(a) ) - addr = a[3]; - else - { - builtin_error("conversion of non-IPv4 address to net", @ARG@[0]); - addr = 0; - } -#else - addr = a; -#endif - - addr = ntohl(addr); - uint32 a3 = (addr >> 24) & 0xff; - uint32 a2 = (addr >> 16) & 0xff; - uint32 a1 = (addr >> 8) & 0xff; - uint32 a0 = addr & 0xff; - - char buf[256]; - sprintf(buf, "%u.%u.%u.%u.in-addr.arpa", a0, a1, a2, a3); - - return new StringVal(buf); + return new StringVal(a->AsAddr().PtrName().c_str()); %} # Transforms n0.n1.n2.n3 -> addr. @@ -2389,10 +2476,11 @@ function addr_to_ptr_name%(a: addr%): string ## ## Returns: The IP address as type :bro:type:`addr`. ## -## .. bro:see:: addr_to_ptr_name parse_dotted_addr addr_to_count +## .. bro:see:: addr_to_ptr_name parse_dotted_addr function parse_dotted_addr%(s: string%): addr %{ - return new AddrVal(dotted_to_addr(s->CheckString())); + IPAddr a(s->CheckString()); + return new AddrVal(a); %} %%{ @@ -2444,7 +2532,7 @@ static Val* parse_eftp(const char* line) RecordVal* r = new RecordVal(ftp_port); int net_proto = 0; // currently not used - uint32 addr = 0; + IPAddr addr; // unspecified IPv6 address (all 128 bits zero) int port = 0; int good = 0; @@ -2454,29 +2542,51 @@ static Val* parse_eftp(const char* line) ++line; char delimiter = *line; - good = 1; char* next_delim; - ++line; // cut off delimiter - net_proto = strtol(line, &next_delim, 10); // currently ignored - if ( *next_delim != delimiter ) - good = 0; - - line = next_delim + 1; - if ( *line != delimiter ) // default of 0 is ok + if ( *line ) { - addr = dotted_to_addr(line); - if ( addr == 0 ) + good = 1; + ++line; // skip delimiter + + net_proto = strtol(line, &next_delim, 10); + if ( *next_delim != delimiter ) good = 0; + + line = next_delim; + if ( *line ) + ++line; + + if ( *line && *line != delimiter ) + { + const char* nptr = strchr(line, delimiter); + if ( nptr == NULL ) + { + nptr = line + strlen(line); + good = 0; + } + + string s(line, nptr-line); // extract IP address + IPAddr tmp(s); + // on error, "tmp" will have all 128 bits zero + if ( tmp == addr ) + good = 0; + + addr = tmp; + } + + line = strchr(line, delimiter); + + if ( line != NULL ) + { + ++line; // now the port + port = strtol(line, &next_delim, 10); + if ( *next_delim != delimiter ) + good = 0; + } + } - // FIXME: check for garbage between IP and delimiter. - line = strchr(line, delimiter); - - ++line; // now the port - port = strtol(line, &next_delim, 10); - if ( *next_delim != delimiter ) - good = 0; } r->Assign(0, new AddrVal(addr)); @@ -2504,7 +2614,7 @@ function parse_ftp_port%(s: string%): ftp_port ## The format is ``EPRT``, ## where ```` is a delimiter in the ASCII range 33-126 (usually ``|``). ## -## s: The string of the FTP PORT command, e.g., ``"10,0,0,1,4,31"``. +## s: The string of the FTP EPRT command, e.g., ``"|1|10.0.0.1|1055|"``. ## ## Returns: The FTP PORT, e.g., ``[h=10.0.0.1, p=1055/tcp, valid=T]`` ## @@ -2544,7 +2654,7 @@ function parse_ftp_pasv%(str: string%): ftp_port ## The format is `` ()``, where ```` is a ## delimiter in the ASCII range 33-126 (usually ``|``). ## -## str: The string containing the result of the FTP PASV command. +## str: The string containing the result of the FTP EPSV command. ## ## Returns: The FTP PORT, e.g., ``[h=10.0.0.1, p=1055/tcp, valid=T]`` ## @@ -2570,20 +2680,23 @@ function parse_ftp_epsv%(str: string%): ftp_port ## .. bro:see:: parse_ftp_port parse_eftp_port parse_ftp_pasv parse_ftp_epsv function fmt_ftp_port%(a: addr, p: port%): string %{ -#ifdef BROv6 - if ( ! is_v4_addr(a) ) - builtin_error("conversion of non-IPv4 address to net", @ARG@[0]); - - uint32 addr = to_v4_addr(a); -#else - uint32 addr = a; -#endif - addr = ntohl(addr); - uint32 pn = p->Port(); - return new StringVal(fmt("%d,%d,%d,%d,%d,%d", - addr >> 24, (addr >> 16) & 0xff, - (addr >> 8) & 0xff, addr & 0xff, - pn >> 8, pn & 0xff)); + const uint32* addr; + int len = a->AsAddr().GetBytes(&addr); + if ( len == 1 ) + { + uint32 a = ntohl(addr[0]); + uint32 pn = p->Port(); + return new StringVal(fmt("%d,%d,%d,%d,%d,%d", + a >> 24, (a >> 16) & 0xff, + (a >> 8) & 0xff, a & 0xff, + pn >> 8, pn & 0xff)); + } + else + { + builtin_error("conversion of non-IPv4 address in fmt_ftp_port", + @ARG@[0]); + return new StringVal(""); + } %} ## Decode a NetBIOS name. See http://support.microsoft.com/kb/194203. @@ -2896,7 +3009,7 @@ function strftime%(fmt: string, d: time%) : string ## .. bro:see:: remask_addr function mask_addr%(a: addr, top_bits_to_keep: count%): subnet %{ - return new SubNetVal(mask_addr(a, top_bits_to_keep), top_bits_to_keep); + return new SubNetVal(a->AsAddr(), top_bits_to_keep); %} ## Takes some top bits (e.g., subnet address) from one address and the other @@ -2909,29 +3022,20 @@ function mask_addr%(a: addr, top_bits_to_keep: count%): subnet ## a2: The address to take the remaining bits from. ## ## top_bits_from_a1: The number of top bits to keep in *a1*; must be greater -## than 0 and less than 33. +## than 0 and less than 129. This value is always interpreted +## relative to the IPv6 bit width (v4-mapped addresses start +## at bit number 96). ## ## Returns: The address *a* masked down to *top_bits_to_keep* bits. ## ## .. bro:see:: mask_addr function remask_addr%(a1: addr, a2: addr, top_bits_from_a1: count%): addr %{ -#ifdef BROv6 - if ( ! is_v4_addr(a1) || ! is_v4_addr(a2) ) - { - builtin_error("cannot use remask_addr on IPv6 addresses"); - return new AddrVal(a1); - } - - uint32 x1 = to_v4_addr(a1); - uint32 x2 = to_v4_addr(a2); -#else - uint32 x1 = a1; - uint32 x2 = a2; -#endif - return new AddrVal( - mask_addr(x1, top_bits_from_a1) | - (x2 ^ mask_addr(x2, top_bits_from_a1)) ); + IPAddr addr1(a1->AsAddr()); + addr1.Mask(top_bits_from_a1); + IPAddr addr2(a2->AsAddr()); + addr2.ReverseMask(top_bits_from_a1); + return new AddrVal(addr1|addr2); %} ## Checks whether a given :bro:type:`port` has TCP as transport protocol. @@ -3104,12 +3208,13 @@ const char* conn_id_string(Val* c) Val* id = (*(c->AsRecord()))[0]; const val_list* vl = id->AsRecord(); - addr_type orig_h = (*vl)[0]->AsAddr(); + const IPAddr& orig_h = (*vl)[0]->AsAddr(); uint32 orig_p = (*vl)[1]->AsPortVal()->Port(); - addr_type resp_h = (*vl)[2]->AsAddr(); + const IPAddr& resp_h = (*vl)[2]->AsAddr(); uint32 resp_p = (*vl)[3]->AsPortVal()->Port(); - return fmt("%s/%u -> %s/%u\n", dotted_addr(orig_h), orig_p, dotted_addr(resp_h), resp_p); + return fmt("%s/%u -> %s/%u\n", orig_h.AsString().c_str(), orig_p, + resp_h.AsString().c_str(), resp_p); } %%} @@ -3339,29 +3444,8 @@ function lookup_addr%(host: addr%) : string frame->SetDelayed(); trigger->Hold(); -#ifdef BROv6 - if ( ! is_v4_addr(host) ) - { - // FIXME: This is a temporary work-around until we get this - // fixed. We warn the user once, and always trigger a timeout. - // Ticket #355 records the problem. - static bool warned = false; - if ( ! warned ) - { - reporter->Warning("lookup_addr() only supports IPv4 addresses currently"); - warned = true; - } - - trigger->Timeout(); - return 0; - } - - dns_mgr->AsyncLookupAddr(to_v4_addr(host), + dns_mgr->AsyncLookupAddr(host->AsAddr(), new LookupHostCallback(trigger, frame->GetCall(), true)); -#else - dns_mgr->AsyncLookupAddr(host, - new LookupHostCallback(trigger, frame->GetCall(), true)); -#endif return 0; %} @@ -3453,8 +3537,6 @@ function lookup_location%(a: addr%) : geo_location else have_city_db = true; -#ifdef BROv6 - #ifdef HAVE_GEOIP_CITY_EDITION_REV0_V6 geoip_v6 = open_geoip_db(GEOIP_CITY_EDITION_REV0_V6); if ( geoip_v6 ) @@ -3467,16 +3549,13 @@ function lookup_location%(a: addr%) : geo_location #endif if ( ! geoip_v6 ) builtin_error("Can't initialize GeoIPv6 City/Country database"); -#endif } -#ifdef BROv6 - #ifdef HAVE_GEOIP_COUNTRY_EDITION_V6 - if ( geoip_v6 && ! is_v4_addr(a) ) + if ( geoip_v6 && a->AsAddr().GetFamily() == IPv6 ) { geoipv6_t ga; - memcpy(&ga, a, 16); + a->AsAddr().CopyIPv6(&ga); if ( have_cityv6_db ) gir = GeoIP_record_by_ipnum_v6(geoip_v6, ga); else @@ -3485,25 +3564,16 @@ function lookup_location%(a: addr%) : geo_location else #endif - if ( geoip && is_v4_addr(a) ) + if ( geoip && a->AsAddr().GetFamily() == IPv4 ) { - uint32 addr = to_v4_addr(a); + const uint32* bytes; + a->AsAddr().GetBytes(&bytes); if ( have_city_db ) - gir = GeoIP_record_by_ipnum(geoip, ntohl(addr)); + gir = GeoIP_record_by_ipnum(geoip, ntohl(*bytes)); else - cc = GeoIP_country_code_by_ipnum(geoip, ntohl(addr)); + cc = GeoIP_country_code_by_ipnum(geoip, ntohl(*bytes)); } -#else // not BROv6 - if ( geoip ) - { - if ( have_city_db ) - gir = GeoIP_record_by_ipnum(geoip, ntohl(a)); - else - cc = GeoIP_country_code_by_ipnum(geoip, ntohl(a)); - } -#endif - if ( gir ) { if ( gir->country_code ) @@ -3575,28 +3645,23 @@ function lookup_asn%(a: addr%) : count if ( geoip_asn ) { -#ifdef BROv6 - // IPv6 support showed up in 1.4.5. #ifdef HAVE_GEOIP_COUNTRY_EDITION_V6 - if ( ! is_v4_addr(a) ) + if ( a->AsAddr().GetFamily() == IPv6 ) { geoipv6_t ga; - memcpy(&ga, a, 16); + a->AsAddr().CopyIPv6(&ga); gir = GeoIP_name_by_ipnum_v6(geoip_asn, ga); } else #endif - if ( is_v4_addr(a) ) + if ( a->AsAddr().GetFamily() == IPv4 ) { - uint32 addr = to_v4_addr(a); - gir = GeoIP_name_by_ipnum(geoip_asn, ntohl(addr)); + const uint32* bytes; + a->AsAddr().GetBytes(&bytes); + gir = GeoIP_name_by_ipnum(geoip_asn, ntohl(*bytes)); } - -#else // not BROv6 - gir = GeoIP_name_by_ipnum(geoip_asn, ntohl(a)); -#endif } if ( gir ) @@ -3878,9 +3943,9 @@ function file_mode%(mode: count%): string function expect_connection%(orig: addr, resp: addr, resp_p: port, analyzer: count, tout: interval%) : any %{ - dpm->ExpectConnection(orig, resp, resp_p->Port(), resp_p->PortType(), - (AnalyzerTag::Tag) analyzer, tout, 0); - return 0; + dpm->ExpectConnection(orig->AsAddr(), resp->AsAddr(), resp_p->Port(), + resp_p->PortType(), (AnalyzerTag::Tag) analyzer, tout, 0); + return new Val(1, TYPE_BOOL); %} ## Disables the analyzer which raised the current event (if the analyzer @@ -4628,6 +4693,8 @@ function pcap_error%(%): string ## Installs a filter to drop packets from a given IP source address with ## a certain probability if none of a given set of TCP flags are set. +## Note that for IPv6 packets with a Destination options header that has +## the Home Address option, this filters out against that home address. ## ## ip: The IP address to drop. ## @@ -4652,7 +4719,7 @@ function pcap_error%(%): string ## .. todo:: The return value should be changed to any. function install_src_addr_filter%(ip: addr, tcp_flags: count, prob: double%) : bool %{ - sessions->GetPacketFilter()->AddSrc(ip, tcp_flags, prob); + sessions->GetPacketFilter()->AddSrc(ip->AsAddr(), tcp_flags, prob); return new Val(1, TYPE_BOOL); %} @@ -4704,7 +4771,7 @@ function install_src_net_filter%(snet: subnet, tcp_flags: count, prob: double%) ## pcap_error function uninstall_src_addr_filter%(ip: addr%) : bool %{ - return new Val(sessions->GetPacketFilter()->RemoveSrc(ip), TYPE_BOOL); + return new Val(sessions->GetPacketFilter()->RemoveSrc(ip->AsAddr()), TYPE_BOOL); %} ## Removes a source subnet filter. @@ -4730,6 +4797,9 @@ function uninstall_src_net_filter%(snet: subnet%) : bool ## Installs a filter to drop packets destined to a given IP address with ## a certain probability if none of a given set of TCP flags are set. +## Note that for IPv6 packets with a routing type header and non-zero +## segments left, this filters out against the final destination of the +## packet according to the routing extension header. ## ## ip: Drop packets to this IP address. ## @@ -4754,7 +4824,7 @@ function uninstall_src_net_filter%(snet: subnet%) : bool ## .. todo:: The return value should be changed to any. function install_dst_addr_filter%(ip: addr, tcp_flags: count, prob: double%) : bool %{ - sessions->GetPacketFilter()->AddDst(ip, tcp_flags, prob); + sessions->GetPacketFilter()->AddDst(ip->AsAddr(), tcp_flags, prob); return new Val(1, TYPE_BOOL); %} @@ -4806,7 +4876,7 @@ function install_dst_net_filter%(snet: subnet, tcp_flags: count, prob: double%) ## pcap_error function uninstall_dst_addr_filter%(ip: addr%) : bool %{ - return new Val(sessions->GetPacketFilter()->RemoveDst(ip), TYPE_BOOL); + return new Val(sessions->GetPacketFilter()->RemoveDst(ip->AsAddr()), TYPE_BOOL); %} ## Removes a destination subnet filter. @@ -4947,7 +5017,7 @@ function capture_state_updates%(filename: string%) : bool ## send_id function connect%(ip: addr, p: port, our_class: string, retry: interval, ssl: bool%) : count %{ - return new Val(uint32(remote_serializer->Connect(ip, p->Port(), + return new Val(uint32(remote_serializer->Connect(ip->AsAddr(), p->Port(), our_class->CheckString(), retry, ssl)), TYPE_COUNT); %} @@ -5062,7 +5132,7 @@ function set_compression_level%(p: event_peer, level: count%) : bool ## .. bro:see:: connect disconnect function listen%(ip: addr, p: port, ssl: bool %) : bool %{ - return new Val(remote_serializer->Listen(ip, p->Port(), ssl), TYPE_BOOL); + return new Val(remote_serializer->Listen(ip->AsAddr(), p->Port(), ssl), TYPE_BOOL); %} ## Checks whether the last raised event came from a remote peer. @@ -5318,14 +5388,14 @@ function preserve_prefix%(a: addr, width: count%): any AnonymizeIPAddr* ip_anon = ip_anonymizer[PREFIX_PRESERVING_A50]; if ( ip_anon ) { -#ifdef BROv6 - if ( ! is_v4_addr(a) ) + if ( a->AsAddr().GetFamily() == IPv6 ) builtin_error("preserve_prefix() not supported for IPv6 addresses"); else - ip_anon->PreservePrefix(a[3], width); -#else - ip_anon->PreservePrefix(a, width); -#endif + { + const uint32* bytes; + a->AsAddr().GetBytes(&bytes); + ip_anon->PreservePrefix(*bytes, width); + } } @@ -5343,18 +5413,18 @@ function preserve_prefix%(a: addr, width: count%): any ## .. todo:: Currently dysfunctional. function preserve_subnet%(a: subnet%): any %{ - DEBUG_MSG("%s/%d\n", dotted_addr(a->AsAddr()), a->Width()); + DEBUG_MSG("%s/%d\n", a->Prefix().AsString().c_str(), a->Width()); AnonymizeIPAddr* ip_anon = ip_anonymizer[PREFIX_PRESERVING_A50]; if ( ip_anon ) { -#ifdef BROv6 - if ( ! is_v4_addr(a->AsAddr()) ) + if ( a->AsSubNet().Prefix().GetFamily() == IPv6 ) builtin_error("preserve_subnet() not supported for IPv6 addresses"); else - ip_anon->PreservePrefix(a->AsAddr()[3], a->Width()); -#else - ip_anon->PreservePrefix(a->AsAddr(), a->Width()); -#endif + { + const uint32* bytes; + a->AsSubNet().Prefix().GetBytes(&bytes); + ip_anon->PreservePrefix(*bytes, a->AsSubNet().Length()); + } } return 0; @@ -5383,19 +5453,18 @@ function anonymize_addr%(a: addr, cl: IPAddrAnonymizationClass%): addr if ( anon_class < 0 || anon_class >= NUM_ADDR_ANONYMIZATION_CLASSES ) builtin_error("anonymize_addr(): invalid ip addr anonymization class"); -#ifdef BROv6 - if ( ! is_v4_addr(a) ) + if ( a->AsAddr().GetFamily() == IPv6 ) { builtin_error("anonymize_addr() not supported for IPv6 addresses"); return 0; } else - return new AddrVal(anonymize_ip(a[3], + { + const uint32* bytes; + a->AsAddr().GetBytes(&bytes); + return new AddrVal(anonymize_ip(*bytes, (enum ip_addr_anonymization_class_t) anon_class)); -#else - return new AddrVal(anonymize_ip(a, - (enum ip_addr_anonymization_class_t) anon_class)); -#endif + } %} ## Deprecated. Will be removed. @@ -5445,7 +5514,7 @@ function generate_idmef%(src_ip: addr, src_port: port, newNode(newAddress( newAttribute("category","ipv4-addr"), newSimpleElement("address", - copy_string(dotted_addr(src_ip))), + copy_string(src_ip->AsAddr().AsString().c_str())), NULL), NULL), newService( newSimpleElement("port", @@ -5455,7 +5524,7 @@ function generate_idmef%(src_ip: addr, src_port: port, newNode(newAddress( newAttribute("category","ipv4-addr"), newSimpleElement("address", - copy_string(dotted_addr(dst_ip))), + copy_string(dst_ip->AsAddr().AsString().c_str())), NULL), NULL), newService( newSimpleElement("port", @@ -5470,13 +5539,3 @@ function generate_idmef%(src_ip: addr, src_port: port, return new Val(0, TYPE_BOOL); #endif %} - -## Deprecated. Will be removed. -function bro_has_ipv6%(%) : bool - %{ -#ifdef BROv6 - return new Val(1, TYPE_BOOL); -#else - return new Val(0, TYPE_BOOL); -#endif - %} diff --git a/src/bro_inet_ntop.c b/src/bro_inet_ntop.c new file mode 100644 index 0000000000..c66c1daeda --- /dev/null +++ b/src/bro_inet_ntop.c @@ -0,0 +1,189 @@ +/* Taken/adapted from FreeBSD 9.0.0 inet_ntop.c (CVS revision 1.3.16.1.2.1) */ +/* + * Copyright (c) 2004 by Internet Systems Consortium, Inc. ("ISC") + * Copyright (c) 1996-1999 by Internet Software Consortium. + * + * Permission to use, copy, modify, and distribute this software for any + * purpose with or without fee is hereby granted, provided that the above + * copyright notice and this permission notice appear in all copies. + * + * THE SOFTWARE IS PROVIDED "AS IS" AND ISC DISCLAIMS ALL WARRANTIES + * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF + * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL ISC BE LIABLE FOR + * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES + * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN + * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT + * OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. + */ + +#include "bro_inet_ntop.h" + +#include +#include +#include + +#include +#include +#include + +#include +#include +#include + +/*% + * WARNING: Don't even consider trying to compile this on a system where + * sizeof(int) < 4. sizeof(int) > 4 is fine; all the world's not a VAX. + */ + +static const char *bro_inet_ntop4(const u_char *src, char *dst, socklen_t size); +static const char *bro_inet_ntop6(const u_char *src, char *dst, socklen_t size); + +/* char * + * bro_inet_ntop(af, src, dst, size) + * convert a network format address to presentation format. + * return: + * pointer to presentation format address (`dst'), or NULL (see errno). + * author: + * Paul Vixie, 1996. + */ +const char * +bro_inet_ntop(int af, const void * __restrict src, char * __restrict dst, + socklen_t size) +{ + switch (af) { + case AF_INET: + return (bro_inet_ntop4(src, dst, size)); + case AF_INET6: + return (bro_inet_ntop6(src, dst, size)); + default: + errno = EAFNOSUPPORT; + return (NULL); + } + /* NOTREACHED */ +} + +/* const char * + * bro_inet_ntop4(src, dst, size) + * format an IPv4 address + * return: + * `dst' (as a const) + * notes: + * (1) uses no statics + * (2) takes a u_char* not an in_addr as input + * author: + * Paul Vixie, 1996. Modified by Jon Siwek, 2012, to replace strlcpy + */ +static const char * +bro_inet_ntop4(const u_char *src, char *dst, socklen_t size) +{ + static const char fmt[] = "%u.%u.%u.%u"; + char tmp[sizeof "255.255.255.255"]; + int l; + + l = snprintf(tmp, sizeof(tmp), fmt, src[0], src[1], src[2], src[3]); + if (l <= 0 || (socklen_t) l >= size) { + errno = ENOSPC; + return (NULL); + } + strncpy(dst, tmp, size - 1); + dst[size - 1] = 0; + return (dst); +} + +/* const char * + * bro_inet_ntop6(src, dst, size) + * convert IPv6 binary address into presentation (printable) format + * author: + * Paul Vixie, 1996. Modified by Jon Siwek, 2012, for IPv4-translated format + */ +static const char * +bro_inet_ntop6(const u_char *src, char *dst, socklen_t size) +{ + /* + * Note that int32_t and int16_t need only be "at least" large enough + * to contain a value of the specified size. On some systems, like + * Crays, there is no such thing as an integer variable with 16 bits. + * Keep this in mind if you think this function should have been coded + * to use pointer overlays. All the world's not a VAX. + */ + char tmp[sizeof "ffff:ffff:ffff:ffff:ffff:ffff:255.255.255.255"], *tp; + struct { int base, len; } best, cur; + u_int words[NS_IN6ADDRSZ / NS_INT16SZ]; + int i; + + /* + * Preprocess: + * Copy the input (bytewise) array into a wordwise array. + * Find the longest run of 0x00's in src[] for :: shorthanding. + */ + memset(words, '\0', sizeof words); + for (i = 0; i < NS_IN6ADDRSZ; i++) + words[i / 2] |= (src[i] << ((1 - (i % 2)) << 3)); + best.base = -1; + best.len = 0; + cur.base = -1; + cur.len = 0; + for (i = 0; i < (NS_IN6ADDRSZ / NS_INT16SZ); i++) { + if (words[i] == 0) { + if (cur.base == -1) + cur.base = i, cur.len = 1; + else + cur.len++; + } else { + if (cur.base != -1) { + if (best.base == -1 || cur.len > best.len) + best = cur; + cur.base = -1; + } + } + } + if (cur.base != -1) { + if (best.base == -1 || cur.len > best.len) + best = cur; + } + if (best.base != -1 && best.len < 2) + best.base = -1; + + /* + * Format the result. + */ + tp = tmp; + for (i = 0; i < (NS_IN6ADDRSZ / NS_INT16SZ); i++) { + /* Are we inside the best run of 0x00's? */ + if (best.base != -1 && i >= best.base && + i < (best.base + best.len)) { + if (i == best.base) + *tp++ = ':'; + continue; + } + /* Are we following an initial run of 0x00s or any real hex? */ + if (i != 0) + *tp++ = ':'; + /* Is this address an encapsulated IPv4? */ + if (i == 6 && best.base == 0 && (best.len == 6 || + (best.len == 7 && words[7] != 0x0001) || + (best.len == 5 && words[5] == 0xffff) || + (best.len == 4 && words[4] == 0xffff && words[5] == 0))) { + if (!bro_inet_ntop4(src+12, tp, sizeof tmp - (tp - tmp))) + return (NULL); + tp += strlen(tp); + break; + } + tp += sprintf(tp, "%x", words[i]); + } + /* Was it a trailing run of 0x00's? */ + if (best.base != -1 && (best.base + best.len) == + (NS_IN6ADDRSZ / NS_INT16SZ)) + *tp++ = ':'; + *tp++ = '\0'; + + /* + * Check for overflow, copy, and we're done. + */ + if ((socklen_t)(tp - tmp) > size) { + errno = ENOSPC; + return (NULL); + } + strcpy(dst, tmp); + return (dst); +} diff --git a/src/bro_inet_ntop.h b/src/bro_inet_ntop.h new file mode 100644 index 0000000000..00326b092e --- /dev/null +++ b/src/bro_inet_ntop.h @@ -0,0 +1,18 @@ +#ifndef BRO_INET_NTOP_H +#define BRO_INET_NTOP_H + +#ifdef __cplusplus +extern "C" { +#endif + +#include + +const char * +bro_inet_ntop(int af, const void * __restrict src, char * __restrict dst, + socklen_t size); + +#ifdef __cplusplus +} +#endif + +#endif diff --git a/src/dhcp-analyzer.pac b/src/dhcp-analyzer.pac index a9f1c6bab0..5267075445 100644 --- a/src/dhcp-analyzer.pac +++ b/src/dhcp-analyzer.pac @@ -55,33 +55,18 @@ flow DHCP_Flow(is_orig: bool) { vector::const_iterator ptr; // Requested IP address to the server. -#ifdef BROv6 - ::uint32 req_addr[4], serv_addr[4]; - - req_addr[0] = req_addr[1] = req_addr[2] = req_addr[3] = 0; - serv_addr[0] = serv_addr[1] = serv_addr[2] = serv_addr[3] = 0; -#else - addr_type req_addr = 0, serv_addr = 0; -#endif + ::uint32 req_addr = 0, serv_addr = 0; for ( ptr = options->begin(); ptr != options->end() && ! (*ptr)->last(); ++ptr ) { switch ( (*ptr)->code() ) { case REQ_IP_OPTION: -#ifdef BROv6 - req_addr[3] = htonl((*ptr)->info()->req_addr()); -#else req_addr = htonl((*ptr)->info()->req_addr()); -#endif break; case SERV_ID_OPTION: -#ifdef BROv6 - serv_addr[3] = htonl((*ptr)->info()->serv_addr()); -#else serv_addr = htonl((*ptr)->info()->serv_addr()); -#endif break; } } @@ -91,13 +76,14 @@ flow DHCP_Flow(is_orig: bool) { case DHCPDISCOVER: BifEvent::generate_dhcp_discover(connection()->bro_analyzer(), connection()->bro_analyzer()->Conn(), - dhcp_msg_val_->Ref(), req_addr); + dhcp_msg_val_->Ref(), new AddrVal(req_addr)); break; case DHCPREQUEST: BifEvent::generate_dhcp_request(connection()->bro_analyzer(), connection()->bro_analyzer()->Conn(), - dhcp_msg_val_->Ref(), req_addr, serv_addr); + dhcp_msg_val_->Ref(), new AddrVal(req_addr), + new AddrVal(serv_addr)); break; case DHCPDECLINE: @@ -129,15 +115,7 @@ flow DHCP_Flow(is_orig: bool) { // RFC 1533 allows a list of router addresses. TableVal* router_list = 0; -#ifdef BROv6 - ::uint32 subnet_mask[4], serv_addr[4]; - - subnet_mask[0] = subnet_mask[1] = - subnet_mask[2] = subnet_mask[3] = 0; - serv_addr[0] = serv_addr[1] = serv_addr[2] = serv_addr[3] = 0; -#else - addr_type subnet_mask = 0, serv_addr = 0; -#endif + ::uint32 subnet_mask = 0, serv_addr = 0; uint32 lease = 0; @@ -146,13 +124,7 @@ flow DHCP_Flow(is_orig: bool) { { switch ( (*ptr)->code() ) { case SUBNET_OPTION: -#ifdef BROv6 - subnet_mask[0] = - subnet_mask[1] = subnet_mask[2] = 0; - subnet_mask[3] = htonl((*ptr)->info()->mask()); -#else subnet_mask = htonl((*ptr)->info()->mask()); -#endif break; case ROUTER_OPTION: @@ -170,14 +142,8 @@ flow DHCP_Flow(is_orig: bool) { vector* rlist = (*ptr)->info()->router_list(); uint32 raddr = (*rlist)[i]; -#ifdef BROv6 - ::uint32 tmp_addr[4]; - tmp_addr[0] = tmp_addr[1] = tmp_addr[2] = 0; - tmp_addr[3] = htonl(raddr); -#else ::uint32 tmp_addr; tmp_addr = htonl(raddr); -#endif // index starting from 1 Val* index = new Val(i + 1, TYPE_COUNT); router_list->Assign(index, new AddrVal(tmp_addr)); @@ -191,11 +157,7 @@ flow DHCP_Flow(is_orig: bool) { break; case SERV_ID_OPTION: -#ifdef BROv6 - serv_addr[3] = htonl((*ptr)->info()->serv_addr()); -#else serv_addr = htonl((*ptr)->info()->serv_addr()); -#endif break; } } @@ -204,15 +166,15 @@ flow DHCP_Flow(is_orig: bool) { case DHCPOFFER: BifEvent::generate_dhcp_offer(connection()->bro_analyzer(), connection()->bro_analyzer()->Conn(), - dhcp_msg_val_->Ref(), subnet_mask, - router_list, lease, serv_addr); + dhcp_msg_val_->Ref(), new AddrVal(subnet_mask), + router_list, lease, new AddrVal(serv_addr)); break; case DHCPACK: BifEvent::generate_dhcp_ack(connection()->bro_analyzer(), connection()->bro_analyzer()->Conn(), - dhcp_msg_val_->Ref(), subnet_mask, - router_list, lease, serv_addr); + dhcp_msg_val_->Ref(), new AddrVal(subnet_mask), + router_list, lease, new AddrVal(serv_addr)); break; case DHCPNAK: diff --git a/src/dns-analyzer.pac b/src/dns-analyzer.pac index 0c2dc1b491..e92b6ef709 100644 --- a/src/dns-analyzer.pac +++ b/src/dns-analyzer.pac @@ -216,44 +216,42 @@ flow DNS_Flow switch ( rr->rr_type() ) { case TYPE_A: + if ( dns_A_reply ) + { + ::uint32 addr = rd->type_a(); + BifEvent::generate_dns_A_reply(connection()->bro_analyzer(), + connection()->bro_analyzer()->Conn(), + dns_msg_val_->Ref(), build_dns_answer(rr), + new AddrVal(htonl(addr))); + } + break; + case TYPE_A6: - case TYPE_AAAA: - if ( ! dns_A_reply ) - break; - -#ifdef BROv6 - ::uint32 addr[4]; -#else - addr_type addr; -#endif - - if ( rr->rr_type() == TYPE_A ) + if ( dns_A6_reply ) { -#ifdef BROv6 - addr[0] = addr[1] = addr[2] = 0; - addr[3] = htonl(rd->type_a()); -#else - addr = htonl(rd->type_a()); -#endif - } - - else - { -#ifdef BROv6 - for ( int i = 0; i < 4; ++i ) + ::uint32 addr[4]; + for ( unsigned int i = 0; i < 4; ++i ) addr[i] = htonl((*rd->type_aaaa())[i]); -#else - addr = htonl((*rd->type_aaaa())[3]); -#endif - } - // For now, we treat A6 and AAAA as A's. Given the - // above fixes for BROv6, we can probably now introduce - // their own events. (It's not clear A6 is needed - - // do we actually encounter it in practice?) - BifEvent::generate_dns_A_reply(connection()->bro_analyzer(), - connection()->bro_analyzer()->Conn(), - dns_msg_val_->Ref(), build_dns_answer(rr), addr); + BifEvent::generate_dns_A6_reply(connection()->bro_analyzer(), + connection()->bro_analyzer()->Conn(), + dns_msg_val_->Ref(), build_dns_answer(rr), + new AddrVal(addr)); + } + break; + + case TYPE_AAAA: + if ( dns_AAAA_reply ) + { + ::uint32 addr[4]; + for ( unsigned int i = 0; i < 4; ++i ) + addr[i] = htonl((*rd->type_aaaa())[i]); + + BifEvent::generate_dns_AAAA_reply(connection()->bro_analyzer(), + connection()->bro_analyzer()->Conn(), + dns_msg_val_->Ref(), build_dns_answer(rr), + new AddrVal(addr)); + } break; case TYPE_NS: diff --git a/src/event.bif b/src/event.bif index d616c4de09..5ef3e8f04b 100644 --- a/src/event.bif +++ b/src/event.bif @@ -454,11 +454,37 @@ event expected_connection_seen%(c: connection, a: count%); ## ## c: The connection the packet is part of. ## -## p: Informattion from the header of the packet that triggered the event. +## p: Information from the header of the packet that triggered the event. ## ## .. bro:see:: tcp_packet packet_contents event new_packet%(c: connection, p: pkt_hdr%); +## Generated for every IPv6 packet that contains extension headers. +## This is potentially an expensive event to handle if analysiing IPv6 traffic +## that happens to utilize extension headers frequently. +## +## c: The connection the packet is part of. +## +## p: Information from the header of the packet that triggered the event. +## +## .. bro:see:: new_packet tcp_packet packet_contents esp_packet +event ipv6_ext_headers%(c: connection, p: pkt_hdr%); + +## Generated for any packets using the IPv6 Encapsulating Security Payload (ESP) +## extension header. +## +## p: Information from the header of the packet that triggered the event. +## +## .. bro:see:: new_packet tcp_packet ipv6_ext_headers +event esp_packet%(p: pkt_hdr%); + +## Generated for any packet using a Mobile IPv6 Mobility Header. +## +## p: Information from the header of the packet that triggered the event. +## +## .. bro:see:: new_packet tcp_packet ipv6_ext_headers +event mobile_ipv6_message%(p: pkt_hdr%); + ## Generated for every packet that has non-empty transport-layer payload. This is a ## very low-level and expensive event that should be avoided when at all possible. ## It's usually infeasible to handle when processing even medium volumes of @@ -798,6 +824,24 @@ event icmp_echo_request%(c: connection, icmp: icmp_conn, id: count, seq: count, ## icmp_time_exceeded icmp_unreachable event icmp_echo_reply%(c: connection, icmp: icmp_conn, id: count, seq: count, payload: string%); +## Generated for all ICMP error messages that are not handled separately with dedicated +## ICMP events. Bro's ICMP analyzer handles a number of ICMP messages directly +## with dedicated events. This handler acts as a fallback for those it doesn't. +## The *icmp* record provides more information about the message. +## +## See `Wikipedia +## `__ for more +## information about the ICMP protocol. +## +## c: The connection record for the corresponding ICMP flow. +## +## icmp: Additional ICMP-specific information augmenting the standard +## connection record *c*. +## +## .. bro:see:: icmp_echo_reply icmp_echo_request icmp_redirect +## icmp_time_exceeded icmp_unreachable +event icmp_error_message%(c: connection, icmp: icmp_conn, code: count, context: icmp_context%); + ## Generated for ICMP *destination unreachable* messages. ## ## See `Wikipedia @@ -821,6 +865,29 @@ event icmp_echo_reply%(c: connection, icmp: icmp_conn, id: count, seq: count, pa ## icmp_time_exceeded event icmp_unreachable%(c: connection, icmp: icmp_conn, code: count, context: icmp_context%); +## Generated for ICMP *packet too big* messages. +## +## See `Wikipedia +## `__ for more +## information about the ICMP protocol. +## +## c: The connection record for the corresponding ICMP flow. +## +## icmp: Additional ICMP-specific information augmenting the standard connection +## record *c*. +## +## code: The ICMP code of the *too big* message. +## +## context: A record with specifics of the original packet that the message refers +## to. *Too big* messages should include the original IP header from the packet +## that triggered them, and Bro parses that into the *context* structure. Note +## that if the *too big* includes only a partial IP header for some reason, no +## fields of *context* will be filled out. +## +## .. bro:see:: icmp_echo_reply icmp_echo_request icmp_redirect icmp_sent +## icmp_time_exceeded +event icmp_packet_too_big%(c: connection, icmp: icmp_conn, code: count, context: icmp_context%); + ## Generated for ICMP *time exceeded* messages. ## ## See `Wikipedia @@ -844,6 +911,115 @@ event icmp_unreachable%(c: connection, icmp: icmp_conn, code: count, context: ic ## icmp_unreachable event icmp_time_exceeded%(c: connection, icmp: icmp_conn, code: count, context: icmp_context%); +## Generated for ICMP *parameter problem* messages. +## +## See `Wikipedia +## `__ for more +## information about the ICMP protocol. +## +## c: The connection record for the corresponding ICMP flow. +## +## icmp: Additional ICMP-specific information augmenting the standard connection +## record *c*. +## +## code: The ICMP code of the *parameter problem* message. +## +## context: A record with specifics of the original packet that the message refers +## to. *Parameter problem* messages should include the original IP header from the packet +## that triggered them, and Bro parses that into the *context* structure. Note that +## if the *parameter problem* includes only a partial IP header for some reason, no fields +## of *context* will be filled out. +## +## .. bro:see:: icmp_echo_reply icmp_echo_request icmp_redirect icmp_sent +## icmp_unreachable +event icmp_parameter_problem%(c: connection, icmp: icmp_conn, code: count, context: icmp_context%); + +## Generated for ICMP *router solicitation* messages. +## +## See `Wikipedia +## `__ for more +## information about the ICMP protocol. +## +## c: The connection record for the corresponding ICMP flow. +## +## icmp: Additional ICMP-specific information augmenting the standard connection +## record *c*. +## +## .. bro:see:: icmp_echo_reply icmp_echo_request icmp_sent +## icmp_time_exceeded icmp_unreachable +event icmp_router_solicitation%(c: connection, icmp: icmp_conn%); + +## Generated for ICMP *router advertisement* messages. +## +## See `Wikipedia +## `__ for more +## information about the ICMP protocol. +## +## c: The connection record for the corresponding ICMP flow. +## +## icmp: Additional ICMP-specific information augmenting the standard connection +## record *c*. +## +## cur_hop_limit: The default value that should be placed in Hop Count field +## for outgoing IP packets. +## +## managed: Managed address configuration flag, :rfc:`4861`. +## +## other: Other stateful configuration flag, :rfc:`4861`. +## +## home_agent: Mobile IPv6 home agent flag, :rfc:`3775`. +## +## pref: Router selection preferences, :rfc:`4191`. +## +## proxy: Neighbor discovery proxy flag, :rfc:`4389`. +## +## rsv: Remaining two reserved bits of router advertisement flags. +## +## .. bro:see:: icmp_echo_reply icmp_echo_request icmp_sent +## icmp_time_exceeded icmp_unreachable +event icmp_router_advertisement%(c: connection, icmp: icmp_conn, cur_hop_limit: count, managed: bool, other: bool, home_agent: bool, pref: count, proxy: bool, rsv: count, router_lifetime: interval, reachable_time: interval, retrans_timer: interval%); + +## Generated for ICMP *neighbor solicitation* messages. +## +## See `Wikipedia +## `__ for more +## information about the ICMP protocol. +## +## c: The connection record for the corresponding ICMP flow. +## +## icmp: Additional ICMP-specific information augmenting the standard connection +## record *c*. +## +## tgt: The IP address of the target of the solicitation. +## +## .. bro:see:: icmp_echo_reply icmp_echo_request icmp_sent +## icmp_time_exceeded icmp_unreachable +event icmp_neighbor_solicitation%(c: connection, icmp: icmp_conn, tgt:addr%); + +## Generated for ICMP *neighbor advertisement* messages. +## +## See `Wikipedia +## `__ for more +## information about the ICMP protocol. +## +## c: The connection record for the corresponding ICMP flow. +## +## icmp: Additional ICMP-specific information augmenting the standard connection +## record *c*. +## +## router: Flag indicating the sender is a router. +## +## solicited: Flag indicating advertisement is in response to a solicitation. +## +## override: Flag indicating advertisement should override existing caches. +## +## tgt: the Target Address in the soliciting message or the address whose +## link-layer address has changed for unsolicited adverts. +## +## .. bro:see:: icmp_echo_reply icmp_echo_request icmp_sent +## icmp_time_exceeded icmp_unreachable +event icmp_neighbor_advertisement%(c: connection, icmp: icmp_conn, router: bool, solicited: bool, override: bool, tgt:addr%); + ## Generated for ICMP *redirect* messages. ## ## See `Wikipedia @@ -855,11 +1031,16 @@ event icmp_time_exceeded%(c: connection, icmp: icmp_conn, code: count, context: ## icmp: Additional ICMP-specific information augmenting the standard connection ## record *c*. ## +## tgt: The address that is supposed to be a better first hop to use for +## ICMP Destination Address. +## +## dest: The address of the destination which is redirected to the target. +## ## a: The new destination address the message is redirecting to. ## ## .. bro:see:: icmp_echo_reply icmp_echo_request icmp_sent ## icmp_time_exceeded icmp_unreachable -event icmp_redirect%(c: connection, icmp: icmp_conn, a: addr%); +event icmp_redirect%(c: connection, icmp: icmp_conn, tgt: addr, dest: addr%); ## Generated when a TCP connection terminated, passing on statistics about the ## two endpoints. This event is always generated when Bro flushes the internal @@ -3732,19 +3913,13 @@ event non_dns_request%(c: connection, msg: string%) &group="dns"; ## ## a: The address returned by the reply. ## -## .. bro:see:: dns_AAAA_reply dns_CNAME_reply dns_EDNS_addl dns_HINFO_reply +## .. bro:see:: dns_AAAA_reply dns_A6_reply dns_CNAME_reply dns_EDNS_addl dns_HINFO_reply ## dns_MX_reply dns_NS_reply dns_PTR_reply dns_SOA_reply dns_SRV_reply ## dns_TSIG_addl dns_TXT_reply dns_WKS_reply dns_end dns_full_request ## dns_mapping_altered dns_mapping_lost_name dns_mapping_new_name ## dns_mapping_unverified dns_mapping_valid dns_message dns_query_reply ## dns_rejected dns_request non_dns_request dns_max_queries dns_session_timeout ## dns_skip_addl dns_skip_all_addl dns_skip_all_auth dns_skip_auth -## -## .. note: This event is currently also raised for ``AAAA`` records. In that -## case, the address *a* will correspond to the lower-order 4 bytes of the -## IPv6 address. This will go away once IPv6 support is improved. -## -## .. todo: IPv6 handling is obviously very broken here ... event dns_A_reply%(c: connection, msg: dns_msg, ans: dns_answer, a: addr%) &group="dns"; ## Generated for DNS replies of type *AAAA*. For replies with multiple answers, an @@ -3762,18 +3937,38 @@ event dns_A_reply%(c: connection, msg: dns_msg, ans: dns_answer, a: addr%) &grou ## ## a: The address returned by the reply. ## -## .. bro:see:: dns_A_reply dns_CNAME_reply dns_EDNS_addl dns_HINFO_reply dns_MX_reply +## .. bro:see:: dns_A_reply dns_A6_reply dns_CNAME_reply dns_EDNS_addl dns_HINFO_reply dns_MX_reply ## dns_NS_reply dns_PTR_reply dns_SOA_reply dns_SRV_reply dns_TSIG_addl ## dns_TXT_reply dns_WKS_reply dns_end dns_full_request dns_mapping_altered ## dns_mapping_lost_name dns_mapping_new_name dns_mapping_unverified ## dns_mapping_valid dns_message dns_query_reply dns_rejected dns_request ## non_dns_request dns_max_queries dns_session_timeout dns_skip_addl ## dns_skip_all_addl dns_skip_all_auth dns_skip_auth +event dns_AAAA_reply%(c: connection, msg: dns_msg, ans: dns_answer, a: addr%) &group="dns"; + +## Generated for DNS replies of type *A6*. For replies with multiple answers, an +## individual event of the corresponding type is raised for each. ## -## .. todo: Raising this event is not implemented currently, not even when -## Bro's compiled IPv6 support. ``AAAA`` are currently always turned into -## :bro:id:`dns_A_reply` events. -event dns_AAAA_reply%(c: connection, msg: dns_msg, ans: dns_answer, a: addr, astr: string%) &group="dns"; +## See `Wikipedia `__ for more +## information about the DNS protocol. Bro analyzes both UDP and TCP DNS sessions. +## +## c: The connection, which may be UDP or TCP depending on the type of the +## transport-layer session being analyzed. +## +## msg: The parsed DNS message header. +## +## ans: The type-independent part of the parsed answer record. +## +## a: The address returned by the reply. +## +## .. bro:see:: dns_A_reply dns_AAAA_reply dns_CNAME_reply dns_EDNS_addl dns_HINFO_reply dns_MX_reply +## dns_NS_reply dns_PTR_reply dns_SOA_reply dns_SRV_reply dns_TSIG_addl +## dns_TXT_reply dns_WKS_reply dns_end dns_full_request dns_mapping_altered +## dns_mapping_lost_name dns_mapping_new_name dns_mapping_unverified +## dns_mapping_valid dns_message dns_query_reply dns_rejected dns_request +## non_dns_request dns_max_queries dns_session_timeout dns_skip_addl +## dns_skip_all_addl dns_skip_all_auth dns_skip_auth +event dns_A6_reply%(c: connection, msg: dns_msg, ans: dns_answer, a: addr%) &group="dns"; ## Generated for DNS replies of type *NS*. For replies with multiple answers, an ## individual event of the corresponding type is raised for each. @@ -4580,7 +4775,6 @@ event ssh_server_version%(c: connection, version: string%); ## ## .. bro:see:: ssl_alert ssl_established ssl_extension ssl_server_hello ## ssl_session_ticket_handshake x509_certificate x509_error x509_extension -## ssl_max_cipherspec_size event ssl_client_hello%(c: connection, version: count, possible_ts: time, session_id: string, ciphers: count_set%); ## Generated for an SSL/TLS servers's initial *hello* message. SSL/TLS sessions @@ -4611,7 +4805,6 @@ event ssl_client_hello%(c: connection, version: count, possible_ts: time, sessio ## ## .. bro:see:: ssl_alert ssl_client_hello ssl_established ssl_extension ## ssl_session_ticket_handshake x509_certificate x509_error x509_extension -## ssl_max_cipherspec_size event ssl_server_hello%(c: connection, version: count, possible_ts: time, session_id: string, cipher: count, comp_method: count%); ## Generated for SSL/TLS extensions seen in an initial handshake. SSL/TLS sessions diff --git a/src/logging.bif b/src/logging.bif index 31e1bebacd..c8960b4e38 100644 --- a/src/logging.bif +++ b/src/logging.bif @@ -3,8 +3,9 @@ module Log; %%{ -#include "LogMgr.h" #include "NetVar.h" + +#include "logging/Manager.h" %%} type Filter: record; diff --git a/src/LogMgr.cc b/src/logging/Manager.cc similarity index 66% rename from src/LogMgr.cc rename to src/logging/Manager.cc index 28e9a2ac1f..9437f0099f 100644 --- a/src/LogMgr.cc +++ b/src/logging/Manager.cc @@ -2,33 +2,42 @@ #include -#include "LogMgr.h" -#include "Event.h" -#include "EventHandler.h" -#include "NetVar.h" -#include "Net.h" +#include "../Event.h" +#include "../EventHandler.h" +#include "../NetVar.h" +#include "../Net.h" -#include "LogWriterAscii.h" -#include "LogWriterNone.h" +#include "threading/SerialTypes.h" + +#include "Manager.h" +#include "WriterFrontend.h" +#include "WriterBackend.h" + +#include "writers/Ascii.h" +#include "writers/None.h" + +using namespace logging; +using threading::Value; +using threading::Field; // Structure describing a log writer type. -struct LogWriterDefinition { +struct WriterDefinition { bro_int_t type; // The type. const char *name; // Descriptive name for error messages. bool (*init)(); // An optional one-time initialization function. - LogWriter* (*factory)(); // A factory function creating instances. + WriterBackend* (*factory)(WriterFrontend* frontend); // A factory function creating instances. }; // Static table defining all availabel log writers. -LogWriterDefinition log_writers[] = { - { BifEnum::Log::WRITER_NONE, "None", 0, LogWriterNone::Instantiate }, - { BifEnum::Log::WRITER_ASCII, "Ascii", 0, LogWriterAscii::Instantiate }, +WriterDefinition log_writers[] = { + { BifEnum::Log::WRITER_NONE, "None", 0, writer::None::Instantiate }, + { BifEnum::Log::WRITER_ASCII, "Ascii", 0, writer::Ascii::Instantiate }, // End marker, don't touch. - { BifEnum::Log::WRITER_DEFAULT, "None", 0, (LogWriter* (*)())0 } + { BifEnum::Log::WRITER_DEFAULT, "None", 0, (WriterBackend* (*)(WriterFrontend* frontend))0 } }; -struct LogMgr::Filter { +struct Manager::Filter { string name; EnumVal* id; Func* pred; @@ -42,7 +51,7 @@ struct LogMgr::Filter { Func* postprocessor; int num_fields; - LogField** fields; + Field** fields; // Vector indexed by field number. Each element is a list of record // indices defining a path leading to the value across potential @@ -52,16 +61,16 @@ struct LogMgr::Filter { ~Filter(); }; -struct LogMgr::WriterInfo { +struct Manager::WriterInfo { EnumVal* type; double open_time; Timer* rotation_timer; double interval; Func* postprocessor; - LogWriter* writer; + WriterFrontend* writer; }; -struct LogMgr::Stream { +struct Manager::Stream { EnumVal* id; bool enabled; string name; @@ -78,317 +87,7 @@ struct LogMgr::Stream { ~Stream(); }; -bool LogField::Read(SerializationFormat* fmt) - { - int t; - int st; - - bool success = (fmt->Read(&name, "name") && fmt->Read(&t, "type") && fmt->Read(&st, "subtype") ); - type = (TypeTag) t; - subtype = (TypeTag) st; - - return success; - } - -bool LogField::Write(SerializationFormat* fmt) const - { - return (fmt->Write(name, "name") && fmt->Write((int)type, "type") && fmt->Write((int)subtype, "subtype")); - } - -LogVal::~LogVal() - { - if ( (type == TYPE_ENUM || type == TYPE_STRING || type == TYPE_FILE || type == TYPE_FUNC) - && present ) - delete val.string_val; - - if ( type == TYPE_TABLE && present ) - { - for ( int i = 0; i < val.set_val.size; i++ ) - delete val.set_val.vals[i]; - - delete [] val.set_val.vals; - } - - if ( type == TYPE_VECTOR && present ) - { - for ( int i = 0; i < val.vector_val.size; i++ ) - delete val.vector_val.vals[i]; - - delete [] val.vector_val.vals; - } - } - -bool LogVal::IsCompatibleType(BroType* t, bool atomic_only) - { - if ( ! t ) - return false; - - switch ( t->Tag() ) { - case TYPE_BOOL: - case TYPE_INT: - case TYPE_COUNT: - case TYPE_COUNTER: - case TYPE_PORT: - case TYPE_SUBNET: - case TYPE_ADDR: - case TYPE_DOUBLE: - case TYPE_TIME: - case TYPE_INTERVAL: - case TYPE_ENUM: - case TYPE_STRING: - case TYPE_FILE: - case TYPE_FUNC: - return true; - - case TYPE_RECORD: - return ! atomic_only; - - case TYPE_TABLE: - { - if ( atomic_only ) - return false; - - if ( ! t->IsSet() ) - return false; - - return IsCompatibleType(t->AsSetType()->Indices()->PureType(), true); - } - - case TYPE_VECTOR: - { - if ( atomic_only ) - return false; - - return IsCompatibleType(t->AsVectorType()->YieldType(), true); - } - - default: - return false; - } - - return false; - } - -bool LogVal::Read(SerializationFormat* fmt) - { - int ty; - - if ( ! (fmt->Read(&ty, "type") && fmt->Read(&present, "present")) ) - return false; - - type = (TypeTag)(ty); - - if ( ! present ) - return true; - - switch ( type ) { - case TYPE_BOOL: - case TYPE_INT: - return fmt->Read(&val.int_val, "int"); - - case TYPE_COUNT: - case TYPE_COUNTER: - case TYPE_PORT: - return fmt->Read(&val.uint_val, "uint"); - - case TYPE_SUBNET: - { - uint32 net[4]; - if ( ! (fmt->Read(&net[0], "net0") && - fmt->Read(&net[1], "net1") && - fmt->Read(&net[2], "net2") && - fmt->Read(&net[3], "net3") && - fmt->Read(&val.subnet_val.width, "width")) ) - return false; - -#ifdef BROv6 - val.subnet_val.net[0] = net[0]; - val.subnet_val.net[1] = net[1]; - val.subnet_val.net[2] = net[2]; - val.subnet_val.net[3] = net[3]; -#else - val.subnet_val.net = net[0]; -#endif - return true; - } - - case TYPE_ADDR: - { - uint32 addr[4]; - if ( ! (fmt->Read(&addr[0], "addr0") && - fmt->Read(&addr[1], "addr1") && - fmt->Read(&addr[2], "addr2") && - fmt->Read(&addr[3], "addr3")) ) - return false; - - val.addr_val[0] = addr[0]; -#ifdef BROv6 - val.addr_val[1] = addr[1]; - val.addr_val[2] = addr[2]; - val.addr_val[3] = addr[3]; -#endif - return true; - } - - case TYPE_DOUBLE: - case TYPE_TIME: - case TYPE_INTERVAL: - return fmt->Read(&val.double_val, "double"); - - case TYPE_ENUM: - case TYPE_STRING: - case TYPE_FILE: - case TYPE_FUNC: - { - val.string_val = new string; - return fmt->Read(val.string_val, "string"); - } - - case TYPE_TABLE: - { - if ( ! fmt->Read(&val.set_val.size, "set_size") ) - return false; - - val.set_val.vals = new LogVal* [val.set_val.size]; - - for ( int i = 0; i < val.set_val.size; ++i ) - { - val.set_val.vals[i] = new LogVal; - - if ( ! val.set_val.vals[i]->Read(fmt) ) - return false; - } - - return true; - } - - case TYPE_VECTOR: - { - if ( ! fmt->Read(&val.vector_val.size, "vector_size") ) - return false; - - val.vector_val.vals = new LogVal* [val.vector_val.size]; - - for ( int i = 0; i < val.vector_val.size; ++i ) - { - val.vector_val.vals[i] = new LogVal; - - if ( ! val.vector_val.vals[i]->Read(fmt) ) - return false; - } - - return true; - } - - default: - reporter->InternalError("unsupported type %s in LogVal::Write", type_name(type)); - } - - return false; - } - -bool LogVal::Write(SerializationFormat* fmt) const - { - if ( ! (fmt->Write((int)type, "type") && - fmt->Write(present, "present")) ) - return false; - - if ( ! present ) - return true; - - switch ( type ) { - case TYPE_BOOL: - case TYPE_INT: - return fmt->Write(val.int_val, "int"); - - case TYPE_COUNT: - case TYPE_COUNTER: - case TYPE_PORT: - return fmt->Write(val.uint_val, "uint"); - - case TYPE_SUBNET: - { - uint32 net[4]; -#ifdef BROv6 - net[0] = val.subnet_val.net[0]; - net[1] = val.subnet_val.net[1]; - net[2] = val.subnet_val.net[2]; - net[3] = val.subnet_val.net[3]; -#else - net[0] = val.subnet_val.net; - net[1] = net[2] = net[3] = 0; -#endif - return fmt->Write(net[0], "net0") && - fmt->Write(net[1], "net1") && - fmt->Write(net[2], "net2") && - fmt->Write(net[3], "net3") && - fmt->Write(val.subnet_val.width, "width"); - } - - case TYPE_ADDR: - { - uint32 addr[4]; - addr[0] = val.addr_val[0]; -#ifdef BROv6 - addr[1] = val.addr_val[1]; - addr[2] = val.addr_val[2]; - addr[3] = val.addr_val[3]; -#else - addr[1] = addr[2] = addr[3] = 0; -#endif - return fmt->Write(addr[0], "addr0") && - fmt->Write(addr[1], "addr1") && - fmt->Write(addr[2], "addr2") && - fmt->Write(addr[3], "addr3"); - } - - case TYPE_DOUBLE: - case TYPE_TIME: - case TYPE_INTERVAL: - return fmt->Write(val.double_val, "double"); - - case TYPE_ENUM: - case TYPE_STRING: - case TYPE_FILE: - case TYPE_FUNC: - return fmt->Write(*val.string_val, "string"); - - case TYPE_TABLE: - { - if ( ! fmt->Write(val.set_val.size, "set_size") ) - return false; - - for ( int i = 0; i < val.set_val.size; ++i ) - { - if ( ! val.set_val.vals[i]->Write(fmt) ) - return false; - } - - return true; - } - - case TYPE_VECTOR: - { - if ( ! fmt->Write(val.vector_val.size, "vector_size") ) - return false; - - for ( int i = 0; i < val.vector_val.size; ++i ) - { - if ( ! val.vector_val.vals[i]->Write(fmt) ) - return false; - } - - return true; - } - - default: - reporter->InternalError("unsupported type %s in LogVal::REad", type_name(type)); - } - - return false; - } - -LogMgr::Filter::~Filter() +Manager::Filter::~Filter() { for ( int i = 0; i < num_fields; ++i ) delete fields[i]; @@ -398,7 +97,7 @@ LogMgr::Filter::~Filter() Unref(path_val); } -LogMgr::Stream::~Stream() +Manager::Stream::~Stream() { Unref(columns); @@ -406,9 +105,6 @@ LogMgr::Stream::~Stream() { WriterInfo* winfo = i->second; - if ( ! winfo ) - continue; - if ( winfo->rotation_timer ) timer_mgr->Cancel(winfo->rotation_timer); @@ -421,17 +117,73 @@ LogMgr::Stream::~Stream() delete *f; } -LogMgr::LogMgr() +Manager::Manager() { } -LogMgr::~LogMgr() +Manager::~Manager() { for ( vector::iterator s = streams.begin(); s != streams.end(); ++s ) delete *s; } -LogMgr::Stream* LogMgr::FindStream(EnumVal* id) +WriterBackend* Manager::CreateBackend(WriterFrontend* frontend, bro_int_t type) + { + WriterDefinition* ld = log_writers; + + while ( true ) + { + if ( ld->type == BifEnum::Log::WRITER_DEFAULT ) + { + reporter->Error("unknow writer when creating writer"); + return 0; + } + + if ( ld->type != type ) + { + // Not the right one. + ++ld; + continue; + } + + // If the writer has an init function, call it. + if ( ld->init ) + { + if ( (*ld->init)() ) + // Clear the init function so that we won't + // call it again later. + ld->init = 0; + else + { + // Init failed, disable by deleting factory + // function. + ld->factory = 0; + + DBG_LOG(DBG_LOGGING, "failed to init writer class %s", + ld->name); + + return false; + } + } + + if ( ! ld->factory ) + // Oops, we can't instantiate this guy. + return 0; + + // All done. + break; + } + + assert(ld->factory); + + WriterBackend* backend = (*ld->factory)(frontend); + assert(backend); + + frontend->ty_name = ld->name; + return backend; + } + +Manager::Stream* Manager::FindStream(EnumVal* id) { unsigned int idx = id->AsEnum(); @@ -441,7 +193,7 @@ LogMgr::Stream* LogMgr::FindStream(EnumVal* id) return streams[idx]; } -LogMgr::WriterInfo* LogMgr::FindWriter(LogWriter* writer) +Manager::WriterInfo* Manager::FindWriter(WriterFrontend* writer) { for ( vector::iterator s = streams.begin(); s != streams.end(); ++s ) { @@ -452,7 +204,7 @@ LogMgr::WriterInfo* LogMgr::FindWriter(LogWriter* writer) { WriterInfo* winfo = i->second; - if ( winfo && winfo->writer == writer ) + if ( winfo->writer == writer ) return winfo; } } @@ -460,14 +212,15 @@ LogMgr::WriterInfo* LogMgr::FindWriter(LogWriter* writer) return 0; } -void LogMgr::RemoveDisabledWriters(Stream* stream) +void Manager::RemoveDisabledWriters(Stream* stream) { list disabled; for ( Stream::WriterMap::iterator j = stream->writers.begin(); j != stream->writers.end(); j++ ) { - if ( j->second && j->second->writer->Disabled() ) + if ( j->second->writer->Disabled() ) { + j->second->writer->Stop(); delete j->second; disabled.push_back(j->first); } @@ -477,7 +230,7 @@ void LogMgr::RemoveDisabledWriters(Stream* stream) stream->writers.erase(*j); } -bool LogMgr::CreateStream(EnumVal* id, RecordVal* sval) +bool Manager::CreateStream(EnumVal* id, RecordVal* sval) { RecordType* rtype = sval->Type()->AsRecordType(); @@ -497,7 +250,7 @@ bool LogMgr::CreateStream(EnumVal* id, RecordVal* sval) if ( ! (columns->FieldDecl(i)->FindAttr(ATTR_LOG)) ) continue; - if ( ! LogVal::IsCompatibleType(columns->FieldType(i)) ) + if ( ! threading::Value::IsCompatibleType(columns->FieldType(i)) ) { reporter->Error("type of field '%s' is not support for logging output", columns->FieldName(i)); @@ -569,7 +322,7 @@ bool LogMgr::CreateStream(EnumVal* id, RecordVal* sval) return true; } -bool LogMgr::EnableStream(EnumVal* id) +bool Manager::EnableStream(EnumVal* id) { Stream* stream = FindStream(id); @@ -585,7 +338,7 @@ bool LogMgr::EnableStream(EnumVal* id) return true; } -bool LogMgr::DisableStream(EnumVal* id) +bool Manager::DisableStream(EnumVal* id) { Stream* stream = FindStream(id); @@ -602,7 +355,7 @@ bool LogMgr::DisableStream(EnumVal* id) } // Helper for recursive record field unrolling. -bool LogMgr::TraverseRecord(Stream* stream, Filter* filter, RecordType* rt, +bool Manager::TraverseRecord(Stream* stream, Filter* filter, RecordType* rt, TableVal* include, TableVal* exclude, string path, list indices) { for ( int i = 0; i < rt->NumFields(); ++i ) @@ -696,9 +449,9 @@ bool LogMgr::TraverseRecord(Stream* stream, Filter* filter, RecordType* rt, filter->indices.push_back(new_indices); - filter->fields = (LogField**) + filter->fields = (Field**) realloc(filter->fields, - sizeof(LogField) * ++filter->num_fields); + sizeof(Field) * ++filter->num_fields); if ( ! filter->fields ) { @@ -706,24 +459,23 @@ bool LogMgr::TraverseRecord(Stream* stream, Filter* filter, RecordType* rt, return false; } - LogField* field = new LogField(); + Field* field = new Field(); field->name = new_path; field->type = t->Tag(); - if ( field->type == TYPE_TABLE ) - { - field->subtype = t->AsSetType()->Indices()->PureType()->Tag(); - } - else if ( field->type == TYPE_VECTOR ) - { - field->subtype = t->AsVectorType()->YieldType()->Tag(); - } + + if ( field->type == TYPE_TABLE ) + field->subtype = t->AsSetType()->Indices()->PureType()->Tag(); + + else if ( field->type == TYPE_VECTOR ) + field->subtype = t->AsVectorType()->YieldType()->Tag(); + filter->fields[filter->num_fields - 1] = field; } return true; } -bool LogMgr::AddFilter(EnumVal* id, RecordVal* fval) +bool Manager::AddFilter(EnumVal* id, RecordVal* fval) { RecordType* rtype = fval->Type()->AsRecordType(); @@ -819,7 +571,7 @@ bool LogMgr::AddFilter(EnumVal* id, RecordVal* fval) for ( int i = 0; i < filter->num_fields; i++ ) { - LogField* field = filter->fields[i]; + Field* field = filter->fields[i]; DBG_LOG(DBG_LOGGING, " field %10s: %s", field->name.c_str(), type_name(field->type)); } @@ -828,12 +580,12 @@ bool LogMgr::AddFilter(EnumVal* id, RecordVal* fval) return true; } -bool LogMgr::RemoveFilter(EnumVal* id, StringVal* name) +bool Manager::RemoveFilter(EnumVal* id, StringVal* name) { return RemoveFilter(id, name->AsString()->CheckString()); } -bool LogMgr::RemoveFilter(EnumVal* id, string name) +bool Manager::RemoveFilter(EnumVal* id, string name) { Stream* stream = FindStream(id); if ( ! stream ) @@ -860,7 +612,7 @@ bool LogMgr::RemoveFilter(EnumVal* id, string name) return true; } -bool LogMgr::Write(EnumVal* id, RecordVal* columns) +bool Manager::Write(EnumVal* id, RecordVal* columns) { bool error = false; @@ -924,11 +676,11 @@ bool LogMgr::Write(EnumVal* id, RecordVal* columns) Val* path_arg; if ( filter->path_val ) - path_arg = filter->path_val; + path_arg = filter->path_val->Ref(); else path_arg = new StringVal(""); - vl.append(path_arg->Ref()); + vl.append(path_arg); Val* rec_arg; BroType* rt = filter->path_func->FType()->Args()->FieldType("rec"); @@ -962,7 +714,6 @@ bool LogMgr::Write(EnumVal* id, RecordVal* columns) if ( ! filter->path_val ) { - Unref(path_arg); filter->path = v->AsString()->CheckString(); filter->path_val = v->Ref(); } @@ -980,83 +731,42 @@ bool LogMgr::Write(EnumVal* id, RecordVal* columns) Stream::WriterMap::iterator w = stream->writers.find(Stream::WriterPathPair(filter->writer->AsEnum(), path)); - LogWriter* writer = 0; + WriterFrontend* writer = 0; if ( w != stream->writers.end() ) // We know this writer already. - writer = w->second ? w->second->writer : 0; + writer = w->second->writer; else { // No, need to create one. - // Copy the fields for LogWriter::Init() as it will take - // ownership. - LogField** arg_fields = new LogField*[filter->num_fields]; + // Copy the fields for WriterFrontend::Init() as it + // will take ownership. + Field** arg_fields = new Field*[filter->num_fields]; for ( int j = 0; j < filter->num_fields; ++j ) - arg_fields[j] = new LogField(*filter->fields[j]); + arg_fields[j] = new Field(*filter->fields[j]); - if ( filter->remote ) - remote_serializer->SendLogCreateWriter(stream->id, - filter->writer, - path, - filter->num_fields, - arg_fields); + writer = CreateWriter(stream->id, filter->writer, + path, filter->num_fields, + arg_fields, filter->local, filter->remote); - if ( filter->local ) + if ( ! writer ) { - writer = CreateWriter(stream->id, filter->writer, - path, filter->num_fields, - arg_fields); - - if ( ! writer ) - { - Unref(columns); - return false; - } + Unref(columns); + return false; } - else - { - // Insert a null pointer into the map to make - // sure we don't try creating it again. - stream->writers.insert(Stream::WriterMap::value_type( - Stream::WriterPathPair(filter->writer->AsEnum(), path), 0)); - for( int i = 0; i < filter->num_fields; ++i) - delete arg_fields[i]; - - delete [] arg_fields; - } } // Alright, can do the write now. - if ( filter->local || filter->remote ) - { - LogVal** vals = RecordToFilterVals(stream, filter, columns); - - if ( filter->remote ) - remote_serializer->SendLogWrite(stream->id, - filter->writer, - path, - filter->num_fields, - vals); - - if ( filter->local ) - { - assert(writer); - - // Write takes ownership of vals. - if ( ! writer->Write(filter->num_fields, vals) ) - error = true; - } - - else - DeleteVals(filter->num_fields, vals); - - } + threading::Value** vals = RecordToFilterVals(stream, filter, columns); + // Write takes ownership of vals. + assert(writer); + writer->Write(filter->num_fields, vals); #ifdef DEBUG DBG_LOG(DBG_LOGGING, "Wrote record to filter '%s' on stream '%s'", @@ -1072,15 +782,15 @@ bool LogMgr::Write(EnumVal* id, RecordVal* columns) return true; } -LogVal* LogMgr::ValToLogVal(Val* val, BroType* ty) +threading::Value* Manager::ValToLogVal(Val* val, BroType* ty) { if ( ! ty ) ty = val->Type(); if ( ! val ) - return new LogVal(ty->Tag(), false); + return new threading::Value(ty->Tag(), false); - LogVal* lval = new LogVal(ty->Tag()); + threading::Value* lval = new threading::Value(ty->Tag()); switch ( lval->type ) { case TYPE_BOOL: @@ -1103,23 +813,17 @@ LogVal* LogMgr::ValToLogVal(Val* val, BroType* ty) break; case TYPE_PORT: - lval->val.uint_val = val->AsPortVal()->Port(); + lval->val.port_val.port = val->AsPortVal()->Port(); + lval->val.port_val.proto = val->AsPortVal()->PortType(); break; case TYPE_SUBNET: - lval->val.subnet_val = *val->AsSubNet(); + val->AsSubNet().ConvertToThreadingValue(&lval->val.subnet_val); break; case TYPE_ADDR: - { - addr_type t = val->AsAddr(); -#ifdef BROv6 - copy_addr(t, lval->val.addr_val); -#else - copy_addr(&t, lval->val.addr_val); -#endif + val->AsAddr().ConvertToThreadingValue(&lval->val.addr_val); break; - } case TYPE_DOUBLE: case TYPE_TIME: @@ -1160,7 +864,7 @@ LogVal* LogMgr::ValToLogVal(Val* val, BroType* ty) set = new ListVal(TYPE_INT); lval->val.set_val.size = set->Length(); - lval->val.set_val.vals = new LogVal* [lval->val.set_val.size]; + lval->val.set_val.vals = new threading::Value* [lval->val.set_val.size]; for ( int i = 0; i < lval->val.set_val.size; i++ ) lval->val.set_val.vals[i] = ValToLogVal(set->Index(i)); @@ -1174,7 +878,7 @@ LogVal* LogMgr::ValToLogVal(Val* val, BroType* ty) VectorVal* vec = val->AsVectorVal(); lval->val.vector_val.size = vec->Size(); lval->val.vector_val.vals = - new LogVal* [lval->val.vector_val.size]; + new threading::Value* [lval->val.vector_val.size]; for ( int i = 0; i < lval->val.vector_val.size; i++ ) { @@ -1193,10 +897,10 @@ LogVal* LogMgr::ValToLogVal(Val* val, BroType* ty) return lval; } -LogVal** LogMgr::RecordToFilterVals(Stream* stream, Filter* filter, +Value** Manager::RecordToFilterVals(Stream* stream, Filter* filter, RecordVal* columns) { - LogVal** vals = new LogVal*[filter->num_fields]; + Value** vals = new Value*[filter->num_fields]; for ( int i = 0; i < filter->num_fields; ++i ) { @@ -1215,7 +919,7 @@ LogVal** LogMgr::RecordToFilterVals(Stream* stream, Filter* filter, if ( ! val ) { // Value, or any of its parents, is not set. - vals[i] = new LogVal(filter->fields[i]->type, false); + vals[i] = new Value(filter->fields[i]->type, false); break; } } @@ -1227,8 +931,8 @@ LogVal** LogMgr::RecordToFilterVals(Stream* stream, Filter* filter, return vals; } -LogWriter* LogMgr::CreateWriter(EnumVal* id, EnumVal* writer, string path, - int num_fields, LogField** fields) +WriterFrontend* Manager::CreateWriter(EnumVal* id, EnumVal* writer, string path, + int num_fields, const Field* const* fields, bool local, bool remote) { Stream* stream = FindStream(id); @@ -1239,61 +943,15 @@ LogWriter* LogMgr::CreateWriter(EnumVal* id, EnumVal* writer, string path, Stream::WriterMap::iterator w = stream->writers.find(Stream::WriterPathPair(writer->AsEnum(), path)); - if ( w != stream->writers.end() && w->second ) + if ( w != stream->writers.end() ) // If we already have a writer for this. That's fine, we just // return it. return w->second->writer; - // Need to instantiate a new writer. + WriterFrontend* writer_obj = new WriterFrontend(id, writer, local, remote); + assert(writer_obj); - LogWriterDefinition* ld = log_writers; - - while ( true ) - { - if ( ld->type == BifEnum::Log::WRITER_DEFAULT ) - { - reporter->Error("unknow writer when creating writer"); - return 0; - } - - if ( ld->type == writer->AsEnum() ) - break; - - if ( ! ld->factory ) - // Oops, we can't instantiate this guy. - return 0; - - // If the writer has an init function, call it. - if ( ld->init ) - { - if ( (*ld->init)() ) - // Clear the init function so that we won't - // call it again later. - ld->init = 0; - else - // Init failed, disable by deleting factory - // function. - ld->factory = 0; - - DBG_LOG(DBG_LOGGING, "failed to init writer class %s", - ld->name); - - return false; - } - - ++ld; - } - - assert(ld->factory); - LogWriter* writer_obj = (*ld->factory)(); - - if ( ! writer_obj->Init(path, num_fields, fields) ) - { - DBG_LOG(DBG_LOGGING, "failed to init instance of writer %s", - ld->name); - - return 0; - } + writer_obj->Init(path, num_fields, fields); WriterInfo* winfo = new WriterInfo; winfo->type = writer->Ref()->AsEnumVal(); @@ -1338,16 +996,17 @@ LogWriter* LogMgr::CreateWriter(EnumVal* id, EnumVal* writer, string path, return writer_obj; } -void LogMgr::DeleteVals(int num_fields, LogVal** vals) +void Manager::DeleteVals(int num_fields, Value** vals) { + // Note this code is duplicated in WriterBackend::DeleteVals(). for ( int i = 0; i < num_fields; i++ ) delete vals[i]; delete [] vals; } -bool LogMgr::Write(EnumVal* id, EnumVal* writer, string path, int num_fields, - LogVal** vals) +bool Manager::Write(EnumVal* id, EnumVal* writer, string path, int num_fields, + Value** vals) { Stream* stream = FindStream(id); @@ -1357,7 +1016,7 @@ bool LogMgr::Write(EnumVal* id, EnumVal* writer, string path, int num_fields, #ifdef DEBUG ODesc desc; id->Describe(&desc); - DBG_LOG(DBG_LOGGING, "unknown stream %s in LogMgr::Write()", + DBG_LOG(DBG_LOGGING, "unknown stream %s in Manager::Write()", desc.Description()); #endif DeleteVals(num_fields, vals); @@ -1379,23 +1038,23 @@ bool LogMgr::Write(EnumVal* id, EnumVal* writer, string path, int num_fields, #ifdef DEBUG ODesc desc; id->Describe(&desc); - DBG_LOG(DBG_LOGGING, "unknown writer %s in LogMgr::Write()", + DBG_LOG(DBG_LOGGING, "unknown writer %s in Manager::Write()", desc.Description()); #endif DeleteVals(num_fields, vals); return false; } - bool success = (w->second ? w->second->writer->Write(num_fields, vals) : true); + w->second->writer->Write(num_fields, vals); DBG_LOG(DBG_LOGGING, - "Wrote pre-filtered record to path '%s' on stream '%s' [%s]", - path.c_str(), stream->name.c_str(), (success ? "ok" : "error")); + "Wrote pre-filtered record to path '%s' on stream '%s'", + path.c_str(), stream->name.c_str()); - return success; + return true; } -void LogMgr::SendAllWritersTo(RemoteSerializer::PeerID peer) +void Manager::SendAllWritersTo(RemoteSerializer::PeerID peer) { for ( vector::iterator s = streams.begin(); s != streams.end(); ++s ) { @@ -1407,10 +1066,7 @@ void LogMgr::SendAllWritersTo(RemoteSerializer::PeerID peer) for ( Stream::WriterMap::iterator i = stream->writers.begin(); i != stream->writers.end(); i++ ) { - if ( ! i->second ) - continue; - - LogWriter* writer = i->second->writer; + WriterFrontend* writer = i->second->writer; EnumVal writer_val(i->first.first, BifType::Enum::Log::Writer); remote_serializer->SendLogCreateWriter(peer, (*s)->id, @@ -1422,7 +1078,7 @@ void LogMgr::SendAllWritersTo(RemoteSerializer::PeerID peer) } } -bool LogMgr::SetBuf(EnumVal* id, bool enabled) +bool Manager::SetBuf(EnumVal* id, bool enabled) { Stream* stream = FindStream(id); if ( ! stream ) @@ -1430,17 +1086,14 @@ bool LogMgr::SetBuf(EnumVal* id, bool enabled) for ( Stream::WriterMap::iterator i = stream->writers.begin(); i != stream->writers.end(); i++ ) - { - if ( i->second ) - i->second->writer->SetBuf(enabled); - } + i->second->writer->SetBuf(enabled); RemoveDisabledWriters(stream); return true; } -bool LogMgr::Flush(EnumVal* id) +bool Manager::Flush(EnumVal* id) { Stream* stream = FindStream(id); if ( ! stream ) @@ -1451,26 +1104,26 @@ bool LogMgr::Flush(EnumVal* id) for ( Stream::WriterMap::iterator i = stream->writers.begin(); i != stream->writers.end(); i++ ) - { - if ( i->second ) - i->second->writer->Flush(); - } + i->second->writer->Flush(); RemoveDisabledWriters(stream); return true; } -void LogMgr::Error(LogWriter* writer, const char* msg) +void Manager::Terminate() { - reporter->Error("error with writer for %s: %s", - writer->Path().c_str(), msg); + for ( vector::iterator s = streams.begin(); s != streams.end(); ++s ) + { + if ( *s ) + Flush((*s)->id); + } } // Timer which on dispatching rotates the filter. class RotationTimer : public Timer { public: - RotationTimer(double t, LogMgr::WriterInfo* arg_winfo, bool arg_rotate) + RotationTimer(double t, Manager::WriterInfo* arg_winfo, bool arg_rotate) : Timer(t, TIMER_ROTATE) { winfo = arg_winfo; @@ -1482,7 +1135,7 @@ public: void Dispatch(double t, int is_expire); protected: - LogMgr::WriterInfo* winfo; + Manager::WriterInfo* winfo; bool rotate; }; @@ -1506,7 +1159,7 @@ void RotationTimer::Dispatch(double t, int is_expire) } } -void LogMgr::InstallRotationTimer(WriterInfo* winfo) +void Manager::InstallRotationTimer(WriterInfo* winfo) { if ( terminating ) return; @@ -1548,7 +1201,7 @@ void LogMgr::InstallRotationTimer(WriterInfo* winfo) } } -void LogMgr::Rotate(WriterInfo* winfo) +void Manager::Rotate(WriterInfo* winfo) { DBG_LOG(DBG_LOGGING, "Rotating %s at %.6f", winfo->writer->Path().c_str(), network_time); @@ -1568,7 +1221,7 @@ void LogMgr::Rotate(WriterInfo* winfo) winfo->writer->Rotate(tmp, winfo->open_time, network_time, terminating); } -bool LogMgr::FinishedRotation(LogWriter* writer, string new_name, string old_name, +bool Manager::FinishedRotation(WriterFrontend* writer, string new_name, string old_name, double open, double close, bool terminating) { DBG_LOG(DBG_LOGGING, "Finished rotating %s at %.6f, new name %s", diff --git a/src/logging/Manager.h b/src/logging/Manager.h new file mode 100644 index 0000000000..bf097c5e1a --- /dev/null +++ b/src/logging/Manager.h @@ -0,0 +1,205 @@ +// See the file "COPYING" in the main distribution directory for copyright. +// +// A class managing log writers and filters. + +#ifndef LOGGING_MANAGER_H +#define LOGGING_MANAGER_H + +#include "../Val.h" +#include "../EventHandler.h" +#include "../RemoteSerializer.h" + +class SerializationFormat; +class RemoteSerializer; +class RotationTimer; + +namespace logging { + + +class WriterBackend; +class WriterFrontend; +class RotationFinishedMessage; + +/** + * Singleton class for managing log streams. + */ +class Manager { +public: + /** + * Constructor. + */ + Manager(); + + /** + * Destructor. + */ + ~Manager(); + + /** + * Creates a new log stream. + * + * @param id The enum value corresponding the log stream. + * + * @param stream A record of script type \c Log::Stream. + * + * This method corresponds directly to the internal BiF defined in + * logging.bif, which just forwards here. + */ + bool CreateStream(EnumVal* id, RecordVal* stream); + + /** + * Enables a log log stream. + * + * @param id The enum value corresponding the log stream. + * + * This method corresponds directly to the internal BiF defined in + * logging.bif, which just forwards here. + */ + bool EnableStream(EnumVal* id); + + /** + * Disables a log stream. + * + * @param id The enum value corresponding the log stream. + * + * This methods corresponds directly to the internal BiF defined in + * logging.bif, which just forwards here. + */ + bool DisableStream(EnumVal* id); + + /** + * Adds a filter to a log stream. + * + * @param id The enum value corresponding the log stream. + * + * @param filter A record of script type \c Log::Filter. + * + * This methods corresponds directly to the internal BiF defined in + * logging.bif, which just forwards here. + */ + bool AddFilter(EnumVal* id, RecordVal* filter); + + /** + * Removes a filter from a log stream. + * + * @param id The enum value corresponding the log stream. + * + * @param name The name of the filter to remove. + * + * This methods corresponds directly to the internal BiF defined in + * logging.bif, which just forwards here. + */ + bool RemoveFilter(EnumVal* id, StringVal* name); + + /** + * Removes a filter from a log stream. + * + * @param id The enum value corresponding the log stream. + * + * @param name The name of the filter to remove. + * + * This methods corresponds directly to the internal BiF defined in + * logging.bif, which just forwards here. + */ + bool RemoveFilter(EnumVal* id, string name); + + /** + * Write a record to a log stream. + * + * @param id The enum value corresponding the log stream. + * + * @param colums A record of the type defined for the stream's + * columns. + * + * This methods corresponds directly to the internal BiF defined in + * logging.bif, which just forwards here. + */ + bool Write(EnumVal* id, RecordVal* columns); + + /** + * Sets log streams buffering state. This adjusts all associated + * writers to the new state. + * + * @param id The enum value corresponding the log stream. + * + * @param enabled False to disable buffering (default is enabled). + * + * This methods corresponds directly to the internal BiF defined in + * logging.bif, which just forwards here. + */ + bool SetBuf(EnumVal* id, bool enabled); + + /** + * Flushes a log stream. This flushed all associated writers. + * + * @param id The enum value corresponding the log stream. + * + * This methods corresponds directly to the internal BiF defined in + * logging.bif, which just forwards here. + */ + bool Flush(EnumVal* id); + + /** + * Prepares the log manager to terminate. This will flush all log + * stream. + */ + void Terminate(); + +protected: + friend class WriterFrontend; + friend class RotationFinishedMessage; + friend class ::RemoteSerializer; + friend class ::RotationTimer; + + // Instantiates a new WriterBackend of the given type (note that + // doing so creates a new thread!). + WriterBackend* CreateBackend(WriterFrontend* frontend, bro_int_t type); + + //// Function also used by the RemoteSerializer. + + // Takes ownership of fields. + WriterFrontend* CreateWriter(EnumVal* id, EnumVal* writer, string path, + int num_fields, const threading::Field* const* fields, + bool local, bool remote); + + // Takes ownership of values.. + bool Write(EnumVal* id, EnumVal* writer, string path, + int num_fields, threading::Value** vals); + + // Announces all instantiated writers to peer. + void SendAllWritersTo(RemoteSerializer::PeerID peer); + + // Signals that a file has been rotated. + bool FinishedRotation(WriterFrontend* writer, string new_name, string old_name, + double open, double close, bool terminating); + + // Deletes the values as passed into Write(). + void DeleteVals(int num_fields, threading::Value** vals); + +private: + struct Filter; + struct Stream; + struct WriterInfo; + + bool TraverseRecord(Stream* stream, Filter* filter, RecordType* rt, + TableVal* include, TableVal* exclude, string path, list indices); + + threading::Value** RecordToFilterVals(Stream* stream, Filter* filter, + RecordVal* columns); + + threading::Value* ValToLogVal(Val* val, BroType* ty = 0); + Stream* FindStream(EnumVal* id); + void RemoveDisabledWriters(Stream* stream); + void InstallRotationTimer(WriterInfo* winfo); + void Rotate(WriterInfo* info); + Filter* FindFilter(EnumVal* id, StringVal* filter); + WriterInfo* FindWriter(WriterFrontend* writer); + + vector streams; // Indexed by stream enum. +}; + +} + +extern logging::Manager* log_mgr; + +#endif diff --git a/src/logging/WriterBackend.cc b/src/logging/WriterBackend.cc new file mode 100644 index 0000000000..7c71c09604 --- /dev/null +++ b/src/logging/WriterBackend.cc @@ -0,0 +1,281 @@ +// See the file "COPYING" in the main distribution directory for copyright. + +#include "util.h" +#include "threading/SerialTypes.h" + +#include "WriterBackend.h" +#include "WriterFrontend.h" + +// Messages sent from backend to frontend (i.e., "OutputMessages"). + +using threading::Value; +using threading::Field; + +namespace logging { + +class RotationFinishedMessage : public threading::OutputMessage +{ +public: + RotationFinishedMessage(WriterFrontend* writer, string new_name, string old_name, + double open, double close, bool terminating) + : threading::OutputMessage("RotationFinished", writer), + new_name(new_name), old_name(old_name), open(open), + close(close), terminating(terminating) { } + + virtual bool Process() + { + return log_mgr->FinishedRotation(Object(), new_name, old_name, open, close, terminating); + } + +private: + string new_name; + string old_name; + double open; + double close; + bool terminating; +}; + +class FlushWriteBufferMessage : public threading::OutputMessage +{ +public: + FlushWriteBufferMessage(WriterFrontend* writer) + : threading::OutputMessage("FlushWriteBuffer", writer) {} + + virtual bool Process() { Object()->FlushWriteBuffer(); return true; } +}; + +class DisableMessage : public threading::OutputMessage +{ +public: + DisableMessage(WriterFrontend* writer) + : threading::OutputMessage("Disable", writer) {} + + virtual bool Process() { Object()->SetDisable(); return true; } +}; + +} + +// Backend methods. + +using namespace logging; + +WriterBackend::WriterBackend(WriterFrontend* arg_frontend) : MsgThread() + { + path = ""; + num_fields = 0; + fields = 0; + buffering = true; + frontend = arg_frontend; + + SetName(frontend->Name()); + } + +WriterBackend::~WriterBackend() + { + if ( fields ) + { + for(int i = 0; i < num_fields; ++i) + delete fields[i]; + + delete [] fields; + } + } + +void WriterBackend::DeleteVals(int num_writes, Value*** vals) + { + for ( int j = 0; j < num_writes; ++j ) + { + // Note this code is duplicated in Manager::DeleteVals(). + for ( int i = 0; i < num_fields; i++ ) + delete vals[j][i]; + + delete [] vals[j]; + } + + delete [] vals; + } + +bool WriterBackend::FinishedRotation(string new_name, string old_name, + double open, double close, bool terminating) + { + SendOut(new RotationFinishedMessage(frontend, new_name, old_name, open, close, terminating)); + return true; + } + +void WriterBackend::DisableFrontend() + { + SendOut(new DisableMessage(frontend)); + } + +bool WriterBackend::Init(string arg_path, int arg_num_fields, const Field* const* arg_fields) + { + path = arg_path; + num_fields = arg_num_fields; + fields = arg_fields; + + string name = Fmt("%s/%s", path.c_str(), frontend->Name().c_str()); + + SetName(name); + + if ( ! DoInit(arg_path, arg_num_fields, arg_fields) ) + { + DisableFrontend(); + return false; + } + + return true; + } + +bool WriterBackend::Write(int arg_num_fields, int num_writes, Value*** vals) + { + // Double-check that the arguments match. If we get this from remote, + // something might be mixed up. + if ( num_fields != arg_num_fields ) + { + +#ifdef DEBUG + const char* msg = Fmt("Number of fields don't match in WriterBackend::Write() (%d vs. %d)", + arg_num_fields, num_fields); + Debug(DBG_LOGGING, msg); +#endif + + DeleteVals(num_writes, vals); + DisableFrontend(); + return false; + } + +#ifdef DEBUG + // Double-check all the types match. + for ( int j = 0; j < num_writes; j++ ) + { + for ( int i = 0; i < num_fields; ++i ) + { + if ( vals[j][i]->type != fields[i]->type ) + { + const char* msg = Fmt("Field type doesn't match in WriterBackend::Write() (%d vs. %d)", + vals[j][i]->type, fields[i]->type); + Debug(DBG_LOGGING, msg); + + DisableFrontend(); + DeleteVals(num_writes, vals); + return false; + } + } + } +#endif + + bool success = true; + + for ( int j = 0; j < num_writes; j++ ) + { + success = DoWrite(num_fields, fields, vals[j]); + + if ( ! success ) + break; + } + + DeleteVals(num_writes, vals); + + if ( ! success ) + DisableFrontend(); + + return success; + } + +bool WriterBackend::SetBuf(bool enabled) + { + if ( enabled == buffering ) + // No change. + return true; + + buffering = enabled; + + if ( ! DoSetBuf(enabled) ) + { + DisableFrontend(); + return false; + } + + return true; + } + +bool WriterBackend::Rotate(string rotated_path, double open, + double close, bool terminating) + { + if ( ! DoRotate(rotated_path, open, close, terminating) ) + { + DisableFrontend(); + return false; + } + + return true; + } + +bool WriterBackend::Flush() + { + if ( ! DoFlush() ) + { + DisableFrontend(); + return false; + } + + return true; + } + +bool WriterBackend::Finish() + { + if ( ! DoFlush() ) + { + DisableFrontend(); + return false; + } + + return true; + } + +bool WriterBackend::DoHeartbeat(double network_time, double current_time) + { + MsgThread::DoHeartbeat(network_time, current_time); + + SendOut(new FlushWriteBufferMessage(frontend)); + + return true; + } + +string WriterBackend::Render(const threading::Value::addr_t& addr) const + { + if ( addr.family == IPv4 ) + { + char s[INET_ADDRSTRLEN]; + + if ( inet_ntop(AF_INET, &addr.in.in4, s, INET_ADDRSTRLEN) == NULL ) + return ""; + else + return s; + } + else + { + char s[INET6_ADDRSTRLEN]; + + if ( inet_ntop(AF_INET6, &addr.in.in6, s, INET6_ADDRSTRLEN) == NULL ) + return ""; + else + return s; + } + } + +string WriterBackend::Render(const threading::Value::subnet_t& subnet) const + { + char l[16]; + + if ( subnet.prefix.family == IPv4 ) + modp_uitoa10(subnet.length - 96, l); + else + modp_uitoa10(subnet.length, l); + + string s = Render(subnet.prefix) + "/" + l; + + return s; + } + + diff --git a/src/logging/WriterBackend.h b/src/logging/WriterBackend.h new file mode 100644 index 0000000000..eea0927fd2 --- /dev/null +++ b/src/logging/WriterBackend.h @@ -0,0 +1,319 @@ +// See the file "COPYING" in the main distribution directory for copyright. +// +// Bridge class between main process and writer threads. + +#ifndef LOGGING_WRITERBACKEND_H +#define LOGGING_WRITERBACKEND_H + +#include "Manager.h" + +#include "threading/MsgThread.h" + +namespace logging { + +/** + * Base class for writer implementation. When the logging::Manager creates a + * new logging filter, it instantiates a WriterFrontend. That then in turn + * creates a WriterBackend of the right type. The frontend then forwards + * messages over the backend as its methods are called. + * + * All of this methods must be called only from the corresponding child + * thread (the constructor and destructor are the exceptions.) + */ +class WriterBackend : public threading::MsgThread +{ +public: + /** + * Constructor. + * + * @param frontend The frontend writer that created this backend. The + * *only* purpose of this value is to be passed back via messages as + * a argument to callbacks. One must not otherwise access the + * frontend, it's running in a different thread. + * + * @param name A descriptive name for writer's type (e.g., \c Ascii). + * + */ + WriterBackend(WriterFrontend* frontend); + + /** + * Destructor. + */ + virtual ~WriterBackend(); + + /** + * One-time initialization of the writer to define the logged fields. + * + * @param path A string left to the interpretation of the writer + * implementation; it corresponds to the value configured on the + * script-level for the logging filter. + * + * @param num_fields The number of log fields for the stream. + * + * @param fields An array of size \a num_fields with the log fields. + * The methods takes ownership of the array. + * + * @return False if an error occured. + */ + bool Init(string path, int num_fields, const threading::Field* const* fields); + + /** + * Writes one log entry. + * + * @param num_fields: The number of log fields for this stream. The + * value must match what was passed to Init(). + * + * @param An array of size \a num_fields with the log values. Their + * types musst match with the field passed to Init(). The method + * takes ownership of \a vals.. + * + * Returns false if an error occured, in which case the writer must + * not be used any further. + * + * @return False if an error occured. + */ + bool Write(int num_fields, int num_writes, threading::Value*** vals); + + /** + * Sets the buffering status for the writer, assuming the writer + * supports that. (If not, it will be ignored). + * + * @param enabled False if buffering is to be disabled (by default + * it's on). + * + * @return False if an error occured. + */ + bool SetBuf(bool enabled); + + /** + * Flushes any currently buffered output, assuming the writer + * supports that. (If not, it will be ignored). + * + * @return False if an error occured. + */ + bool Flush(); + + /** + * Triggers rotation, if the writer supports that. (If not, it will + * be ignored). + * + * @return False if an error occured. + */ + bool Rotate(string rotated_path, double open, double close, bool terminating); + + /** + * Finishes writing to this logger in a regularl fashion. Must not be + * called if an error has been indicated earlier. After calling this, + * no further writing must be performed. + * + * @return False if an error occured. + */ + bool Finish(); + + /** + * Disables the frontend that has instantiated this backend. Once + * disabled,the frontend will not send any further message over. + */ + void DisableFrontend(); + + /** + * Returns the log path as passed into the constructor. + */ + const string Path() const { return path; } + + /** + * Returns the number of log fields as passed into the constructor. + */ + int NumFields() const { return num_fields; } + + /** + * Returns the log fields as passed into the constructor. + */ + const threading::Field* const * Fields() const { return fields; } + + /** + * Returns the current buffering state. + * + * @return True if buffering is enabled. + */ + bool IsBuf() { return buffering; } + + /** + * Signals that a file has been rotated. This must be called by a + * writer's implementation of DoRotate() once rotation has finished. + * + * Most of the parameters should be passed through from DoRotate(). + * + * @param new_name The filename of the rotated file. + * + * @param old_name The filename of the original file. + * + * @param open: The timestamp when the original file was opened. + * + * @param close: The timestamp when the origina file was closed. + * + * @param terminating: True if the original rotation request occured + * due to the main Bro process shutting down. + */ + bool FinishedRotation(string new_name, string old_name, + double open, double close, bool terminating); + + /** Helper method to render an IP address as a string. + * + * @param addr The address. + * + * @return An ASCII representation of the address. + */ + string Render(const threading::Value::addr_t& addr) const; + + /** Helper method to render an subnet value as a string. + * + * @param addr The address. + * + * @return An ASCII representation of the address. + */ + string Render(const threading::Value::subnet_t& subnet) const; + +protected: + /** + * Writer-specific intialization method. + * + * A writer implementation must override this method. If it returns + * false, it will be assumed that a fatal error has occured that + * prevents the writer from further operation; it will then be + * disabled and eventually deleted. When returning false, an + * implementation should also call Error() to indicate what happened. + */ + virtual bool DoInit(string path, int num_fields, + const threading::Field* const* fields) = 0; + + /** + * Writer-specific output method implementing recording of fone log + * entry. + * + * A writer implementation must override this method. If it returns + * false, it will be assumed that a fatal error has occured that + * prevents the writer from further operation; it will then be + * disabled and eventually deleted. When returning false, an + * implementation should also call Error() to indicate what happened. + */ + virtual bool DoWrite(int num_fields, const threading::Field* const* fields, + threading::Value** vals) = 0; + + /** + * Writer-specific method implementing a change of fthe buffering + * state. If buffering is disabled, the writer should attempt to + * write out information as quickly as possible even if doing so may + * have a performance impact. If enabled (which is the default), it + * may buffer data as helpful and write it out later in a way + * optimized for performance. The current buffering state can be + * queried via IsBuf(). + * + * A writer implementation must override this method but it can just + * ignore calls if buffering doesn't align with its semantics. + * + * If the method returns false, it will be assumed that a fatal error + * has occured that prevents the writer from further operation; it + * will then be disabled and eventually deleted. When returning + * false, an implementation should also call Error() to indicate what + * happened. + */ + virtual bool DoSetBuf(bool enabled) = 0; + + /** + * Writer-specific method implementing flushing of its output. + * + * A writer implementation must override this method but it can just + * ignore calls if flushing doesn't align with its semantics. + * + * If the method returns false, it will be assumed that a fatal error + * has occured that prevents the writer from further operation; it + * will then be disabled and eventually deleted. When returning + * false, an implementation should also call Error() to indicate what + * happened. + */ + virtual bool DoFlush() = 0; + + /** + * Writer-specific method implementing log rotation. Most directly + * this only applies to writers writing into files, which should then + * close the current file and open a new one. However, a writer may + * also trigger other apppropiate actions if semantics are similar. * + * Once rotation has finished, the implementation must call + * FinishedRotation() to signal the log manager that potential + * postprocessors can now run. + * + * A writer implementation must override this method but it can just + * ignore calls if flushing doesn't align with its semantics. It + * still needs to call FinishedRotation() though. + * + * If the method returns false, it will be assumed that a fatal error + * has occured that prevents the writer from further operation; it + * will then be disabled and eventually deleted. When returning + * false, an implementation should also call Error() to indicate what + * happened. + * + * @param rotate_path Reflects the path to where the rotated output + * is to be moved, with specifics depending on the writer. It should + * generally be interpreted in a way consistent with that of \c path + * as passed into DoInit(). As an example, for file-based output, \c + * rotate_path could be the original filename extended with a + * timestamp indicating the time of the rotation. + * + * @param open The network time when the *current* file was opened. + * + * @param close The network time when the *current* file was closed. + * + * @param terminating Indicates whether the rotation request occurs + * due the main Bro prcoess terminating (and not because we've + * reached a regularly scheduled time for rotation). + */ + virtual bool DoRotate(string rotated_path, double open, double close, + bool terminating) = 0; + + /** + * Writer-specific method implementing log output finalization at + * termination. Not called when any of the other methods has + * previously signaled an error, i.e., executing this method signals + * a regular shutdown of the writer. + * + * A writer implementation must override this method but it can just + * ignore calls if flushing doesn't align with its semantics. + * + * If the method returns false, it will be assumed that a fatal error + * has occured that prevents the writer from further operation; it + * will then be disabled and eventually deleted. When returning + * false, an implementation should also call Error() to indicate what + * happened. + */ + virtual bool DoFinish() = 0; + + /** + * Triggered by regular heartbeat messages from the main thread. + * + * This method can be overridden but once must call + * WriterBackend::DoHeartbeat(). + */ + virtual bool DoHeartbeat(double network_time, double current_time); + +private: + /** + * Deletes the values as passed into Write(). + */ + void DeleteVals(int num_writes, threading::Value*** vals); + + // Frontend that instantiated us. This object must not be access from + // this class, it's running in a different thread! + WriterFrontend* frontend; + + string path; // Log path. + int num_fields; // Number of log fields. + const threading::Field* const* fields; // Log fields. + bool buffering; // True if buffering is enabled. +}; + + +} + +#endif + diff --git a/src/logging/WriterFrontend.cc b/src/logging/WriterFrontend.cc new file mode 100644 index 0000000000..c100e99f90 --- /dev/null +++ b/src/logging/WriterFrontend.cc @@ -0,0 +1,279 @@ + +#include "Net.h" +#include "threading/SerialTypes.h" + +#include "WriterFrontend.h" +#include "WriterBackend.h" + +using threading::Value; +using threading::Field; + +namespace logging { + +// Messages sent from frontend to backend (i.e., "InputMessages"). + +class InitMessage : public threading::InputMessage +{ +public: + InitMessage(WriterBackend* backend, const string path, const int num_fields, const Field* const* fields) + : threading::InputMessage("Init", backend), + path(path), num_fields(num_fields), fields(fields) { } + + virtual bool Process() { return Object()->Init(path, num_fields, fields); } + +private: + const string path; + const int num_fields; + const Field * const* fields; +}; + +class RotateMessage : public threading::InputMessage +{ +public: + RotateMessage(WriterBackend* backend, WriterFrontend* frontend, const string rotated_path, const double open, + const double close, const bool terminating) + : threading::InputMessage("Rotate", backend), + frontend(frontend), + rotated_path(rotated_path), open(open), + close(close), terminating(terminating) { } + + virtual bool Process() { return Object()->Rotate(rotated_path, open, close, terminating); } + +private: + WriterFrontend* frontend; + const string rotated_path; + const double open; + const double close; + const bool terminating; +}; + +class WriteMessage : public threading::InputMessage +{ +public: + WriteMessage(WriterBackend* backend, int num_fields, int num_writes, Value*** vals) + : threading::InputMessage("Write", backend), + num_fields(num_fields), num_writes(num_writes), vals(vals) {} + + virtual bool Process() { return Object()->Write(num_fields, num_writes, vals); } + +private: + int num_fields; + int num_writes; + Value ***vals; +}; + +class SetBufMessage : public threading::InputMessage +{ +public: + SetBufMessage(WriterBackend* backend, const bool enabled) + : threading::InputMessage("SetBuf", backend), + enabled(enabled) { } + + virtual bool Process() { return Object()->SetBuf(enabled); } + +private: + const bool enabled; +}; + +class FlushMessage : public threading::InputMessage +{ +public: + FlushMessage(WriterBackend* backend) + : threading::InputMessage("Flush", backend) {} + + virtual bool Process() { return Object()->Flush(); } +}; + +class FinishMessage : public threading::InputMessage +{ +public: + FinishMessage(WriterBackend* backend) + : threading::InputMessage("Finish", backend) {} + + virtual bool Process() { return Object()->Finish(); } +}; + +} + +// Frontend methods. + +using namespace logging; + +WriterFrontend::WriterFrontend(EnumVal* arg_stream, EnumVal* arg_writer, bool arg_local, bool arg_remote) + { + stream = arg_stream; + writer = arg_writer; + Ref(stream); + Ref(writer); + + disabled = initialized = false; + buf = true; + local = arg_local; + remote = arg_remote; + write_buffer = 0; + write_buffer_pos = 0; + ty_name = ""; + + if ( local ) + { + backend = log_mgr->CreateBackend(this, writer->AsEnum()); + assert(backend); + backend->Start(); + } + + else + backend = 0; + } + +WriterFrontend::~WriterFrontend() + { + Unref(stream); + Unref(writer); + } + +string WriterFrontend::Name() const + { + if ( path.size() ) + return ty_name; + + return ty_name + "/" + path; + } + +void WriterFrontend::Stop() + { + FlushWriteBuffer(); + SetDisable(); + + if ( backend ) + backend->Stop(); + } + +void WriterFrontend::Init(string arg_path, int arg_num_fields, const Field* const * arg_fields) + { + if ( disabled ) + return; + + if ( initialized ) + reporter->InternalError("writer initialize twice"); + + path = arg_path; + num_fields = arg_num_fields; + fields = arg_fields; + + initialized = true; + + if ( backend ) + backend->SendIn(new InitMessage(backend, arg_path, arg_num_fields, arg_fields)); + + if ( remote ) + remote_serializer->SendLogCreateWriter(stream, + writer, + arg_path, + arg_num_fields, + arg_fields); + + } + +void WriterFrontend::Write(int num_fields, Value** vals) + { + if ( disabled ) + return; + + if ( remote ) + remote_serializer->SendLogWrite(stream, + writer, + path, + num_fields, + vals); + + if ( ! backend ) + { + DeleteVals(vals); + return; + } + + if ( ! write_buffer ) + { + // Need new buffer. + write_buffer = new Value**[WRITER_BUFFER_SIZE]; + write_buffer_pos = 0; + } + + write_buffer[write_buffer_pos++] = vals; + + if ( write_buffer_pos >= WRITER_BUFFER_SIZE || ! buf || terminating ) + // Buffer full (or no bufferin desired or termiating). + FlushWriteBuffer(); + + } + +void WriterFrontend::FlushWriteBuffer() + { + if ( ! write_buffer_pos ) + // Nothing to do. + return; + + if ( backend ) + backend->SendIn(new WriteMessage(backend, num_fields, write_buffer_pos, write_buffer)); + + // Clear buffer (no delete, we pass ownership to child thread.) + write_buffer = 0; + write_buffer_pos = 0; + } + +void WriterFrontend::SetBuf(bool enabled) + { + if ( disabled ) + return; + + buf = enabled; + + if ( backend ) + backend->SendIn(new SetBufMessage(backend, enabled)); + + if ( ! buf ) + // Make sure no longer buffer any still queued data. + FlushWriteBuffer(); + } + +void WriterFrontend::Flush() + { + if ( disabled ) + return; + + FlushWriteBuffer(); + + if ( backend ) + backend->SendIn(new FlushMessage(backend)); + } + +void WriterFrontend::Rotate(string rotated_path, double open, double close, bool terminating) + { + if ( disabled ) + return; + + FlushWriteBuffer(); + + if ( backend ) + backend->SendIn(new RotateMessage(backend, this, rotated_path, open, close, terminating)); + } + +void WriterFrontend::Finish() + { + if ( disabled ) + return; + + FlushWriteBuffer(); + + if ( backend ) + backend->SendIn(new FinishMessage(backend)); + } + +void WriterFrontend::DeleteVals(Value** vals) + { + // Note this code is duplicated in Manager::DeleteVals(). + for ( int i = 0; i < num_fields; i++ ) + delete vals[i]; + + delete [] vals; + } diff --git a/src/logging/WriterFrontend.h b/src/logging/WriterFrontend.h new file mode 100644 index 0000000000..b83250a5b8 --- /dev/null +++ b/src/logging/WriterFrontend.h @@ -0,0 +1,222 @@ +// See the file "COPYING" in the main distribution directory for copyright. + +#ifndef LOGGING_WRITERFRONTEND_H +#define LOGGING_WRITERFRONTEND_H + +#include "Manager.h" + +#include "threading/MsgThread.h" + +namespace logging { + +class WriterBackend; + +/** + * Bridge class between the logging::Manager and backend writer threads. The + * Manager instantiates one \a WriterFrontend for each open logging filter. + * Each frontend in turns instantiates a WriterBackend-derived class + * internally that's specific to the particular output format. That backend + * runs in a new thread, and it receives messages from the frontend that + * correspond to method called by the manager. + * + */ +class WriterFrontend { +public: + /** + * Constructor. + * + * stream: The logging stream. + * + * writer: The backend writer type, with the value corresponding to the + * script-level \c Log::Writer enum (e.g., \a WRITER_ASCII). The + * frontend will internally instantiate a WriterBackend of the + * corresponding type. + * + * local: If true, the writer will instantiate a local backend. + * + * remote: If true, the writer will forward all data to remote + * clients. + * + * Frontends must only be instantiated by the main thread. + */ + WriterFrontend(EnumVal* stream, EnumVal* writer, bool local, bool remote); + + /** + * Destructor. + * + * Frontends must only be destroyed by the main thread. + */ + virtual ~WriterFrontend(); + + /** + * Stops all output to this writer. Calling this methods disables all + * message forwarding to the backend and stops the backend thread. + * + * This method must only be called from the main thread. + */ + void Stop(); + + /** + * Initializes the writer. + * + * This method generates a message to the backend writer and triggers + * the corresponding message there. If the backend method fails, it + * sends a message back that will asynchronously call Disable(). + * + * See WriterBackend::Init() for arguments. The method takes + * ownership of \a fields. + * + * This method must only be called from the main thread. + */ + void Init(string path, int num_fields, const threading::Field* const* fields); + + /** + * Write out a record. + * + * This method generates a message to the backend writer and triggers + * the corresponding message there. If the backend method fails, it + * sends a message back that will asynchronously call Disable(). + * + * As an optimization, if buffering is enabled (which is the default) + * this method may buffer several writes and send them over to the + * backend in bulk with a single message. An explicit bulk write of + * all currently buffered data can be triggered with + * FlushWriteBuffer(). The backend writer triggers this with a + * message at every heartbeat. + * + * See WriterBackend::Writer() for arguments (except that this method + * takes only a single record, not an array). The method takes + * ownership of \a vals. + * + * This method must only be called from the main thread. + */ + void Write(int num_fields, threading::Value** vals); + + /** + * Sets the buffering state. + * + * This method generates a message to the backend writer and triggers + * the corresponding message there. If the backend method fails, it + * sends a message back that will asynchronously call Disable(). + * + * See WriterBackend::SetBuf() for arguments. + * + * This method must only be called from the main thread. + */ + void SetBuf(bool enabled); + + /** + * Flushes the output. + * + * This method generates a message to the backend writer and triggers + * the corresponding message there. In addition, it also triggers + * FlushWriteBuffer(). If the backend method fails, it sends a + * message back that will asynchronously call Disable(). + * + * This method must only be called from the main thread. + */ + void Flush(); + + /** + * Triggers log rotation. + * + * This method generates a message to the backend writer and triggers + * the corresponding message there. If the backend method fails, it + * sends a message back that will asynchronously call Disable(). + * + * See WriterBackend::Rotate() for arguments. + * + * This method must only be called from the main thread. + */ + void Rotate(string rotated_path, double open, double close, bool terminating); + + /** + * Finalizes writing to this tream. + * + * This method generates a message to the backend writer and triggers + * the corresponding message there. If the backend method fails, it + * sends a message back that will asynchronously call Disable(). + * + * This method must only be called from the main thread. + */ + void Finish(); + + /** + * Explicitly triggers a transfer of all potentially buffered Write() + * operations over to the backend. + * + * This method must only be called from the main thread. + */ + void FlushWriteBuffer(); + + /** + * Disables the writer frontend. From now on, all method calls that + * would normally send message over to the backend, turn into no-ops. + * Note though that it does not stop the backend itself, use Stop() + * to do thast as well (this method is primarily for use as callback + * when the backend wants to disable the frontend). + * + * Disabled frontend will eventually be discarded by the + * logging::Manager. + * + * This method must only be called from the main thread. + */ + void SetDisable() { disabled = true; } + + /** + * Returns true if the writer frontend has been disabled with SetDisable(). + */ + bool Disabled() { return disabled; } + + /** + * Returns the log path as passed into the constructor. + */ + const string Path() const { return path; } + + /** + * Returns the number of log fields as passed into the constructor. + */ + int NumFields() const { return num_fields; } + + /** + * Returns a descriptive name for the writer, including the type of + * the backend and the path used. + * + * This method is safe to call from any thread. + */ + string Name() const; + + /** + * Returns the log fields as passed into the constructor. + */ + const threading::Field* const * Fields() const { return fields; } + +protected: + friend class Manager; + + void DeleteVals(threading::Value** vals); + + EnumVal* stream; + EnumVal* writer; + + WriterBackend* backend; // The backend we have instanatiated. + bool disabled; // True if disabled. + bool initialized; // True if initialized. + bool buf; // True if buffering is enabled (default). + bool local; // True if logging locally. + bool remote; // True if loggin remotely. + + string ty_name; // Name of the backend type. Set by the manager. + string path; // The log path. + int num_fields; // The number of log fields. + const threading::Field* const* fields; // The log fields. + + // Buffer for bulk writes. + static const int WRITER_BUFFER_SIZE = 1000; + int write_buffer_pos; // Position of next write in buffer. + threading::Value*** write_buffer; // Buffer of size WRITER_BUFFER_SIZE. +}; + +} + +#endif diff --git a/src/LogWriterAscii.cc b/src/logging/writers/Ascii.cc similarity index 88% rename from src/LogWriterAscii.cc rename to src/logging/writers/Ascii.cc index d2c1d91370..0759e60a82 100644 --- a/src/LogWriterAscii.cc +++ b/src/logging/writers/Ascii.cc @@ -3,10 +3,17 @@ #include #include -#include "LogWriterAscii.h" #include "NetVar.h" +#include "threading/SerialTypes.h" -LogWriterAscii::LogWriterAscii() +#include "Ascii.h" + +using namespace logging; +using namespace writer; +using threading::Value; +using threading::Field; + +Ascii::Ascii(WriterFrontend* frontend) : WriterBackend(frontend) { file = 0; @@ -42,7 +49,7 @@ LogWriterAscii::LogWriterAscii() desc.AddEscapeSequence(separator, separator_len); } -LogWriterAscii::~LogWriterAscii() +Ascii::~Ascii() { if ( file ) fclose(file); @@ -54,7 +61,7 @@ LogWriterAscii::~LogWriterAscii() delete [] header_prefix; } -bool LogWriterAscii::WriteHeaderField(const string& key, const string& val) +bool Ascii::WriteHeaderField(const string& key, const string& val) { string str = string(header_prefix, header_prefix_len) + key + string(separator, separator_len) + val + "\n"; @@ -62,8 +69,8 @@ bool LogWriterAscii::WriteHeaderField(const string& key, const string& val) return (fwrite(str.c_str(), str.length(), 1, file) == 1); } -bool LogWriterAscii::DoInit(string path, int num_fields, - const LogField* const * fields) +bool Ascii::DoInit(string path, int num_fields, + const Field* const * fields) { if ( output_to_stdout ) path = "/dev/stdout"; @@ -108,7 +115,7 @@ bool LogWriterAscii::DoInit(string path, int num_fields, types += string(separator, separator_len); } - const LogField* field = fields[i]; + const Field* field = fields[i]; names += field->name; types += type_name(field->type); if ( (field->type == TYPE_TABLE) || (field->type == TYPE_VECTOR) ) @@ -131,17 +138,18 @@ write_error: return false; } -bool LogWriterAscii::DoFlush() +bool Ascii::DoFlush() { fflush(file); return true; } -void LogWriterAscii::DoFinish() +bool Ascii::DoFinish() { + return true; } -bool LogWriterAscii::DoWriteOne(ODesc* desc, LogVal* val, const LogField* field) +bool Ascii::DoWriteOne(ODesc* desc, Value* val, const Field* field) { if ( ! val->present ) { @@ -161,18 +169,19 @@ bool LogWriterAscii::DoWriteOne(ODesc* desc, LogVal* val, const LogField* field) case TYPE_COUNT: case TYPE_COUNTER: - case TYPE_PORT: desc->Add(val->val.uint_val); break; + case TYPE_PORT: + desc->Add(val->val.port_val.port); + break; + case TYPE_SUBNET: - desc->Add(dotted_addr(val->val.subnet_val.net)); - desc->Add("/"); - desc->Add(val->val.subnet_val.width); + desc->Add(Render(val->val.subnet_val)); break; case TYPE_ADDR: - desc->Add(dotted_addr(val->val.addr_val)); + desc->Add(Render(val->val.addr_val)); break; case TYPE_TIME: @@ -184,7 +193,7 @@ bool LogWriterAscii::DoWriteOne(ODesc* desc, LogVal* val, const LogField* field) case TYPE_DOUBLE: desc->Add(val->val.double_val); - break; + break; case TYPE_ENUM: case TYPE_STRING: @@ -281,8 +290,8 @@ bool LogWriterAscii::DoWriteOne(ODesc* desc, LogVal* val, const LogField* field) return true; } -bool LogWriterAscii::DoWrite(int num_fields, const LogField* const * fields, - LogVal** vals) +bool Ascii::DoWrite(int num_fields, const Field* const * fields, + Value** vals) { if ( ! file ) DoInit(Path(), NumFields(), Fields()); @@ -312,8 +321,7 @@ bool LogWriterAscii::DoWrite(int num_fields, const LogField* const * fields, return true; } -bool LogWriterAscii::DoRotate(string rotated_path, double open, - double close, bool terminating) +bool Ascii::DoRotate(string rotated_path, double open, double close, bool terminating) { // Don't rotate special files or if there's not one currently open. if ( ! file || IsSpecial(Path()) ) @@ -334,13 +342,13 @@ bool LogWriterAscii::DoRotate(string rotated_path, double open, return true; } -bool LogWriterAscii::DoSetBuf(bool enabled) +bool Ascii::DoSetBuf(bool enabled) { // Nothing to do. return true; } -string LogWriterAscii::LogExt() +string Ascii::LogExt() { const char* ext = getenv("BRO_LOG_SUFFIX"); if ( ! ext ) ext = "log"; diff --git a/src/LogWriterAscii.h b/src/logging/writers/Ascii.h similarity index 53% rename from src/LogWriterAscii.h rename to src/logging/writers/Ascii.h index 72127c8b1f..6f507aff01 100644 --- a/src/LogWriterAscii.h +++ b/src/logging/writers/Ascii.h @@ -2,33 +2,36 @@ // // Log writer for delimiter-separated ASCII logs. -#ifndef LOGWRITERASCII_H -#define LOGWRITERASCII_H +#ifndef LOGGING_WRITER_ASCII_H +#define LOGGING_WRITER_ASCII_H -#include "LogWriter.h" +#include "../WriterBackend.h" -class LogWriterAscii : public LogWriter { +namespace logging { namespace writer { + +class Ascii : public WriterBackend { public: - LogWriterAscii(); - ~LogWriterAscii(); + Ascii(WriterFrontend* frontend); + ~Ascii(); - static LogWriter* Instantiate() { return new LogWriterAscii; } + static WriterBackend* Instantiate(WriterFrontend* frontend) + { return new Ascii(frontend); } static string LogExt(); protected: virtual bool DoInit(string path, int num_fields, - const LogField* const * fields); - virtual bool DoWrite(int num_fields, const LogField* const * fields, - LogVal** vals); + const threading::Field* const* fields); + virtual bool DoWrite(int num_fields, const threading::Field* const* fields, + threading::Value** vals); virtual bool DoSetBuf(bool enabled); - virtual bool DoRotate(string rotated_path, double open, double close, - bool terminating); + virtual bool DoRotate(string rotated_path, double open, + double close, bool terminating); virtual bool DoFlush(); - virtual void DoFinish(); + virtual bool DoFinish(); private: bool IsSpecial(string path) { return path.find("/dev/") == 0; } - bool DoWriteOne(ODesc* desc, LogVal* val, const LogField* field); + bool DoWriteOne(ODesc* desc, threading::Value* val, const threading::Field* field); bool WriteHeaderField(const string& key, const string& value); FILE* file; @@ -55,4 +58,8 @@ private: int header_prefix_len; }; +} +} + + #endif diff --git a/src/LogWriterNone.cc b/src/logging/writers/None.cc similarity index 54% rename from src/LogWriterNone.cc rename to src/logging/writers/None.cc index 592772afdb..a9a7872f85 100644 --- a/src/LogWriterNone.cc +++ b/src/logging/writers/None.cc @@ -1,8 +1,10 @@ -#include "LogWriterNone.h" +#include "None.h" -bool LogWriterNone::DoRotate(string rotated_path, double open, - double close, bool terminating) +using namespace logging; +using namespace writer; + +bool None::DoRotate(string rotated_path, double open, double close, bool terminating) { if ( ! FinishedRotation(string("/dev/null"), Path(), open, close, terminating)) { diff --git a/src/logging/writers/None.h b/src/logging/writers/None.h new file mode 100644 index 0000000000..cce48953d1 --- /dev/null +++ b/src/logging/writers/None.h @@ -0,0 +1,36 @@ +// See the file "COPYING" in the main distribution directory for copyright. +// +// Dummy log writer that just discards everything (but still pretends to rotate). + +#ifndef LOGGING_WRITER_NONE_H +#define LOGGING_WRITER_NONE_H + +#include "../WriterBackend.h" + +namespace logging { namespace writer { + +class None : public WriterBackend { +public: + None(WriterFrontend* frontend) : WriterBackend(frontend) {} + ~None() {}; + + static WriterBackend* Instantiate(WriterFrontend* frontend) + { return new None(frontend); } + +protected: + virtual bool DoInit(string path, int num_fields, + const threading::Field* const * fields) { return true; } + + virtual bool DoWrite(int num_fields, const threading::Field* const* fields, + threading::Value** vals) { return true; } + virtual bool DoSetBuf(bool enabled) { return true; } + virtual bool DoRotate(string rotated_path, double open, + double close, bool terminating); + virtual bool DoFlush() { return true; } + virtual bool DoFinish() { return true; } +}; + +} +} + +#endif diff --git a/src/main.cc b/src/main.cc index bcc0498123..ff33a3859d 100644 --- a/src/main.cc +++ b/src/main.cc @@ -29,7 +29,6 @@ extern "C" void OPENSSL_add_all_algorithms_conf(void); #include "Event.h" #include "File.h" #include "Reporter.h" -#include "LogMgr.h" #include "Net.h" #include "NetVar.h" #include "Var.h" @@ -44,11 +43,13 @@ extern "C" void OPENSSL_add_all_algorithms_conf(void); #include "PersistenceSerializer.h" #include "EventRegistry.h" #include "Stats.h" -#include "ConnCompressor.h" #include "DPM.h" #include "BroDoc.h" #include "Brofiler.h" -#include "LogWriterAscii.h" + +#include "threading/Manager.h" +#include "logging/Manager.h" +#include "logging/writers/Ascii.h" #include "binpac_bro.h" @@ -64,7 +65,7 @@ extern "C" { #include "setsignal.h" }; -#ifdef USE_PERFTOOLS +#ifdef USE_PERFTOOLS_DEBUG HeapLeakChecker* heap_checker = 0; int perftools_leaks = 0; int perftools_profile = 0; @@ -75,7 +76,8 @@ char* writefile = 0; name_list prefixes; DNS_Mgr* dns_mgr; TimerMgr* timer_mgr; -LogMgr* log_mgr; +logging::Manager* log_mgr = 0; +threading::Manager* thread_mgr = 0; Stmt* stmts; EventHandlerPtr net_done = 0; RuleMatcher* rule_matcher = 0; @@ -95,7 +97,6 @@ int do_notice_analysis = 0; int rule_bench = 0; int generate_documentation = 0; SecondaryPath* secondary_path = 0; -ConnCompressor* conn_compressor = 0; extern char version[]; char* command_line_policy = 0; vector params; @@ -176,7 +177,7 @@ void usage() fprintf(stderr, " -W|--watchdog | activate watchdog timer\n"); fprintf(stderr, " -Z|--doc-scripts | generate documentation for all loaded scripts\n"); -#ifdef USE_PERFTOOLS +#ifdef USE_PERFTOOLS_DEBUG fprintf(stderr, " -m|--mem-leaks | show leaks [perftools]\n"); fprintf(stderr, " -M|--mem-profile | record heap [perftools]\n"); #endif @@ -197,7 +198,7 @@ void usage() fprintf(stderr, " $BRO_PREFIXES | prefix list (%s)\n", bro_prefixes()); fprintf(stderr, " $BRO_DNS_FAKE | disable DNS lookups (%s)\n", bro_dns_fake()); fprintf(stderr, " $BRO_SEED_FILE | file to load seeds from (not set)\n"); - fprintf(stderr, " $BRO_LOG_SUFFIX | ASCII log file extension (.%s)\n", LogWriterAscii::LogExt().c_str()); + fprintf(stderr, " $BRO_LOG_SUFFIX | ASCII log file extension (.%s)\n", logging::writer::Ascii::LogExt().c_str()); fprintf(stderr, " $BRO_PROFILER_FILE | Output file for script execution statistics (not set)\n"); exit(1); @@ -243,7 +244,7 @@ void done_with_network() net_finish(1); -#ifdef USE_PERFTOOLS +#ifdef USE_PERFTOOLS_DEBUG if ( perftools_profile ) { @@ -287,6 +288,9 @@ void terminate_bro() if ( remote_serializer ) remote_serializer->LogStats(); + log_mgr->Terminate(); + thread_mgr->Terminate(); + delete timer_mgr; delete dns_mgr; delete persistence_serializer; @@ -295,10 +299,10 @@ void terminate_bro() delete state_serializer; delete event_registry; delete secondary_path; - delete conn_compressor; delete remote_serializer; delete dpm; delete log_mgr; + delete thread_mgr; delete reporter; } @@ -327,6 +331,13 @@ RETSIGTYPE sig_handler(int signo) { set_processing_status("TERMINATING", "sig_handler"); signal_val = signo; + + if ( thread_mgr->Terminating() && (signal_val == SIGTERM || signal_val == SIGINT) ) + // If the thread manager is already terminating (i.e., + // waiting for child threads to exit), another term signal + // will send the threads a kill. + thread_mgr->KillThreads(); + return RETSIGVAL; } @@ -413,7 +424,7 @@ int main(int argc, char** argv) #ifdef USE_IDMEF {"idmef-dtd", required_argument, 0, 'n'}, #endif -#ifdef USE_PERFTOOLS +#ifdef USE_PERFTOOLS_DEBUG {"mem-leaks", no_argument, 0, 'm'}, {"mem-profile", no_argument, 0, 'M'}, #endif @@ -455,7 +466,7 @@ int main(int argc, char** argv) safe_strncpy(opts, "B:D:e:f:I:i:K:l:n:p:R:r:s:T:t:U:w:x:X:y:Y:z:CFGLOPSWbdghvZ", sizeof(opts)); -#ifdef USE_PERFTOOLS +#ifdef USE_PERFTOOLS_DEBUG strncat(opts, "mM", 2); #endif @@ -611,7 +622,7 @@ int main(int argc, char** argv) exit(0); break; -#ifdef USE_PERFTOOLS +#ifdef USE_PERFTOOLS_DEBUG case 'm': perftools_leaks = 1; break; @@ -661,7 +672,9 @@ int main(int argc, char** argv) set_processing_status("INITIALIZING", "main"); bro_start_time = current_time(true); + reporter = new Reporter(); + thread_mgr = new threading::Manager(); #ifdef DEBUG if ( debug_streams ) @@ -727,7 +740,7 @@ int main(int argc, char** argv) persistence_serializer = new PersistenceSerializer(); remote_serializer = new RemoteSerializer(); event_registry = new EventRegistry(); - log_mgr = new LogMgr(); + log_mgr = new logging::Manager(); if ( events_file ) event_player = new EventPlayer(events_file); @@ -745,14 +758,14 @@ int main(int argc, char** argv) // nevertheless reported; see perftools docs), thus // we suppress some messages here. -#ifdef USE_PERFTOOLS +#ifdef USE_PERFTOOLS_DEBUG { HeapLeakChecker::Disabler disabler; #endif yyparse(); -#ifdef USE_PERFTOOLS +#ifdef USE_PERFTOOLS_DEBUG } #endif @@ -813,8 +826,6 @@ int main(int argc, char** argv) delete [] script_rule_files; - conn_compressor = new ConnCompressor(); - if ( g_policy_debug ) // ### Add support for debug command file. dbg_init_debugger(0); @@ -842,7 +853,7 @@ int main(int argc, char** argv) if ( dns_type != DNS_PRIME ) net_init(interfaces, read_files, netflows, flow_files, - writefile, "tcp or udp or icmp", + writefile, "", secondary_path->Filter(), do_watchdog); BroFile::SetDefaultRotation(log_rotate_interval, log_max_size); @@ -1001,12 +1012,14 @@ int main(int argc, char** argv) have_pending_timers = ! reading_traces && timer_mgr->Size() > 0; + io_sources.Register(thread_mgr, true); + if ( io_sources.Size() > 0 || have_pending_timers ) { if ( profiling_logger ) profiling_logger->Log(); -#ifdef USE_PERFTOOLS +#ifdef USE_PERFTOOLS_DEBUG if ( perftools_leaks ) heap_checker = new HeapLeakChecker("net_run"); diff --git a/src/nb_dns.c b/src/nb_dns.c index 475ba3b8b0..d3b3c5c4de 100644 --- a/src/nb_dns.c +++ b/src/nb_dns.c @@ -186,7 +186,7 @@ _nb_dns_cmpsockaddr(register struct sockaddr *sa1, #endif static const char serr[] = "answer from wrong nameserver (%d)"; - if (sa1->sa_family != sa1->sa_family) { + if (sa1->sa_family != sa2->sa_family) { snprintf(errstr, NB_DNS_ERRSIZE, serr, 1); return (-1); } @@ -381,7 +381,7 @@ nb_dns_addr_request2(register struct nb_dns_info *nd, char *addrp, size -= i; cp += i; } - snprintf(cp, size, "ip6.int"); + snprintf(cp, size, "ip6.arpa"); break; #endif diff --git a/src/net_util.cc b/src/net_util.cc index c0bacc98b2..d91cf02de9 100644 --- a/src/net_util.cc +++ b/src/net_util.cc @@ -2,17 +2,17 @@ #include "config.h" -#ifdef BROv6 #include #include #include #include -#endif #include "Reporter.h" #include "net_util.h" +#include "IPAddr.h" +#include "IP.h" // - adapted from tcpdump // Returns the ones-complement checksum of a chunk of b short-aligned bytes. @@ -32,80 +32,13 @@ int ones_complement_checksum(const void* p, int b, uint32 sum) return sum; } -int tcp_checksum(const struct ip* ip, const struct tcphdr* tp, int len) +int ones_complement_checksum(const IPAddr& a, uint32 sum) { - // ### Note, this is only correct for IPv4. This routine is only - // used by the connection compressor (which we turn off for IPv6 - // traffic). - - int tcp_len = tp->th_off * 4 + len; - uint32 sum; - - if ( len % 2 == 1 ) - // Add in pad byte. - sum = htons(((const u_char*) tp)[tcp_len - 1] << 8); - else - sum = 0; - - sum = ones_complement_checksum((void*) &ip->ip_src.s_addr, 4, sum); - sum = ones_complement_checksum((void*) &ip->ip_dst.s_addr, 4, sum); - - uint32 addl_pseudo = - (htons(IPPROTO_TCP) << 16) | htons((unsigned short) tcp_len); - - sum = ones_complement_checksum((void*) &addl_pseudo, 4, sum); - sum = ones_complement_checksum((void*) tp, tcp_len, sum); - - return sum; + const uint32* bytes; + int len = a.GetBytes(&bytes); + return ones_complement_checksum(bytes, len*4, sum); } -int udp_checksum(const struct ip* ip, const struct udphdr* up, int len) - { - uint32 sum; - - if ( len % 2 == 1 ) - // Add in pad byte. - sum = htons(((const u_char*) up)[len - 1] << 8); - else - sum = 0; - - sum = ones_complement_checksum((void*) &ip->ip_src.s_addr, 4, sum); - sum = ones_complement_checksum((void*) &ip->ip_dst.s_addr, 4, sum); - - uint32 addl_pseudo = - (htons(IPPROTO_UDP) << 16) | htons((unsigned short) len); - - sum = ones_complement_checksum((void*) &addl_pseudo, 4, sum); - sum = ones_complement_checksum((void*) up, len, sum); - - return sum; - } - -#ifdef BROv6 -int udp6_checksum(const struct ip6_hdr* ip6, const struct udphdr* up, int len) - { - uint32 sum; - - if ( len % 2 == 1 ) - // Add in pad byte. - sum = htons(((const u_char*) up)[len - 1] << 8); - else - sum = 0; - - sum = ones_complement_checksum((void*) ip6->ip6_src.s6_addr, 16, sum); - sum = ones_complement_checksum((void*) ip6->ip6_dst.s6_addr, 16, sum); - - uint32 l = htonl(len); - sum = ones_complement_checksum((void*) &l, 4, sum); - - uint32 addl_pseudo = htons(IPPROTO_UDP); - sum = ones_complement_checksum((void*) &addl_pseudo, 4, sum); - sum = ones_complement_checksum((void*) up, len, sum); - - return sum; - } -#endif - int icmp_checksum(const struct icmp* icmpp, int len) { uint32 sum; @@ -121,6 +54,58 @@ int icmp_checksum(const struct icmp* icmpp, int len) return sum; } +#ifdef ENABLE_MOBILE_IPV6 +int mobility_header_checksum(const IP_Hdr* ip) + { + const ip6_mobility* mh = ip->MobilityHeader(); + + if ( ! mh ) return 0; + + uint32 sum = 0; + uint8 mh_len = 8 + 8 * mh->ip6mob_len; + + if ( mh_len % 2 == 1 ) + reporter->Weird(ip->SrcAddr(), ip->DstAddr(), "odd_mobility_hdr_len"); + + sum = ones_complement_checksum(ip->SrcAddr(), sum); + sum = ones_complement_checksum(ip->DstAddr(), sum); + // Note, for IPv6, strictly speaking the protocol and length fields are + // 32 bits rather than 16 bits. But because the upper bits are all zero, + // we get the same checksum either way. + sum += htons(IPPROTO_MOBILITY); + sum += htons(mh_len); + sum = ones_complement_checksum(mh, mh_len, sum); + + return sum; + } +#endif + +int icmp6_checksum(const struct icmp* icmpp, const IP_Hdr* ip, int len) + { + // ICMP6 uses the same checksum function as ICMP4 but a different + // pseudo-header over which it is computed. + uint32 sum; + + if ( len % 2 == 1 ) + // Add in pad byte. + sum = htons(((const u_char*) icmpp)[len - 1] << 8); + else + sum = 0; + + // Pseudo-header as for UDP over IPv6 above. + sum = ones_complement_checksum(ip->SrcAddr(), sum); + sum = ones_complement_checksum(ip->DstAddr(), sum); + uint32 l = htonl(len); + sum = ones_complement_checksum((void*) &l, 4, sum); + + uint32 addl_pseudo = htons(IPPROTO_ICMPV6); + sum = ones_complement_checksum((void*) &addl_pseudo, 4, sum); + + sum = ones_complement_checksum((void*) icmpp, len, sum); + + return sum; + } + #define CLASS_A 0x00000000 #define CLASS_B 0x80000000 @@ -143,223 +128,24 @@ char addr_to_class(uint32 addr) return 'A'; } -uint32 addr_to_net(uint32 addr) +const char* fmt_conn_id(const IPAddr& src_addr, uint32 src_port, + const IPAddr& dst_addr, uint32 dst_port) { - if ( CHECK_CLASS(addr, CLASS_D) ) - ; // class D's are left alone ### - else if ( CHECK_CLASS(addr, CLASS_C) ) - addr = addr & 0xffffff00; - else if ( CHECK_CLASS(addr, CLASS_B) ) - addr = addr & 0xffff0000; - else - addr = addr & 0xff000000; + static char buffer[512]; - return addr; - } + safe_snprintf(buffer, sizeof(buffer), "%s:%d > %s:%d", + string(src_addr).c_str(), src_port, + string(dst_addr).c_str(), dst_port); -const char* dotted_addr(uint32 addr, int alternative) - { - addr = ntohl(addr); - const char* fmt = alternative ? "%d,%d.%d.%d" : "%d.%d.%d.%d"; - - static char buf[32]; - snprintf(buf, sizeof(buf), fmt, - addr >> 24, (addr >> 16) & 0xff, - (addr >> 8) & 0xff, addr & 0xff); - - return buf; - } - -const char* dotted_addr(const uint32* addr, int alternative) - { -#ifdef BROv6 - if ( is_v4_addr(addr) ) - return dotted_addr(addr[3], alternative); - - static char buf[256]; - - if ( inet_ntop(AF_INET6, addr, buf, sizeof buf) == NULL ) - return ""; - - return buf; - -#else - return dotted_addr(to_v4_addr(addr), alternative); -#endif - } - -const char* dotted_net(uint32 addr) - { - addr = ntohl(addr); - - static char buf[32]; - - if ( CHECK_CLASS(addr, CLASS_D) ) - sprintf(buf, "%d.%d.%d.%d", - addr >> 24, (addr >> 16) & 0xff, - (addr >> 8) & 0xff, addr & 0xff); - - else if ( CHECK_CLASS(addr, CLASS_C) ) - sprintf(buf, "%d.%d.%d", - addr >> 24, (addr >> 16) & 0xff, (addr >> 8) & 0xff); - - else - // Same for class A's and B's. - sprintf(buf, "%d.%d", addr >> 24, (addr >> 16) & 0xff); - - return buf; - } - -#ifdef BROv6 -const char* dotted_net6(const uint32* addr) - { - if ( is_v4_addr(addr) ) - return dotted_net(to_v4_addr(addr)); - else - // ### this isn't right, but net's should go away eventually ... - return dotted_addr(addr); - } -#endif - -uint32 dotted_to_addr(const char* addr_text) - { - int addr[4]; - - if ( sscanf(addr_text, - "%d.%d.%d.%d", addr+0, addr+1, addr+2, addr+3) != 4 ) - { - reporter->Error("bad dotted address: %s", addr_text ); - return 0; - } - - if ( addr[0] < 0 || addr[1] < 0 || addr[2] < 0 || addr[3] < 0 || - addr[0] > 255 || addr[1] > 255 || addr[2] > 255 || addr[3] > 255 ) - { - reporter->Error("bad dotted address: %s", addr_text); - return 0; - } - - uint32 a = (addr[0] << 24) | (addr[1] << 16) | (addr[2] << 8) | addr[3]; - - // ### perhaps do gethostbyaddr here? - - return uint32(htonl(a)); - } - -#ifdef BROv6 -uint32* dotted_to_addr6(const char* addr_text) - { - uint32* addr = new uint32[4]; - if ( inet_pton(AF_INET6, addr_text, addr) <= 0 ) - { - reporter->Error("bad IPv6 address: %s", addr_text ); - addr[0] = addr[1] = addr[2] = addr[3] = 0; - } - - return addr; - } - -#endif - -#ifdef BROv6 -int is_v4_addr(const uint32 addr[4]) - { - return addr[0] == 0 && addr[1] == 0 && addr[2] == 0; - } -#endif - -uint32 to_v4_addr(const uint32* addr) - { -#ifdef BROv6 - if ( ! is_v4_addr(addr) ) - reporter->InternalError("conversion of non-IPv4 address to IPv4 address"); - return addr[3]; -#else - return addr[0]; -#endif - } - -uint32 mask_addr(uint32 a, uint32 top_bits_to_keep) - { - if ( top_bits_to_keep > 32 ) - { - reporter->Error("bad address mask value %d", top_bits_to_keep); - return a; - } - - if ( top_bits_to_keep == 0 ) - // The shifts below don't have any effect with 0, i.e., - // 1 << 32 does not yield 0; either due to compiler - // misoptimization or language semantics. - return 0; - - uint32 addr = ntohl(a); - - int shift = 32 - top_bits_to_keep; - addr >>= shift; - addr <<= shift; - - return htonl(addr); - } - -const uint32* mask_addr(const uint32* a, uint32 top_bits_to_keep) - { -#ifdef BROv6 - static uint32 addr[4]; - - addr[0] = a[0]; - addr[1] = a[1]; - addr[2] = a[2]; - addr[3] = a[3]; - - // This is a bit dicey: if it's a v4 address, then we interpret - // the mask as being with respect to 32 bits total, even though - // strictly speaking, the v4 address comprises the least-significant - // bits out of 128, rather than the most significant. However, - // we only do this if the mask itself is consistent for a 32-bit - // address. - uint32 max_bits = (is_v4_addr(a) && top_bits_to_keep <= 32) ? 32 : 128; - - if ( top_bits_to_keep == 0 || top_bits_to_keep > max_bits ) - { - reporter->Error("bad address mask value %s", top_bits_to_keep); - return addr; - } - - int word = 3; // start zeroing out with word #3 - int bits_to_chop = max_bits - top_bits_to_keep; // bits to discard - while ( bits_to_chop >= 32 ) - { // there's an entire word to discard - addr[word] = 0; - --word; // move on to next, more significant word - bits_to_chop -= 32; // we just go rid of 32 bits - } - - // All that's left to work with now is the word pointed to by "word". - uint32 addr32 = ntohl(addr[word]); - addr32 >>= bits_to_chop; - addr32 <<= bits_to_chop; - addr[word] = htonl(addr32); - - return addr; -#else - return a; -#endif + return buffer; } const char* fmt_conn_id(const uint32* src_addr, uint32 src_port, const uint32* dst_addr, uint32 dst_port) { - char addr1[128], addr2[128]; - static char buffer[512]; - - strcpy(addr1, dotted_addr(src_addr)); - strcpy(addr2, dotted_addr(dst_addr)); - - safe_snprintf(buffer, sizeof(buffer), "%s:%d > %s:%d", - addr1, src_port, addr2, dst_port); - - return buffer; + IPAddr src(IPv6, src_addr, IPAddr::Network); + IPAddr dst(IPv6, dst_addr, IPAddr::Network); + return fmt_conn_id(src, src_port, dst, dst_port); } uint32 extract_uint32(const u_char* data) diff --git a/src/net_util.h b/src/net_util.h index 28abf4dbcb..733d946564 100644 --- a/src/net_util.h +++ b/src/net_util.h @@ -5,6 +5,13 @@ #include "config.h" +// Define first. +typedef enum { + TRANSPORT_UNKNOWN, TRANSPORT_TCP, TRANSPORT_UDP, TRANSPORT_ICMP, +} TransportProto; + +typedef enum { IPv4, IPv6 } IPFamily; + #include #include @@ -24,37 +31,82 @@ #ifdef HAVE_NETINET_IP6_H #include -#else -struct ip6_hdr { - uint16 ip6_plen; - uint8 ip6_nxt; - uint8 ip6_hlim; + +#ifndef HAVE_IP6_OPT +struct ip6_opt { + uint8 ip6o_type; + uint8 ip6o_len; }; -#endif +#endif // HAVE_IP6_OPT -#include "util.h" - -#ifdef BROv6 -typedef uint32* addr_type; // a pointer to 4 uint32's -typedef const uint32* const_addr_type; -#define NUM_ADDR_WORDS 4 - -typedef struct { - uint32 net[4]; - uint32 width; -} subnet_type; +#ifndef HAVE_IP6_EXT +struct ip6_ext { + uint8 ip6e_nxt; + uint8 ip6e_len; +}; +#endif // HAVE_IP6_EXT #else -typedef uint32 addr_type; -typedef const uint32 const_addr_type; -#define NUM_ADDR_WORDS 1 -typedef struct { - uint32 net; - uint32 width; -} subnet_type; +struct ip6_hdr { + union { + struct ip6_hdrctl { + uint32 ip6_un1_flow; /* 4 bits version, 8 bits TC, 20 bits + flow-ID */ + uint16 ip6_un1_plen; /* payload length */ + uint8 ip6_un1_nxt; /* next header */ + uint8 ip6_un1_hlim; /* hop limit */ + } ip6_un1; + uint8 ip6_un2_vfc; /* 4 bits version, top 4 bits tclass */ + } ip6_ctlun; + struct in6_addr ip6_src; /* source address */ + struct in6_addr ip6_dst; /* destination address */ +}; -#endif +#define ip6_vfc ip6_ctlun.ip6_un2_vfc +#define ip6_flow ip6_ctlun.ip6_un1.ip6_un1_flow +#define ip6_plen ip6_ctlun.ip6_un1.ip6_un1_plen +#define ip6_nxt ip6_ctlun.ip6_un1.ip6_un1_nxt +#define ip6_hlim ip6_ctlun.ip6_un1.ip6_un1_hlim +#define ip6_hops ip6_ctlun.ip6_un1.ip6_un1_hlim + +struct ip6_opt { + uint8 ip6o_type; + uint8 ip6o_len; +}; + +struct ip6_ext { + uint8 ip6e_nxt; + uint8 ip6e_len; +}; + +struct ip6_frag { + uint8 ip6f_nxt; /* next header */ + uint8 ip6f_reserved; /* reserved field */ + uint16 ip6f_offlg; /* offset, reserved, and flag */ + uint32 ip6f_ident; /* identification */ +}; + +struct ip6_hbh { + uint8 ip6h_nxt; /* next header */ + uint8 ip6h_len; /* length in units of 8 octets */ + /* followed by options */ +}; + +struct ip6_dest { + uint8 ip6d_nxt; /* next header */ + uint8 ip6d_len; /* length in units of 8 octets */ + /* followed by options */ +}; + +struct ip6_rthdr { + uint8 ip6r_nxt; /* next header */ + uint8 ip6r_len; /* length in units of 8 octets */ + uint8 ip6r_type; /* routing type */ + uint8 ip6r_segleft; /* segments left */ + /* followed by routing type specific data */ +}; +#endif // HAVE_NETINET_IP6_H // For Solaris. #if !defined(TCPOPT_WINDOW) && defined(TCPOPT_WSCALE) @@ -81,83 +133,29 @@ inline int seq_delta(uint32 a, uint32 b) return int(a-b); } +class IPAddr; +class IP_Hdr; + // Returns the ones-complement checksum of a chunk of b short-aligned bytes. extern int ones_complement_checksum(const void* p, int b, uint32 sum); -extern int tcp_checksum(const struct ip* ip, const struct tcphdr* tp, int len); -extern int udp_checksum(const struct ip* ip, const struct udphdr* up, int len); -#ifdef BROv6 -extern int udp6_checksum(const struct ip6_hdr* ip, const struct udphdr* up, - int len); -#endif +extern int ones_complement_checksum(const IPAddr& a, uint32 sum); + +extern int icmp6_checksum(const struct icmp* icmpp, const IP_Hdr* ip, int len); extern int icmp_checksum(const struct icmp* icmpp, int len); -// Given an address in host order, returns its "classical network prefix", -// also in host order. -extern uint32 addr_to_net(uint32 addr); +#ifdef ENABLE_MOBILE_IPV6 +extern int mobility_header_checksum(const IP_Hdr* ip); +#endif + // Returns 'A', 'B', 'C' or 'D' extern char addr_to_class(uint32 addr); -// Returns a pointer to static storage giving the ASCII dotted representation -// of the given address, which should be passed in network order. -extern const char* dotted_addr(uint32 addr, int alternative=0); -extern const char* dotted_addr(const uint32* addr, int alternative=0); - -// Same, but for the network prefix. -extern const char* dotted_net(uint32 addr); -extern const char* dotted_net6(const uint32* addr); - -// Given an ASCII dotted representation, returns the corresponding address -// in network order. -extern uint32 dotted_to_addr(const char* addr_text); -extern uint32* dotted_to_addr6(const char* addr_text); - -extern int is_v4_addr(const uint32 addr[4]); -extern uint32 to_v4_addr(const uint32* addr); - -extern uint32 mask_addr(uint32 a, uint32 top_bits_to_keep); -extern const uint32* mask_addr(const uint32* a, uint32 top_bits_to_keep); - +extern const char* fmt_conn_id(const IPAddr& src_addr, uint32 src_port, + const IPAddr& dst_addr, uint32 dst_port); extern const char* fmt_conn_id(const uint32* src_addr, uint32 src_port, const uint32* dst_addr, uint32 dst_port); -inline void copy_addr(const uint32* src_a, uint32* dst_a) - { -#ifdef BROv6 - dst_a[0] = src_a[0]; - dst_a[1] = src_a[1]; - dst_a[2] = src_a[2]; - dst_a[3] = src_a[3]; -#else - dst_a[0] = src_a[0]; -#endif - } - -inline int addr_eq(const uint32* a1, const uint32* a2) - { -#ifdef BROv6 - return a1[0] == a2[0] && - a1[1] == a2[1] && - a1[2] == a2[2] && - a1[3] == a2[3]; -#else - return a1[0] == a2[0]; -#endif - } - -inline int subnet_eq(const subnet_type* s1, const subnet_type* s2) - { -#ifdef BROv6 - return s1->net[0] == s2->net[0] && - s1->net[1] == s2->net[1] && - s1->net[2] == s2->net[2] && - s1->net[3] == s2->net[3] && - s1->width == s2->width; -#else - return s1->net == s2->net && s1->width == s2->width; -#endif - } - // Read 4 bytes from data and return in network order. extern uint32 extract_uint32(const u_char* data); @@ -170,6 +168,8 @@ extern uint32 extract_uint32(const u_char* data); inline double ntohd(double d) { return d; } inline double htond(double d) { return d; } +inline uint64 ntohll(uint64 i) { return i; } +inline uint64 htonll(uint64 i) { return i; } #else @@ -195,6 +195,24 @@ inline double ntohd(double d) inline double htond(double d) { return ntohd(d); } +inline uint64 ntohll(uint64 i) + { + u_char c; + union { + uint64 i; + u_char c[8]; + } x; + + x.i = i; + c = x.c[0]; x.c[0] = x.c[7]; x.c[7] = c; + c = x.c[1]; x.c[1] = x.c[6]; x.c[6] = c; + c = x.c[2]; x.c[2] = x.c[5]; x.c[5] = c; + c = x.c[3]; x.c[3] = x.c[4]; x.c[4] = c; + return x.i; + } + +inline uint64 htonll(uint64 i) { return ntohll(i); } + #endif #endif diff --git a/src/parse.y b/src/parse.y index 1b05171ecf..f78003f08b 100644 --- a/src/parse.y +++ b/src/parse.y @@ -10,7 +10,7 @@ %token TOK_CONSTANT TOK_COPY TOK_COUNT TOK_COUNTER TOK_DEFAULT TOK_DELETE %token TOK_DOUBLE TOK_ELSE TOK_ENUM TOK_EVENT TOK_EXPORT TOK_FILE TOK_FOR %token TOK_FUNCTION TOK_GLOBAL TOK_ID TOK_IF TOK_INT -%token TOK_INTERVAL TOK_LIST TOK_LOCAL TOK_MODULE TOK_MATCH +%token TOK_INTERVAL TOK_LIST TOK_LOCAL TOK_MODULE %token TOK_NEXT TOK_OF TOK_PATTERN TOK_PATTERN_TEXT %token TOK_PORT TOK_PRINT TOK_RECORD TOK_REDEF %token TOK_REMOVE_FROM TOK_RETURN TOK_SCHEDULE TOK_SET @@ -33,7 +33,7 @@ %left ',' '|' %right '=' TOK_ADD_TO TOK_REMOVE_FROM -%right '?' ':' TOK_USING +%right '?' ':' %left TOK_OR %left TOK_AND %nonassoc '<' '>' TOK_LE TOK_GE TOK_EQ TOK_NE @@ -504,12 +504,6 @@ expr: $$ = new VectorConstructorExpr($3); } - | TOK_MATCH expr TOK_USING expr - { - set_location(@1, @4); - $$ = new RecordMatchExpr($2, $4); - } - | expr '(' opt_expr_list ')' { set_location(@1, @4); diff --git a/src/patricia.c b/src/patricia.c index ef9c008f4b..1dbc795ab7 100644 --- a/src/patricia.c +++ b/src/patricia.c @@ -115,16 +115,12 @@ local_inet_pton (int af, const char *src, void *dst) } } #ifdef NT -#ifdef HAVE_IPV6 else if (af == AF_INET6) { struct in6_addr Address; return (inet6_addr(src, &Address)); } -#endif /* HAVE_IPV6 */ -#endif /* NT */ -#ifndef NT +#else else { - errno = EAFNOSUPPORT; return -1; } @@ -160,10 +156,8 @@ my_inet_pton (int af, const char *src, void *dst) } memcpy (dst, xp, 4); return (1); -#ifdef HAVE_IPV6 } else if (af == AF_INET6) { return (local_inet_pton (af, src, dst)); -#endif /* HAVE_IPV6 */ } else { #ifndef NT errno = EAFNOSUPPORT; @@ -217,7 +211,6 @@ prefix_toa2x (prefix_t *prefix, char *buff, int with_len) } return (buff); } -#ifdef HAVE_IPV6 else if (prefix->family == AF_INET6) { char *r; r = (char *) inet_ntop (AF_INET6, &prefix->add.sin6, buff, 48 /* a guess value */ ); @@ -227,7 +220,6 @@ prefix_toa2x (prefix_t *prefix, char *buff, int with_len) } return (buff); } -#endif /* HAVE_IPV6 */ else return (NULL); } @@ -255,17 +247,15 @@ New_Prefix2 (int family, void *dest, int bitlen, prefix_t *prefix) int dynamic_allocated = 0; int default_bitlen = 32; -#ifdef HAVE_IPV6 if (family == AF_INET6) { default_bitlen = 128; if (prefix == NULL) { - prefix = calloc(1, sizeof (prefix6_t)); + prefix = calloc(1, sizeof (prefix_t)); dynamic_allocated++; } memcpy (&prefix->add.sin6, dest, 16); } else -#endif /* HAVE_IPV6 */ if (family == AF_INET) { if (prefix == NULL) { #ifndef NT @@ -308,9 +298,7 @@ ascii2prefix (int family, char *string) u_long bitlen, maxbitlen = 0; char *cp; struct in_addr sin; -#ifdef HAVE_IPV6 struct in6_addr sin6; -#endif /* HAVE_IPV6 */ int result; char save[MAXLINE]; @@ -320,19 +308,15 @@ ascii2prefix (int family, char *string) /* easy way to handle both families */ if (family == 0) { family = AF_INET; -#ifdef HAVE_IPV6 if (strchr (string, ':')) family = AF_INET6; -#endif /* HAVE_IPV6 */ } if (family == AF_INET) { maxbitlen = 32; } -#ifdef HAVE_IPV6 else if (family == AF_INET6) { maxbitlen = 128; } -#endif /* HAVE_IPV6 */ if ((cp = strchr (string, '/')) != NULL) { bitlen = atol (cp + 1); @@ -355,7 +339,6 @@ ascii2prefix (int family, char *string) return (New_Prefix (AF_INET, &sin, bitlen)); } -#ifdef HAVE_IPV6 else if (family == AF_INET6) { // Get rid of this with next IPv6 upgrade #if defined(NT) && !defined(HAVE_INET_NTOP) @@ -367,7 +350,6 @@ ascii2prefix (int family, char *string) #endif /* NT */ return (New_Prefix (AF_INET6, &sin6, bitlen)); } -#endif /* HAVE_IPV6 */ else return (NULL); } diff --git a/src/patricia.h b/src/patricia.h index 4bc2f9b81f..dc67226362 100644 --- a/src/patricia.h +++ b/src/patricia.h @@ -52,13 +52,6 @@ #include - -#ifdef BROv6 -#ifndef HAVE_IPV6 -#define HAVE_IPV6 -#endif -#endif - /* typedef unsigned int u_int; */ typedef void (*void_fn_t)(); /* { from defs.h */ @@ -88,9 +81,7 @@ typedef struct _prefix_t { int ref_count; /* reference count */ union { struct in_addr sin; -#ifdef HAVE_IPV6 struct in6_addr sin6; -#endif /* IPV6 */ } add; } prefix_t; diff --git a/src/rule-scan.l b/src/rule-scan.l index 1ba9bed1de..781c477ff2 100644 --- a/src/rule-scan.l +++ b/src/rule-scan.l @@ -18,7 +18,7 @@ WS [ \t]+ D [0-9]+ H [0-9a-fA-F]+ STRING \"([^\n\"]|\\\")*\" -ID [0-9a-zA-Z_-]+ +ID ([0-9a-zA-Z_-]+::)*[0-9a-zA-Z_-]+ RE \/(\\\/)?([^/]|[^\\]\\\/)*\/ META \.[^ \t]+{WS}[^\n]+ PID ([0-9a-zA-Z_-]|"::")+ diff --git a/src/scan.l b/src/scan.l index 4914783c44..ca7a01574c 100644 --- a/src/scan.l +++ b/src/scan.l @@ -228,6 +228,24 @@ ESCSEQ (\\([^\n]|[0-7]+|x[[:xdigit:]]+)) ++yylloc.last_line; } + /* IPv6 literal constant patterns */ +"["({HEX}:){7}{HEX}"]" { + string s(yytext+1); + RET_CONST(new AddrVal(s.erase(s.size()-1))) +} +"["0x{HEX}({HEX}|:)*"::"({HEX}|:)*"]" { + string s(yytext+3); + RET_CONST(new AddrVal(s.erase(s.size()-1))) +} +"["({HEX}|:)*"::"({HEX}|:)*"]" { + string s(yytext+1); + RET_CONST(new AddrVal(s.erase(s.size()-1))) +} +"["({HEX}|:)*"::"({HEX}|:)*({D}"."){3}{D}"]" { + string s(yytext+1); + RET_CONST(new AddrVal(s.erase(s.size()-1))) +} + [!%*/+\-,:;<=>?()\[\]{}~$|] return yytext[0]; "--" return TOK_DECR; @@ -273,7 +291,6 @@ int return TOK_INT; interval return TOK_INTERVAL; list return TOK_LIST; local return TOK_LOCAL; -match return TOK_MATCH; module return TOK_MODULE; next return TOK_NEXT; of return TOK_OF; @@ -295,7 +312,6 @@ timeout return TOK_TIMEOUT; timer return TOK_TIMER; type return TOK_TYPE; union return TOK_UNION; -using return TOK_USING; vector return TOK_VECTOR; when return TOK_WHEN; @@ -450,11 +466,6 @@ F RET_CONST(new Val(false, TYPE_BOOL)) ({D}"."){3}{D} RET_CONST(new AddrVal(yytext)) -({HEX}:){7}{HEX} RET_CONST(new AddrVal(yytext)) - -0x{HEX}({HEX}|:)*"::"({HEX}|:)* RET_CONST(new AddrVal(yytext+2)) -(({D}|:)({HEX}|:)*)?"::"({HEX}|:)* RET_CONST(new AddrVal(yytext)) - "0x"{HEX}+ RET_CONST(new Val(static_cast(strtol(yytext, 0, 16)), TYPE_COUNT)) {H}("."{H})+ RET_CONST(dns_mgr->LookupHost(yytext)) diff --git a/src/threading/BasicThread.cc b/src/threading/BasicThread.cc new file mode 100644 index 0000000000..e590b13434 --- /dev/null +++ b/src/threading/BasicThread.cc @@ -0,0 +1,174 @@ + +#include +#include + +#include "config.h" +#include "BasicThread.h" +#include "Manager.h" + +#ifdef HAVE_LINUX +#include +#endif + +using namespace threading; + +uint64_t BasicThread::thread_counter = 0; + +BasicThread::BasicThread() + { + started = false; + terminating = false; + pthread = 0; + + buf_len = 2048; + buf = (char*) malloc(buf_len); + + name = Fmt("thread-%d", ++thread_counter); + + thread_mgr->AddThread(this); + } + +BasicThread::~BasicThread() + { + if ( buf ) + free(buf); + } + +void BasicThread::SetName(const string& arg_name) + { + // Slight race condition here with reader threads, but shouldn't matter. + name = arg_name; + } + +void BasicThread::SetOSName(const string& name) + { +#ifdef HAVE_LINUX + prctl(PR_SET_NAME, name.c_str(), 0, 0, 0); +#endif + +#ifdef __APPLE__ + pthread_setname_np(name.c_str()); +#endif + +#ifdef FREEBSD + pthread_set_name_np(pthread_self(), name, name.c_str()); +#endif + } + +const char* BasicThread::Fmt(const char* format, ...) + { + va_list al; + va_start(al, format); + int n = safe_vsnprintf(buf, buf_len, format, al); + va_end(al); + + if ( (unsigned int) n >= buf_len ) + { // Not enough room, grow the buffer. + int tmp_len = n + 32; + char* tmp = (char*) malloc(tmp_len); + + // Is it portable to restart? + va_start(al, format); + n = safe_vsnprintf(tmp, tmp_len, format, al); + va_end(al); + + free(tmp); + } + + return buf; + } + +void BasicThread::Start() + { + if ( started ) + return; + + if ( pthread_mutex_init(&terminate, 0) != 0 ) + reporter->FatalError("Cannot create terminate mutex for thread %s", name.c_str()); + + // We use this like a binary semaphore and acquire it immediately. + if ( pthread_mutex_lock(&terminate) != 0 ) + reporter->FatalError("Cannot aquire terminate mutex for thread %s", name.c_str()); + + if ( pthread_create(&pthread, 0, BasicThread::launcher, this) != 0 ) + reporter->FatalError("Cannot create thread %s", name.c_str()); + + DBG_LOG(DBG_THREADING, "Started thread %s", name.c_str()); + + started = true; + + OnStart(); + } + +void BasicThread::Stop() + { + if ( ! started ) + return; + + if ( terminating ) + return; + + DBG_LOG(DBG_THREADING, "Signaling thread %s to terminate ...", name.c_str()); + + // Signal that it's ok for the thread to exit now by unlocking the + // mutex. + if ( pthread_mutex_unlock(&terminate) != 0 ) + reporter->FatalError("Failure flagging terminate condition for thread %s", name.c_str()); + + terminating = true; + + OnStop(); + } + +void BasicThread::Join() + { + if ( ! started ) + return; + + if ( ! terminating ) + Stop(); + + DBG_LOG(DBG_THREADING, "Joining thread %s ...", name.c_str()); + + if ( pthread_join(pthread, 0) != 0 ) + reporter->FatalError("Failure joining thread %s", name.c_str()); + + pthread_mutex_destroy(&terminate); + + DBG_LOG(DBG_THREADING, "Done with thread %s", name.c_str()); + + pthread = 0; + } + +void BasicThread::Kill() + { + if ( ! (started && pthread) ) + return; + + // I believe this is safe to call from a signal handler ... Not error + // checking so that killing doesn't bail out if we have already + // terminated. + pthread_kill(pthread, SIGKILL); + } + +void* BasicThread::launcher(void *arg) + { + BasicThread* thread = (BasicThread *)arg; + + // Block signals in thread. We handle signals only in the main + // process. + sigset_t mask_set; + sigfillset(&mask_set); + int res = pthread_sigmask(SIG_BLOCK, &mask_set, 0); + assert(res == 0); // + + // Run thread's main function. + thread->Run(); + + // Wait until somebody actually wants us to terminate. + if ( pthread_mutex_lock(&thread->terminate) != 0 ) + reporter->FatalError("Failure acquiring terminate mutex at end of thread %s", thread->Name().c_str()); + + return 0; + } + diff --git a/src/threading/BasicThread.h b/src/threading/BasicThread.h new file mode 100644 index 0000000000..cc87ae03bc --- /dev/null +++ b/src/threading/BasicThread.h @@ -0,0 +1,170 @@ + +#ifndef THREADING_BASICTHREAD_H +#define THREADING_BASICTHREAD_H + +#include +#include + +#include "Queue.h" +#include "util.h" + +using namespace std; + +namespace threading { + +class Manager; + +/** + * Base class for all threads. + * + * This class encapsulates all the OS-level thread handling. All thread + * instances are automatically added to the threading::Manager for management. The + * manager also takes care of deleting them (which must not be done + * manually). + */ +class BasicThread +{ +public: + /** + * Creates a new thread object. Instantiating the object does however + * not yet start the actual OS thread, that requires calling Start(). + * + * Only Bro's main thread may create new thread instances. + * + * @param name A descriptive name for thread the thread. This may + * show up in messages to the user. + */ + BasicThread(); + + /** + * Returns a descriptive name for the thread. If not set via + * SetName(). If not set, a default name is choosen automatically. + * + * This method is safe to call from any thread. + */ + const string& Name() const { return name; } + + /** + * Sets a descriptive name for the thread. This should be a string + * that's useful in output presented to the user and uniquely + * identifies the thread. + * + * This method must be called only from the thread itself. + */ + void SetName(const string& name); + + /** + * Set the name shown by the OS as the thread's description. Not + * supported on all OSs. + */ + void SetOSName(const string& name); + + /** + * Starts the thread. Calling this methods will spawn a new OS thread + * executing Run(). Note that one can't restart a thread after a + * Stop(), doing so will be ignored. + * + * Only Bro's main thread must call this method. + */ + void Start(); + + /** + * Signals the thread to stop. The method lets Terminating() now + * return true. It does however not force the thread to terminate. + * It's up to the Run() method to to query Terminating() and exit + * eventually. + * + * Calling this method has no effect if Start() hasn't been executed + * yet. + * + * Only Bro's main thread must call this method. + */ + void Stop(); + + /** + * Returns true if Stop() has been called. + * + * This method is safe to call from any thread. + */ + bool Terminating() const { return terminating; } + + /** + * A version of fmt() that the thread can safely use. + * + * This is safe to call from Run() but must not be used from any + * other thread than the current one. + */ + const char* Fmt(const char* format, ...); + +protected: + friend class Manager; + + /** + * Entry point for the thread. This must be overridden by derived + * classes and will execute in a separate thread once Start() is + * called. The thread will not terminate before this method finishes. + * An implementation should regularly check Terminating() to see if + * exiting has been requested. + */ + virtual void Run() = 0; + + /** + * Executed with Start(). This is a hook into starting the thread. It + * will be called from Bro's main thread after the OS thread has been + * started. + */ + virtual void OnStart() {} + + /** + * Executed with Stop(). This is a hook into stopping the thread. It + * will be called from Bro's main thread after the thread has been + * signaled to stop. + */ + virtual void OnStop() {} + + /** + * Destructor. This will be called by the manager. + * + * Only Bro's main thread may delete thread instances. + * + */ + virtual ~BasicThread(); + + /** + * Waits until the thread's Run() method has finished and then joins + * it. This is called from the threading::Manager. + */ + void Join(); + + /** + * Kills the thread immediately. One still needs to call Join() + * afterwards. + * + * This is called from the threading::Manager and safe to execute + * during a signal handler. + */ + void Kill(); + +private: + // pthread entry function. + static void* launcher(void *arg); + + string name; + pthread_t pthread; + bool started; // Set to to true once running. + bool terminating; // Set to to true to signal termination. + + // Used as a semaphore to tell the pthread thread when it may + // terminate. + pthread_mutex_t terminate; + + // For implementing Fmt(). + char* buf; + unsigned int buf_len; + + static uint64_t thread_counter; +}; + +} + +#endif diff --git a/src/threading/Manager.cc b/src/threading/Manager.cc new file mode 100644 index 0000000000..abdbf849b5 --- /dev/null +++ b/src/threading/Manager.cc @@ -0,0 +1,147 @@ + +#include "Manager.h" + +using namespace threading; + +Manager::Manager() + { + DBG_LOG(DBG_THREADING, "Creating thread manager ..."); + + did_process = true; + next_beat = 0; + terminating = false; + idle = true; + } + +Manager::~Manager() + { + if ( all_threads.size() ) + Terminate(); + } + +void Manager::Terminate() + { + DBG_LOG(DBG_THREADING, "Terminating thread manager ..."); + + terminating = true; + + // First process remaining thread output for the message threads. + do Process(); while ( did_process ); + + // Signal all to stop. + for ( all_thread_list::iterator i = all_threads.begin(); i != all_threads.end(); i++ ) + (*i)->Stop(); + + // Then join them all. + for ( all_thread_list::iterator i = all_threads.begin(); i != all_threads.end(); i++ ) + { + (*i)->Join(); + delete *i; + } + + all_threads.clear(); + msg_threads.clear(); + + idle = true; + closed = true; + terminating = false; + } + +void Manager::KillThreads() + { + DBG_LOG(DBG_THREADING, "Killing threads ..."); + + for ( all_thread_list::iterator i = all_threads.begin(); i != all_threads.end(); i++ ) + (*i)->Kill(); + } + +void Manager::AddThread(BasicThread* thread) + { + DBG_LOG(DBG_THREADING, "Adding thread %s ...", thread->Name().c_str()); + all_threads.push_back(thread); + idle = false; + } + +void Manager::AddMsgThread(MsgThread* thread) + { + DBG_LOG(DBG_THREADING, "%s is a MsgThread ...", thread->Name().c_str()); + msg_threads.push_back(thread); + } + +void Manager::GetFds(int* read, int* write, int* except) + { + } + +double Manager::NextTimestamp(double* network_time) + { +// fprintf(stderr, "N %.6f %.6f did_process=%d next_next=%.6f\n", ::network_time, timer_mgr->Time(), (int)did_process, next_beat); + + if ( ::network_time && (did_process || ::network_time > next_beat || ! next_beat) ) + // If we had something to process last time (or out heartbeat + // is due or not set yet), we want to check for more asap. + return timer_mgr->Time(); + + return -1.0; + } + +void Manager::Process() + { + bool do_beat = false; + + if ( network_time && (network_time > next_beat || ! next_beat) ) + { + do_beat = true; + next_beat = ::network_time + HEART_BEAT_INTERVAL; + } + + did_process = false; + + for ( msg_thread_list::iterator i = msg_threads.begin(); i != msg_threads.end(); i++ ) + { + MsgThread* t = *i; + + if ( do_beat ) + t->Heartbeat(); + + while ( t->HasOut() ) + { + Message* msg = t->RetrieveOut(); + + if ( msg->Process() ) + { + if ( network_time ) + did_process = true; + } + + else + { + string s = msg->Name() + " failed, terminating thread"; + reporter->Error("%s", s.c_str()); + t->Stop(); + } + + delete msg; + } + } + +// fprintf(stderr, "P %.6f %.6f do_beat=%d did_process=%d next_next=%.6f\n", network_time, timer_mgr->Time(), do_beat, (int)did_process, next_beat); + } + +const threading::Manager::msg_stats_list& threading::Manager::GetMsgThreadStats() + { + stats.clear(); + + for ( msg_thread_list::iterator i = msg_threads.begin(); i != msg_threads.end(); i++ ) + { + MsgThread* t = *i; + + MsgThread::Stats s; + t->GetStats(&s); + + stats.push_back(std::make_pair(t->Name(),s)); + } + + return stats; + } + + diff --git a/src/threading/Manager.h b/src/threading/Manager.h new file mode 100644 index 0000000000..7d9ba766d4 --- /dev/null +++ b/src/threading/Manager.h @@ -0,0 +1,146 @@ + +#ifndef THREADING_MANAGER_H +#define THREADING_MANAGER_H + +#include + +#include "IOSource.h" + +#include "BasicThread.h" +#include "MsgThread.h" + +namespace threading { + +/** + * The thread manager coordinates all child threads. Once a BasicThread is + * instantitated, it gets addedd to the manager, which will delete it later + * once it has terminated. + * + * In addition to basic threads, the manager also provides additional + * functionality specific to MsgThread instances. In particular, it polls + * their outgoing message queue on a regular basis and feeds data sent into + * the rest of Bro. It also triggers the regular heartbeats. + */ +class Manager : public IOSource +{ +public: + /** + * Constructor. Only a single instance of the manager must be + * created. + */ + Manager(); + + /** + * Destructir. + */ + ~Manager(); + + /** + * Terminates the manager's processor. The method signals all threads + * to terminates and wait for them to do so. It then joins them and + * returns to the caller. Afterwards, no more thread instances may be + * created. + */ + void Terminate(); + + /** + * Returns True if we are currently in Terminate() waiting for + * threads to exit. + */ + bool Terminating() const { return terminating; } + + /** + * Immediately kills all child threads. It does however not yet join + * them, one still needs to call Terminate() for that. + * + * This method is safe to call from a signal handler, and can in fact + * be called while Terminate() is already in progress. + */ + void KillThreads(); + + typedef std::list > msg_stats_list; + + /** + * Returns statistics from all current MsgThread instances. + * + * @return A list of statistics, with one entry for each MsgThread. + * Each entry is a tuple of thread name and statistics. The list + * reference remains valid until the next call to this method (or + * termination of the manager). + */ + const msg_stats_list& GetMsgThreadStats(); + + /** + * Returns the number of currently active threads. This counts all + * threads that are not yet joined, includingt any potentially in + * Terminating() state. + */ + int NumThreads() const { return all_threads.size(); } + +protected: + friend class BasicThread; + friend class MsgThread; + + /** + * Registers a new basic thread with the manager. This is + * automatically called by the thread's constructor. + * + * @param thread The thread. + */ + void AddThread(BasicThread* thread); + + /** + * Registers a new message thread with the manager. This is + * automatically called by the thread's constructor. This must be + * called \a in \a addition to AddThread(BasicThread* thread). The + * MsgThread constructor makes sure to do so. + * + * @param thread The thread. + */ + void AddMsgThread(MsgThread* thread); + + /** + * Part of the IOSource interface. + */ + virtual void GetFds(int* read, int* write, int* except); + + /** + * Part of the IOSource interface. + */ + virtual double NextTimestamp(double* network_time); + + /** + * Part of the IOSource interface. + */ + virtual void Process(); + + /** + * Part of the IOSource interface. + */ + virtual const char* Tag() { return "threading::Manager"; } + +private: + static const int HEART_BEAT_INTERVAL = 1; + + typedef std::list all_thread_list; + all_thread_list all_threads; + + typedef std::list msg_thread_list; + msg_thread_list msg_threads; + + bool did_process; // True if the last Process() found some work to do. + double next_beat; // Timestamp when the next heartbeat will be sent. + bool terminating; // True if we are in Terminate(). + + msg_stats_list stats; +}; + +} + +/** + * A singleton instance of the thread manager. All methods must only be + * called from Bro's main thread. + */ +extern threading::Manager* thread_mgr; + +#endif diff --git a/src/threading/MsgThread.cc b/src/threading/MsgThread.cc new file mode 100644 index 0000000000..ddcd3df1dd --- /dev/null +++ b/src/threading/MsgThread.cc @@ -0,0 +1,289 @@ + +#include "DebugLogger.h" + +#include "MsgThread.h" +#include "Manager.h" + +using namespace threading; + +namespace threading { + +////// Messages. + +// Signals child thread to terminate. This is actually a no-op; its only +// purpose is unblock the current read operation so that the child's Run() +// methods can check the termination status. +class TerminateMessage : public InputMessage +{ +public: + TerminateMessage(MsgThread* thread) : InputMessage("Terminate", thread) { } + + virtual bool Process() { return true; } +}; + +/// Sends a heartbeat to the child thread. +class HeartbeatMessage : public InputMessage +{ +public: + HeartbeatMessage(MsgThread* thread, double arg_network_time, double arg_current_time) + : InputMessage("Heartbeat", thread) + { network_time = arg_network_time; current_time = arg_current_time; } + + virtual bool Process() { return Object()->DoHeartbeat(network_time, current_time); } + +private: + double network_time; + double current_time; +}; + +// A message from the child to be passed on to the Reporter. +class ReporterMessage : public OutputMessage +{ +public: + enum Type { + INFO, WARNING, ERROR, FATAL_ERROR, FATAL_ERROR_WITH_CORE, + INTERNAL_WARNING, INTERNAL_ERROR + }; + + ReporterMessage(Type arg_type, MsgThread* thread, const string& arg_msg) + : OutputMessage("ReporterMessage", thread) + { type = arg_type; msg = arg_msg; } + + virtual bool Process(); + +private: + string msg; + Type type; +}; + +#ifdef DEBUG +// A debug message from the child to be passed on to the DebugLogger. +class DebugMessage : public OutputMessage +{ +public: + DebugMessage(DebugStream arg_stream, MsgThread* thread, const string& arg_msg) + : OutputMessage("DebugMessage", thread) + { stream = arg_stream; msg = arg_msg; } + + virtual bool Process() + { + string s = Object()->Name() + ": " + msg; + debug_logger.Log(stream, "%s", s.c_str()); + return true; + } +private: + string msg; + DebugStream stream; +}; +#endif + +} + +////// Methods. + +Message::~Message() + { + } + +bool ReporterMessage::Process() + { + string s = Object()->Name() + ": " + msg; + const char* cmsg = s.c_str(); + + switch ( type ) { + + case INFO: + reporter->Info("%s", cmsg); + break; + + case WARNING: + reporter->Warning("%s", cmsg); + break; + + case ERROR: + reporter->Error("%s", cmsg); + break; + + case FATAL_ERROR: + reporter->FatalError("%s", cmsg); + break; + + case FATAL_ERROR_WITH_CORE: + reporter->FatalErrorWithCore("%s", cmsg); + break; + + case INTERNAL_WARNING: + reporter->InternalWarning("%s", cmsg); + break; + + case INTERNAL_ERROR : + reporter->InternalError("%s", cmsg); + break; + + default: + reporter->InternalError("unknown ReporterMessage type %d", type); + } + + return true; + } + +MsgThread::MsgThread() : BasicThread() + { + cnt_sent_in = cnt_sent_out = 0; + thread_mgr->AddMsgThread(this); + } + +void MsgThread::OnStop() + { + // This is to unblock the current queue read operation. + SendIn(new TerminateMessage(this), true); + } + +void MsgThread::Heartbeat() + { + SendIn(new HeartbeatMessage(this, network_time, current_time())); + } + +bool MsgThread::DoHeartbeat(double network_time, double current_time) + { + string n = Name(); + + n = Fmt("bro: %s (%" PRIu64 "/%" PRIu64 ")", n.c_str(), + cnt_sent_in - queue_in.Size(), + cnt_sent_out - queue_out.Size()); + + SetOSName(n.c_str()); + + return true; + } + +void MsgThread::Info(const char* msg) + { + SendOut(new ReporterMessage(ReporterMessage::INFO, this, msg)); + } + +void MsgThread::Warning(const char* msg) + { + SendOut(new ReporterMessage(ReporterMessage::WARNING, this, msg)); + } + +void MsgThread::Error(const char* msg) + { + SendOut(new ReporterMessage(ReporterMessage::ERROR, this, msg)); + } + +void MsgThread::FatalError(const char* msg) + { + SendOut(new ReporterMessage(ReporterMessage::FATAL_ERROR, this, msg)); + } + +void MsgThread::FatalErrorWithCore(const char* msg) + { + SendOut(new ReporterMessage(ReporterMessage::FATAL_ERROR_WITH_CORE, this, msg)); + } + +void MsgThread::InternalWarning(const char* msg) + { + SendOut(new ReporterMessage(ReporterMessage::INTERNAL_WARNING, this, msg)); + } + +void MsgThread::InternalError(const char* msg) + { + SendOut(new ReporterMessage(ReporterMessage::INTERNAL_ERROR, this, msg)); + } + +#ifdef DEBUG + +void MsgThread::Debug(DebugStream stream, const char* msg) + { + SendOut(new DebugMessage(stream, this, msg)); + } + +#endif + +void MsgThread::SendIn(BasicInputMessage* msg, bool force) + { + if ( Terminating() && ! force ) + { + delete msg; + return; + } + + DBG_LOG(DBG_THREADING, "Sending '%s' to %s ...", msg->Name().c_str(), Name().c_str()); + + queue_in.Put(msg); + ++cnt_sent_in; + } + + +void MsgThread::SendOut(BasicOutputMessage* msg, bool force) + { + if ( Terminating() && ! force ) + { + delete msg; + return; + } + + queue_out.Put(msg); + + ++cnt_sent_out; + } + +BasicOutputMessage* MsgThread::RetrieveOut() + { + BasicOutputMessage* msg = queue_out.Get(); + assert(msg); + + DBG_LOG(DBG_THREADING, "Retrieved '%s' from %s", msg->Name().c_str(), Name().c_str()); + + return msg; + } + +BasicInputMessage* MsgThread::RetrieveIn() + { + BasicInputMessage* msg = queue_in.Get(); + assert(msg); + +#ifdef DEBUG + string s = Fmt("Retrieved '%s' in %s", msg->Name().c_str(), Name().c_str()); + Debug(DBG_THREADING, s.c_str()); +#endif + + return msg; + } + +void MsgThread::Run() + { + while ( true ) + { + // When requested to terminate, we only do so when + // all input has been processed. + if ( Terminating() && ! queue_in.Ready() ) + break; + + BasicInputMessage* msg = RetrieveIn(); + + bool result = msg->Process(); + + if ( ! result ) + { + string s = msg->Name() + " failed, terminating thread"; + Error(s.c_str()); + Stop(); + break; + } + + delete msg; + } + } + +void MsgThread::GetStats(Stats* stats) + { + stats->sent_in = cnt_sent_in; + stats->sent_out = cnt_sent_out; + stats->pending_in = queue_in.Size(); + stats->pending_out = queue_out.Size(); + queue_in.GetStats(&stats->queue_in_stats); + queue_out.GetStats(&stats->queue_out_stats); + } + diff --git a/src/threading/MsgThread.h b/src/threading/MsgThread.h new file mode 100644 index 0000000000..5ac1c0f780 --- /dev/null +++ b/src/threading/MsgThread.h @@ -0,0 +1,402 @@ + +#ifndef THREADING_MSGTHREAD_H +#define THREADING_MSGTHREAD_H + +#include + +#include "DebugLogger.h" + +#include "BasicThread.h" +#include "Queue.h" + +namespace threading { + +class BasicInputMessage; +class BasicOutputMessage; +class HeartbeatMessage; + +/** + * A specialized thread that provides bi-directional message passing between + * Bro's main thread and the child thread. Messages are instances of + * BasicInputMessage and BasicOutputMessage for message sent \a to the child + * thread and received \a from the child thread, respectively. + * + * The thread's Run() method implements main loop that processes incoming + * messages until Terminating() indicates that execution should stop. Once + * that happens, the thread stops accepting any new messages, finishes + * processes all remaining ones still in the queue, and then exits. + */ +class MsgThread : public BasicThread +{ +public: + /** + * Constructor. It automatically registers the thread with the + * threading::Manager. + * + * Only Bro's main thread may instantiate a new thread. + */ + MsgThread(); + + /** + * Sends a message to the child thread. The message will be proceesed + * once the thread has retrieved it from its incoming queue. + * + * Only the main thread may call this method. + * + * @param msg The message. + */ + void SendIn(BasicInputMessage* msg) { return SendIn(msg, false); } + + /** + * Sends a message from the child thread to the main thread. + * + * Only the child thread may call this method. + * + * @param msg The mesasge. + */ + void SendOut(BasicOutputMessage* msg) { return SendOut(msg, false); } + + /** + * Reports an informational message from the child thread. The main + * thread will pass this to the Reporter once received. + * + * Only the child thread may call this method. + * + * @param msg The message. It will be prefixed with the thread's name. + */ + void Info(const char* msg); + + /** + * Reports a warning from the child thread that may indicate a + * problem. The main thread will pass this to the Reporter once + * received. + * + * Only the child thread may call this method. + * + * @param msg The message. It will be prefixed with the thread's name. + */ + void Warning(const char* msg); + + /** + * Reports a non-fatal error from the child thread. The main thread + * will pass this to the Reporter once received. Processing proceeds + * normally after the error has been reported. + * + * Only the child thread may call this method. + * + * @param msg The message. It will be prefixed with the thread's name. + */ + void Error(const char* msg); + + /** + * Reports a fatal error from the child thread. The main thread will + * pass this to the Reporter once received. Bro will terminate after + * the message has been reported. + * + * Only the child thread may call this method. + * + * @param msg The message. It will be prefixed with the thread's name. + */ + void FatalError(const char* msg); + + /** + * Reports a fatal error from the child thread. The main thread will + * pass this to the Reporter once received. Bro will terminate with a + * core dump after the message has been reported. + * + * Only the child thread may call this method. + * + * @param msg The message. It will be prefixed with the thread's name. + */ + void FatalErrorWithCore(const char* msg); + + /** + * Reports a potential internal problem from the child thread. The + * main thread will pass this to the Reporter once received. Bro will + * continue normally. + * + * Only the child thread may call this method. + * + * @param msg The message. It will be prefixed with the thread's name. + */ + void InternalWarning(const char* msg); + + /** + * Reports an internal program error from the child thread. The main + * thread will pass this to the Reporter once received. Bro will + * terminate with a core dump after the message has been reported. + * + * Only the child thread may call this method. + * + * @param msg The message. It will be prefixed with the thread's name. + */ + void InternalError(const char* msg); + +#ifdef DEBUG + /** + * Records a debug message for the given stream from the child + * thread. The main thread will pass this to the DebugLogger once + * received. + * + * Only the child thread may call this method. + * + * @param msg The message. It will be prefixed with the thread's name. + */ + void Debug(DebugStream stream, const char* msg); +#endif + + /** + * Statistics about inter-thread communication. + */ + struct Stats + { + uint64_t sent_in; //! Number of messages sent to the child thread. + uint64_t sent_out; //! Number of messages sent from the child thread to the main thread + uint64_t pending_in; //! Number of messages sent to the child but not yet processed. + uint64_t pending_out; //! Number of messages sent from the child but not yet processed by the main thread. + + /// Statistics from our queues. + Queue::Stats queue_in_stats; + Queue::Stats queue_out_stats; + }; + + /** + * Returns statistics about the inter-thread communication. + * + * @param stats A pointer to a structure that will be filled with + * current numbers. + */ + void GetStats(Stats* stats); + +protected: + friend class Manager; + friend class HeartbeatMessage; + + /** + * Pops a message sent by the child from the child-to-main queue. + * + * This is method is called regularly by the threading::Manager. + * + * @return The message, wth ownership passed to caller. Returns null + * if the queue is empty. + */ + BasicOutputMessage* RetrieveOut(); + + /** + * Triggers a heartbeat message being sent to the client thread. + * + * This is method is called regularly by the threading::Manager. + * + * Can be overriden in derived classed to hook into the heart beat, + * but must call the parent implementation. Note that this method is + * always called by the main thread and must not access data of the + * child thread directly. See DoHeartbeat() if you want to do + * something on the child-side. + */ + virtual void Heartbeat(); + + /** + * Overriden from BasicThread. + * + */ + virtual void Run(); + virtual void OnStop(); + + /** + * Regulatly triggered for execution in the child thread. + * + * When overriding, one must call the parent class' implementation. + * + * network_time: The network_time when the heartbeat was trigger by + * the main thread. + * + * current_time: Wall clock when the heartbeat was trigger by the + * main thread. + */ + virtual bool DoHeartbeat(double network_time, double current_time); + +private: + /** + * Pops a message sent by the main thread from the main-to-chold + * queue. + * + * Must only be called by the child thread. + * + * @return The message, wth ownership passed to caller. Returns null + * if the queue is empty. + */ + BasicInputMessage* RetrieveIn(); + + /** + * Queues a message for the child. + * + * Must only be called by the main thread. + * + * @param msg The message. + * + * @param force: If true, the message will be queued even when we're already + * Terminating(). Normally, the message would be discarded in that + * case. + */ + void SendIn(BasicInputMessage* msg, bool force); + + /** + * Queues a message for the main thread. + * + * Must only be called by the child thread. + * + * @param msg The message. + * + * @param force: If true, the message will be queued even when we're already + * Terminating(). Normally, the message would be discarded in that + * case. + */ + void SendOut(BasicOutputMessage* msg, bool force); + + /** + * Returns true if there's at least one message pending for the child + * thread. + */ + bool HasIn() { return queue_in.Ready(); } + + /** + * Returns true if there's at least one message pending for the main + * thread. + */ + bool HasOut() { return queue_out.Ready(); } + + Queue queue_in; + Queue queue_out; + + uint64_t cnt_sent_in; // Counts message sent to child. + uint64_t cnt_sent_out; // Counts message sent by child. +}; + +/** + * Base class for all message between Bro's main process and a MsgThread. + */ +class Message +{ +public: + /** + * Destructor. + */ + virtual ~Message(); + + /** + * Returns a descriptive name for the message's general type. This is + * what's passed into the constructor and used mainly for debugging + * purposes. + */ + const string& Name() const { return name; } + + /** + * Callback that must be overriden for processing a message. + */ + virtual bool Process() = 0; // Thread will be terminated if returngin false. + +protected: + /** + * Constructor. + * + * @param arg_name A descriptive name for the type of message. Used + * mainly for debugging purposes. + */ + Message(const string& arg_name) { name = arg_name; } + +private: + string name; +}; + +/** + * Base class for messages sent from Bro's main thread to a child MsgThread. + */ +class BasicInputMessage : public Message +{ +protected: + /** + * Constructor. + * + * @param name A descriptive name for the type of message. Used + * mainly for debugging purposes. + */ + BasicInputMessage(const string& name) : Message(name) {} +}; + +/** + * Base class for messages sent from a child MsgThread to Bro's main thread. + */ +class BasicOutputMessage : public Message +{ +protected: + /** + * Constructor. + * + * @param name A descriptive name for the type of message. Used + * mainly for debugging purposes. + */ + BasicOutputMessage(const string& name) : Message(name) {} +}; + +/** + * A paremeterized InputMessage that stores a pointer to an argument object. + * Normally, the objects will be used from the Process() callback. + */ +template +class InputMessage : public BasicInputMessage +{ +public: + /** + * Returns the objects passed to the constructor. + */ + O* Object() const { return object; } + +protected: + /** + * Constructor. + * + * @param name: A descriptive name for the type of message. Used + * mainly for debugging purposes. + * + * @param arg_object: An object to store with the message. + */ + InputMessage(const string& name, O* arg_object) : BasicInputMessage(name) + { object = arg_object; } + +private: + O* object; +}; + +/** + * A paremeterized OututMessage that stores a pointer to an argument object. + * Normally, the objects will be used from the Process() callback. + */ +template +class OutputMessage : public BasicOutputMessage +{ +public: + /** + * Returns the objects passed to the constructor. + */ + O* Object() const { return object; } + +protected: + /** + * Constructor. + * + * @param name A descriptive name for the type of message. Used + * mainly for debugging purposes. + * + * @param arg_object An object to store with the message. + */ + OutputMessage(const string& name, O* arg_object) : BasicOutputMessage(name) + { object = arg_object; } + +private: + O* object; +}; + +} + + +#endif diff --git a/src/threading/Queue.h b/src/threading/Queue.h new file mode 100644 index 0000000000..985ba31714 --- /dev/null +++ b/src/threading/Queue.h @@ -0,0 +1,222 @@ + +#ifndef THREADING_QUEUE_H +#define THREADING_QUEUE_H + +#include +#include +#include +#include + +#include "Reporter.h" + +#undef Queue // Defined elsewhere unfortunately. + +namespace threading { + +/** + * A thread-safe single-reader single-writer queue. + * + * The implementation uses multiple queues and reads/writes in rotary fashion + * in an attempt to limit contention. + * + * All Queue instances must be instantiated by Bro's main thread. + * + * TODO: Unclear how critical performance is for this qeueue. We could like;y + * optimize it further if helpful. + */ +template +class Queue +{ +public: + /** + * Constructor. + */ + Queue(); + + /** + * Destructor. + */ + ~Queue(); + + /** + * Retrieves one elment. + */ + T Get(); + + /** + * Queues one element. + */ + void Put(T data); + + /** + * Returns true if the next Get() operation will succeed. + */ + bool Ready(); + + /** + * Returns the number of queued items not yet retrieved. + */ + uint64_t Size(); + + /** + * Statistics about inter-thread communication. + */ + struct Stats + { + uint64_t num_reads; //! Number of messages read from the queue. + uint64_t num_writes; //! Number of messages written to the queue. + }; + + /** + * Returns statistics about the queue's usage. + * + * @param stats A pointer to a structure that will be filled with + * current numbers. */ + void GetStats(Stats* stats); + +private: + static const int NUM_QUEUES = 8; + + pthread_mutex_t mutex[NUM_QUEUES]; // Mutex protected shared accesses. + pthread_cond_t has_data[NUM_QUEUES]; // Signals when data becomes available + std::queue messages[NUM_QUEUES]; // Actually holds the queued messages + + int read_ptr; // Where the next operation will read from + int write_ptr; // Where the next operation will write to + + // Statistics. + uint64_t num_reads; + uint64_t num_writes; +}; + +inline static void safe_lock(pthread_mutex_t* mutex) + { + if ( pthread_mutex_lock(mutex) != 0 ) + reporter->FatalErrorWithCore("cannot lock mutex"); + } + +inline static void safe_unlock(pthread_mutex_t* mutex) + { + if ( pthread_mutex_unlock(mutex) != 0 ) + reporter->FatalErrorWithCore("cannot unlock mutex"); + } + +template +inline Queue::Queue() + { + read_ptr = 0; + write_ptr = 0; + num_reads = num_writes = 0; + + for( int i = 0; i < NUM_QUEUES; ++i ) + { + if ( pthread_cond_init(&has_data[i], NULL) != 0 ) + reporter->FatalError("cannot init queue condition variable"); + + if ( pthread_mutex_init(&mutex[i], NULL) != 0 ) + reporter->FatalError("cannot init queue mutex"); + } + } + +template +inline Queue::~Queue() + { + for( int i = 0; i < NUM_QUEUES; ++i ) + { + pthread_cond_destroy(&has_data[i]); + pthread_mutex_destroy(&mutex[i]); + } + } + +template +inline T Queue::Get() + { + safe_lock(&mutex[read_ptr]); + + int old_read_ptr = read_ptr; + + if ( messages[read_ptr].empty() ) + pthread_cond_wait(&has_data[read_ptr], &mutex[read_ptr]); + + T data = messages[read_ptr].front(); + messages[read_ptr].pop(); + + read_ptr = (read_ptr + 1) % NUM_QUEUES; + ++num_reads; + + safe_unlock(&mutex[old_read_ptr]); + + return data; + } + +template +inline void Queue::Put(T data) + { + safe_lock(&mutex[write_ptr]); + + int old_write_ptr = write_ptr; + + bool need_signal = messages[write_ptr].empty(); + + messages[write_ptr].push(data); + + if ( need_signal ) + pthread_cond_signal(&has_data[write_ptr]); + + write_ptr = (write_ptr + 1) % NUM_QUEUES; + ++num_writes; + + safe_unlock(&mutex[old_write_ptr]); + } + + +template +inline bool Queue::Ready() + { + safe_lock(&mutex[read_ptr]); + + bool ret = (messages[read_ptr].size()); + + safe_unlock(&mutex[read_ptr]); + + return ret; + } + +template +inline uint64_t Queue::Size() + { + // Need to lock all queues. + for ( int i = 0; i < NUM_QUEUES; i++ ) + safe_lock(&mutex[i]); + + uint64_t size = 0; + + for ( int i = 0; i < NUM_QUEUES; i++ ) + size += messages[i].size(); + + for ( int i = 0; i < NUM_QUEUES; i++ ) + safe_unlock(&mutex[i]); + + return size; + } + +template +inline void Queue::GetStats(Stats* stats) + { + // To be safe, we look all queues. That's probably unneccessary, but + // doesn't really hurt. + for ( int i = 0; i < NUM_QUEUES; i++ ) + safe_lock(&mutex[i]); + + stats->num_reads = num_reads; + stats->num_writes = num_writes; + + for ( int i = 0; i < NUM_QUEUES; i++ ) + safe_unlock(&mutex[i]); + } + +} + + +#endif + diff --git a/src/threading/SerialTypes.cc b/src/threading/SerialTypes.cc new file mode 100644 index 0000000000..a5692b2ffd --- /dev/null +++ b/src/threading/SerialTypes.cc @@ -0,0 +1,355 @@ +// See the file "COPYING" in the main distribution directory for copyright. + + +#include "SerialTypes.h" +#include "../RemoteSerializer.h" + + +using namespace threading; + +bool Field::Read(SerializationFormat* fmt) + { + int t; + int st; + + bool success = (fmt->Read(&name, "name") && fmt->Read(&t, "type") && fmt->Read(&st, "subtype") ); + type = (TypeTag) t; + subtype = (TypeTag) st; + + return success; + } + +bool Field::Write(SerializationFormat* fmt) const + { + return (fmt->Write(name, "name") && fmt->Write((int)type, "type") && fmt->Write((int)subtype, "subtype")); + } + +Value::~Value() + { + if ( (type == TYPE_ENUM || type == TYPE_STRING || type == TYPE_FILE || type == TYPE_FUNC) + && present ) + delete val.string_val; + + if ( type == TYPE_TABLE && present ) + { + for ( int i = 0; i < val.set_val.size; i++ ) + delete val.set_val.vals[i]; + + delete [] val.set_val.vals; + } + + if ( type == TYPE_VECTOR && present ) + { + for ( int i = 0; i < val.vector_val.size; i++ ) + delete val.vector_val.vals[i]; + + delete [] val.vector_val.vals; + } + } + +bool Value::IsCompatibleType(BroType* t, bool atomic_only) + { + if ( ! t ) + return false; + + switch ( t->Tag() ) { + case TYPE_BOOL: + case TYPE_INT: + case TYPE_COUNT: + case TYPE_COUNTER: + case TYPE_PORT: + case TYPE_SUBNET: + case TYPE_ADDR: + case TYPE_DOUBLE: + case TYPE_TIME: + case TYPE_INTERVAL: + case TYPE_ENUM: + case TYPE_STRING: + case TYPE_FILE: + case TYPE_FUNC: + return true; + + case TYPE_RECORD: + return ! atomic_only; + + case TYPE_TABLE: + { + if ( atomic_only ) + return false; + + if ( ! t->IsSet() ) + return false; + + return IsCompatibleType(t->AsSetType()->Indices()->PureType(), true); + } + + case TYPE_VECTOR: + { + if ( atomic_only ) + return false; + + return IsCompatibleType(t->AsVectorType()->YieldType(), true); + } + + default: + return false; + } + + return false; + } + +bool Value::Read(SerializationFormat* fmt) + { + int ty; + + if ( ! (fmt->Read(&ty, "type") && fmt->Read(&present, "present")) ) + return false; + + type = (TypeTag)(ty); + + if ( ! present ) + return true; + + switch ( type ) { + case TYPE_BOOL: + case TYPE_INT: + return fmt->Read(&val.int_val, "int"); + + case TYPE_COUNT: + case TYPE_COUNTER: + return fmt->Read(&val.uint_val, "uint"); + + case TYPE_PORT: { + int proto; + if ( ! (fmt->Read(&val.port_val.port, "port") && fmt->Read(&proto, "proto") ) ) { + return false; + } + + switch ( proto ) { + case 0: + val.port_val.proto = TRANSPORT_UNKNOWN; + break; + case 1: + val.port_val.proto = TRANSPORT_TCP; + break; + case 2: + val.port_val.proto = TRANSPORT_UDP; + break; + case 3: + val.port_val.proto = TRANSPORT_ICMP; + break; + default: + return false; + } + + return true; + } + + case TYPE_ADDR: + { + char family; + + if ( ! fmt->Read(&family, "addr-family") ) + return false; + + switch ( family ) { + case 4: + val.addr_val.family = IPv4; + return fmt->Read(&val.addr_val.in.in4, "addr-in4"); + + case 6: + val.addr_val.family = IPv6; + return fmt->Read(&val.addr_val.in.in6, "addr-in6"); + + } + + // Can't be reached. + abort(); + } + + case TYPE_SUBNET: + { + char length; + char family; + + if ( ! (fmt->Read(&length, "subnet-len") && fmt->Read(&family, "subnet-family")) ) + return false; + + switch ( family ) { + case 4: + val.subnet_val.length = (uint8_t)length; + val.subnet_val.prefix.family = IPv4; + return fmt->Read(&val.subnet_val.prefix.in.in4, "subnet-in4"); + + case 6: + val.subnet_val.length = (uint8_t)length; + val.subnet_val.prefix.family = IPv6; + return fmt->Read(&val.subnet_val.prefix.in.in6, "subnet-in6"); + + } + + // Can't be reached. + abort(); + } + + case TYPE_DOUBLE: + case TYPE_TIME: + case TYPE_INTERVAL: + return fmt->Read(&val.double_val, "double"); + + case TYPE_ENUM: + case TYPE_STRING: + case TYPE_FILE: + case TYPE_FUNC: + { + val.string_val = new string; + return fmt->Read(val.string_val, "string"); + } + + case TYPE_TABLE: + { + if ( ! fmt->Read(&val.set_val.size, "set_size") ) + return false; + + val.set_val.vals = new Value* [val.set_val.size]; + + for ( int i = 0; i < val.set_val.size; ++i ) + { + val.set_val.vals[i] = new Value; + + if ( ! val.set_val.vals[i]->Read(fmt) ) + return false; + } + + return true; + } + + case TYPE_VECTOR: + { + if ( ! fmt->Read(&val.vector_val.size, "vector_size") ) + return false; + + val.vector_val.vals = new Value* [val.vector_val.size]; + + for ( int i = 0; i < val.vector_val.size; ++i ) + { + val.vector_val.vals[i] = new Value; + + if ( ! val.vector_val.vals[i]->Read(fmt) ) + return false; + } + + return true; + } + + default: + reporter->InternalError("unsupported type %s in Value::Write", type_name(type)); + } + + return false; + } + +bool Value::Write(SerializationFormat* fmt) const + { + if ( ! (fmt->Write((int)type, "type") && + fmt->Write(present, "present")) ) + return false; + + if ( ! present ) + return true; + + switch ( type ) { + case TYPE_BOOL: + case TYPE_INT: + return fmt->Write(val.int_val, "int"); + + case TYPE_COUNT: + case TYPE_COUNTER: + return fmt->Write(val.uint_val, "uint"); + + case TYPE_PORT: + return fmt->Write(val.port_val.port, "port") && fmt->Write(val.port_val.proto, "proto"); + + case TYPE_ADDR: + { + switch ( val.addr_val.family ) { + case IPv4: + return fmt->Write((char)4, "addr-family") + && fmt->Write(val.addr_val.in.in4, "addr-in4"); + + case IPv6: + return fmt->Write((char)6, "addr-family") + && fmt->Write(val.addr_val.in.in6, "addr-in6"); + break; + } + + // Can't be reached. + abort(); + } + + case TYPE_SUBNET: + { + if ( ! fmt->Write((char)val.subnet_val.length, "subnet-length") ) + return false; + + switch ( val.subnet_val.prefix.family ) { + case IPv4: + return fmt->Write((char)4, "subnet-family") + && fmt->Write(val.subnet_val.prefix.in.in4, "subnet-in4"); + + case IPv6: + return fmt->Write((char)6, "subnet-family") + && fmt->Write(val.subnet_val.prefix.in.in6, "subnet-in6"); + break; + } + + // Can't be reached. + abort(); + } + + case TYPE_DOUBLE: + case TYPE_TIME: + case TYPE_INTERVAL: + return fmt->Write(val.double_val, "double"); + + case TYPE_ENUM: + case TYPE_STRING: + case TYPE_FILE: + case TYPE_FUNC: + return fmt->Write(*val.string_val, "string"); + + case TYPE_TABLE: + { + if ( ! fmt->Write(val.set_val.size, "set_size") ) + return false; + + for ( int i = 0; i < val.set_val.size; ++i ) + { + if ( ! val.set_val.vals[i]->Write(fmt) ) + return false; + } + + return true; + } + + case TYPE_VECTOR: + { + if ( ! fmt->Write(val.vector_val.size, "vector_size") ) + return false; + + for ( int i = 0; i < val.vector_val.size; ++i ) + { + if ( ! val.vector_val.vals[i]->Write(fmt) ) + return false; + } + + return true; + } + + default: + reporter->InternalError("unsupported type %s in Value::REad", type_name(type)); + } + + return false; + } + diff --git a/src/threading/SerialTypes.h b/src/threading/SerialTypes.h new file mode 100644 index 0000000000..db7dc837bd --- /dev/null +++ b/src/threading/SerialTypes.h @@ -0,0 +1,146 @@ + +#ifndef THREADING_SERIALIZATIONTYPES_H +#define THREADING_SERIALIZATIONTYPES_H + +using namespace std; + +#include +#include +#include + +#include "Type.h" +#include "net_util.h" + +class SerializationFormat; + +namespace threading { + +/** + * Definition of a log file, i.e., one column of a log stream. + */ +struct Field { + string name; //! Name of the field. + TypeTag type; //! Type of the field. + TypeTag subtype; //! Inner type for sets. + + /** + * Constructor. + */ + Field() { subtype = TYPE_VOID; } + + /** + * Copy constructor. + */ + Field(const Field& other) + : name(other.name), type(other.type), subtype(other.subtype) { } + + /** + * Unserializes a field. + * + * @param fmt The serialization format to use. The format handles + * low-level I/O. + * + * @return False if an error occured. + */ + bool Read(SerializationFormat* fmt); + + /** + * Serializes a field. + * + * @param fmt The serialization format to use. The format handles + * low-level I/O. + * + * @return False if an error occured. + */ + bool Write(SerializationFormat* fmt) const; +}; + +/** + * Definition of a log value, i.e., a entry logged by a stream. + * + * This struct essentialy represents a serialization of a Val instance (for + * those Vals supported). + */ +struct Value { + TypeTag type; //! The type of the value. + bool present; //! False for optional record fields that are not set. + + struct set_t { bro_int_t size; Value** vals; }; + typedef set_t vec_t; + struct port_t { bro_uint_t port; TransportProto proto; }; + + struct addr_t { + IPFamily family; + union { + struct in_addr in4; + struct in6_addr in6; + } in; + }; + + struct subnet_t { addr_t prefix; uint8_t length; }; + + /** + * This union is a subset of BroValUnion, including only the types we + * can log directly. See IsCompatibleType(). + */ + union _val { + bro_int_t int_val; + bro_uint_t uint_val; + port_t port_val; + double double_val; + set_t set_val; + vec_t vector_val; + addr_t addr_val; + subnet_t subnet_val; + string* string_val; + } val; + + /** + * Constructor. + * + * arg_type: The type of the value. + * + * arg_present: False if the value represents an optional record field + * that is not set. + */ + Value(TypeTag arg_type = TYPE_ERROR, bool arg_present = true) + : type(arg_type), present(arg_present) {} + + /** + * Destructor. + */ + ~Value(); + + /** + * Unserializes a value. + * + * @param fmt The serialization format to use. The format handles low-level I/O. + * + * @return False if an error occured. + */ + bool Read(SerializationFormat* fmt); + + /** + * Serializes a value. + * + * @param fmt The serialization format to use. The format handles + * low-level I/O. + * + * @return False if an error occured. + */ + bool Write(SerializationFormat* fmt) const; + + /** + * Returns true if the type can be represented by a Value. If + * `atomic_only` is true, will not permit composite types. + */ + static bool IsCompatibleType(BroType* t, bool atomic_only=false); + +private: +friend class ::IPAddr; + Value(const Value& other) { } // Disabled. +}; + +} + +#endif /* THREADING_SERIALIZATIONTZPES_H */ diff --git a/src/util.h b/src/util.h index 498bdf00e4..a4e3aa71b8 100644 --- a/src/util.h +++ b/src/util.h @@ -37,7 +37,7 @@ #endif -#ifdef USE_PERFTOOLS +#ifdef USE_PERFTOOLS_DEBUG #include #include extern HeapLeakChecker* heap_checker; diff --git a/testing/Makefile b/testing/Makefile index f65d5a1fef..1c82580ec4 100644 --- a/testing/Makefile +++ b/testing/Makefile @@ -12,6 +12,7 @@ make-brief: @for repo in $(DIRS); do (cd $$repo && make brief ); done coverage: + @for repo in $(DIRS); do (cd $$repo && echo "Coverage for '$$repo' dir:" && make coverage); done @test -f btest/coverage.log && cp btest/coverage.log `mktemp brocov.tmp.XXX` || true @for f in external/*/coverage.log; do test -f $$f && cp $$f `mktemp brocov.tmp.XXX` || true; done @echo "Complete test suite code coverage:" diff --git a/testing/btest/.gitignore b/testing/btest/.gitignore index 5282177d90..b4c1b7a858 100644 --- a/testing/btest/.gitignore +++ b/testing/btest/.gitignore @@ -1,3 +1,4 @@ .tmp +.btest.failed.dat diag.log coverage.log diff --git a/testing/btest/Baseline/bifs.addr_count_conversion/output b/testing/btest/Baseline/bifs.addr_count_conversion/output new file mode 100644 index 0000000000..08a74512d3 --- /dev/null +++ b/testing/btest/Baseline/bifs.addr_count_conversion/output @@ -0,0 +1,4 @@ +[536939960, 2242052096, 35374, 57701172] +2001:db8:85a3::8a2e:370:7334 +[16909060] +1.2.3.4 diff --git a/testing/btest/Baseline/bifs.addr_to_ptr_name/output b/testing/btest/Baseline/bifs.addr_to_ptr_name/output new file mode 100644 index 0000000000..8de05d1d8e --- /dev/null +++ b/testing/btest/Baseline/bifs.addr_to_ptr_name/output @@ -0,0 +1,2 @@ +2.1.0.1.0.0.0.0.0.0.0.0.0.0.0.0.2.0.8.0.9.0.0.4.0.b.8.f.7.0.6.2.ip6.arpa +52.225.125.74.in-addr.arpa diff --git a/testing/btest/Baseline/bifs.addr_version/out b/testing/btest/Baseline/bifs.addr_version/out new file mode 100644 index 0000000000..328bff6687 --- /dev/null +++ b/testing/btest/Baseline/bifs.addr_version/out @@ -0,0 +1,4 @@ +T +F +F +T diff --git a/testing/btest/Baseline/bifs.install_src_addr_filter/output b/testing/btest/Baseline/bifs.install_src_addr_filter/output new file mode 100644 index 0000000000..bf99083391 --- /dev/null +++ b/testing/btest/Baseline/bifs.install_src_addr_filter/output @@ -0,0 +1,8 @@ +[orig_h=141.142.220.118, orig_p=48649/tcp, resp_h=208.80.152.118, resp_p=80/tcp] +[orig_h=141.142.220.118, orig_p=49996/tcp, resp_h=208.80.152.3, resp_p=80/tcp] +[orig_h=141.142.220.118, orig_p=49997/tcp, resp_h=208.80.152.3, resp_p=80/tcp] +[orig_h=141.142.220.118, orig_p=49998/tcp, resp_h=208.80.152.3, resp_p=80/tcp] +[orig_h=141.142.220.118, orig_p=49999/tcp, resp_h=208.80.152.3, resp_p=80/tcp] +[orig_h=141.142.220.118, orig_p=50000/tcp, resp_h=208.80.152.3, resp_p=80/tcp] +[orig_h=141.142.220.118, orig_p=50001/tcp, resp_h=208.80.152.3, resp_p=80/tcp] +[orig_h=141.142.220.118, orig_p=35642/tcp, resp_h=208.80.152.2, resp_p=80/tcp] diff --git a/testing/btest/Baseline/bifs.net_stats_trace/output b/testing/btest/Baseline/bifs.net_stats_trace/output index a2e25e03a7..55d6693db5 100644 --- a/testing/btest/Baseline/bifs.net_stats_trace/output +++ b/testing/btest/Baseline/bifs.net_stats_trace/output @@ -1 +1 @@ -[pkts_recvd=131, pkts_dropped=0, pkts_link=0] +[pkts_recvd=136, pkts_dropped=0, pkts_link=0] diff --git a/testing/btest/Baseline/bifs.ptr_name_to_addr/output b/testing/btest/Baseline/bifs.ptr_name_to_addr/output new file mode 100644 index 0000000000..7c290027aa --- /dev/null +++ b/testing/btest/Baseline/bifs.ptr_name_to_addr/output @@ -0,0 +1,2 @@ +2607:f8b0:4009:802::1012 +74.125.225.52 diff --git a/testing/btest/Baseline/bifs.remask_addr/output b/testing/btest/Baseline/bifs.remask_addr/output new file mode 100644 index 0000000000..1d276abd4f --- /dev/null +++ b/testing/btest/Baseline/bifs.remask_addr/output @@ -0,0 +1,32 @@ +1: 127.255.0.0 +2: 63.255.0.0 +3: 31.255.0.0 +4: 15.255.0.0 +5: 7.255.0.0 +6: 3.255.0.0 +7: 1.255.0.0 +8: 0.255.0.0 +9: 0.127.0.0 +10: 0.63.0.0 +11: 0.31.0.0 +12: 0.15.0.0 +13: 0.7.0.0 +14: 0.3.0.0 +15: 0.1.0.0 +16: 0.0.0.0 +17: 0.0.128.0 +18: 0.0.192.0 +19: 0.0.224.0 +20: 0.0.240.0 +21: 0.0.248.0 +22: 0.0.252.0 +23: 0.0.254.0 +24: 0.0.255.0 +25: 0.0.255.128 +26: 0.0.255.192 +27: 0.0.255.224 +28: 0.0.255.240 +29: 0.0.255.248 +30: 0.0.255.252 +31: 0.0.255.254 +32: 0.0.255.255 diff --git a/testing/btest/Baseline/bifs.routing0_data_to_addrs/output b/testing/btest/Baseline/bifs.routing0_data_to_addrs/output new file mode 100644 index 0000000000..c79aef89d0 --- /dev/null +++ b/testing/btest/Baseline/bifs.routing0_data_to_addrs/output @@ -0,0 +1 @@ +[2001:78:1:32::1, 2001:78:1:32::2] diff --git a/testing/btest/Baseline/bifs.to_addr/error b/testing/btest/Baseline/bifs.to_addr/error new file mode 100644 index 0000000000..b8ba985f7a --- /dev/null +++ b/testing/btest/Baseline/bifs.to_addr/error @@ -0,0 +1 @@ +error: Bad IP address: not an IP diff --git a/testing/btest/Baseline/bifs.to_addr/output b/testing/btest/Baseline/bifs.to_addr/output new file mode 100644 index 0000000000..ff277498f8 --- /dev/null +++ b/testing/btest/Baseline/bifs.to_addr/output @@ -0,0 +1,9 @@ +to_addr(0.0.0.0) = 0.0.0.0 (SUCCESS) +to_addr(1.2.3.4) = 1.2.3.4 (SUCCESS) +to_addr(01.02.03.04) = 1.2.3.4 (SUCCESS) +to_addr(001.002.003.004) = 1.2.3.4 (SUCCESS) +to_addr(10.20.30.40) = 10.20.30.40 (SUCCESS) +to_addr(100.200.30.40) = 100.200.30.40 (SUCCESS) +to_addr(10.0.0.0) = 10.0.0.0 (SUCCESS) +to_addr(10.00.00.000) = 10.0.0.0 (SUCCESS) +to_addr(not an IP) = :: (SUCCESS) diff --git a/testing/btest/Baseline/bifs.to_subnet/error b/testing/btest/Baseline/bifs.to_subnet/error new file mode 100644 index 0000000000..ee0062cd83 --- /dev/null +++ b/testing/btest/Baseline/bifs.to_subnet/error @@ -0,0 +1 @@ +error: Bad string in SubNetVal ctor: 10.0.0.0 diff --git a/testing/btest/Baseline/bifs.to_subnet/output b/testing/btest/Baseline/bifs.to_subnet/output new file mode 100644 index 0000000000..0775063f89 --- /dev/null +++ b/testing/btest/Baseline/bifs.to_subnet/output @@ -0,0 +1,3 @@ +10.0.0.0/8, T +2607:f8b0::/32, T +::/0, T diff --git a/testing/btest/Baseline/core.checksums/bad.out b/testing/btest/Baseline/core.checksums/bad.out new file mode 100644 index 0000000000..57089a72a6 --- /dev/null +++ b/testing/btest/Baseline/core.checksums/bad.out @@ -0,0 +1,13 @@ +1332784981.078396 weird: bad_IP_checksum +1332784885.686428 weird: bad_TCP_checksum +1332784933.501023 weird: bad_UDP_checksum +1334075363.536871 weird: bad_ICMP_checksum +1332785210.013051 weird: routing0_hdr +1332785210.013051 weird: bad_TCP_checksum +1332782580.798420 weird: routing0_hdr +1332782580.798420 weird: bad_UDP_checksum +1334075111.800086 weird: routing0_hdr +1334075111.800086 weird: bad_ICMP_checksum +1332785250.469132 weird: bad_TCP_checksum +1332781342.923813 weird: bad_UDP_checksum +1334074939.467194 weird: bad_ICMP_checksum diff --git a/testing/btest/Baseline/core.checksums/good.out b/testing/btest/Baseline/core.checksums/good.out new file mode 100644 index 0000000000..4330967d8d --- /dev/null +++ b/testing/btest/Baseline/core.checksums/good.out @@ -0,0 +1,3 @@ +1332785125.596793 weird: routing0_hdr +1332782508.592037 weird: routing0_hdr +1334075027.053380 weird: routing0_hdr diff --git a/testing/btest/Baseline/core.conn-uid/counts b/testing/btest/Baseline/core.conn-uid/counts index a8fa06e1be..38b10c1b2b 100644 --- a/testing/btest/Baseline/core.conn-uid/counts +++ b/testing/btest/Baseline/core.conn-uid/counts @@ -1 +1 @@ -62 +68 diff --git a/testing/btest/Baseline/core.conn-uid/output b/testing/btest/Baseline/core.conn-uid/output index 6f58e7c10c..c77eda4f04 100644 --- a/testing/btest/Baseline/core.conn-uid/output +++ b/testing/btest/Baseline/core.conn-uid/output @@ -1,40 +1,43 @@ [orig_h=141.142.220.202, orig_p=5353/udp, resp_h=224.0.0.251, resp_p=5353/udp], UWkUyAuUGXf -[orig_h=141.142.220.50, orig_p=5353/udp, resp_h=224.0.0.251, resp_p=5353/udp], arKYeMETxOg -[orig_h=141.142.220.118, orig_p=35634/tcp, resp_h=208.80.152.2, resp_p=80/tcp], k6kgXLOoSKl -[orig_h=141.142.220.118, orig_p=48649/tcp, resp_h=208.80.152.118, resp_p=80/tcp], nQcgTWjvg4c -[orig_h=141.142.220.118, orig_p=48649/tcp, resp_h=208.80.152.118, resp_p=80/tcp], nQcgTWjvg4c -[orig_h=141.142.220.118, orig_p=43927/udp, resp_h=141.142.2.2, resp_p=53/udp], j4u32Pc5bif -[orig_h=141.142.220.118, orig_p=37676/udp, resp_h=141.142.2.2, resp_p=53/udp], TEfuqmmG4bh -[orig_h=141.142.220.118, orig_p=40526/udp, resp_h=141.142.2.2, resp_p=53/udp], FrJExwHcSal -[orig_h=141.142.220.118, orig_p=49996/tcp, resp_h=208.80.152.3, resp_p=80/tcp], 5OKnoww6xl4 -[orig_h=141.142.220.118, orig_p=49997/tcp, resp_h=208.80.152.3, resp_p=80/tcp], 3PKsZ2Uye21 -[orig_h=141.142.220.118, orig_p=32902/udp, resp_h=141.142.2.2, resp_p=53/udp], VW0XPVINV8a -[orig_h=141.142.220.118, orig_p=59816/udp, resp_h=141.142.2.2, resp_p=53/udp], fRFu0wcOle6 -[orig_h=141.142.220.118, orig_p=59714/udp, resp_h=141.142.2.2, resp_p=53/udp], qSsw6ESzHV4 -[orig_h=141.142.220.118, orig_p=49998/tcp, resp_h=208.80.152.3, resp_p=80/tcp], iE6yhOq3SF -[orig_h=141.142.220.118, orig_p=58206/udp, resp_h=141.142.2.2, resp_p=53/udp], GSxOnSLghOa -[orig_h=141.142.220.118, orig_p=38911/udp, resp_h=141.142.2.2, resp_p=53/udp], qCaWGmzFtM5 -[orig_h=141.142.220.118, orig_p=59746/udp, resp_h=141.142.2.2, resp_p=53/udp], 70MGiRM1Qf4 -[orig_h=141.142.220.118, orig_p=49999/tcp, resp_h=208.80.152.3, resp_p=80/tcp], h5DsfNtYzi1 -[orig_h=141.142.220.118, orig_p=50000/tcp, resp_h=208.80.152.3, resp_p=80/tcp], P654jzLoe3a -[orig_h=141.142.220.118, orig_p=45000/udp, resp_h=141.142.2.2, resp_p=53/udp], Tw8jXtpTGu6 -[orig_h=141.142.220.118, orig_p=48479/udp, resp_h=141.142.2.2, resp_p=53/udp], c4Zw9TmAE05 -[orig_h=141.142.220.118, orig_p=48128/udp, resp_h=141.142.2.2, resp_p=53/udp], EAr0uf4mhq -[orig_h=141.142.220.118, orig_p=50001/tcp, resp_h=208.80.152.3, resp_p=80/tcp], GvmoxJFXdTa -[orig_h=141.142.220.118, orig_p=56056/udp, resp_h=141.142.2.2, resp_p=53/udp], 0Q4FH8sESw5 -[orig_h=141.142.220.118, orig_p=55092/udp, resp_h=141.142.2.2, resp_p=53/udp], slFea8xwSmb -[orig_h=141.142.220.118, orig_p=35642/tcp, resp_h=208.80.152.2, resp_p=80/tcp], UfGkYA2HI2g -[orig_h=141.142.220.118, orig_p=49997/tcp, resp_h=208.80.152.3, resp_p=80/tcp], 3PKsZ2Uye21 -[orig_h=141.142.220.118, orig_p=49996/tcp, resp_h=208.80.152.3, resp_p=80/tcp], 5OKnoww6xl4 -[orig_h=141.142.220.118, orig_p=49998/tcp, resp_h=208.80.152.3, resp_p=80/tcp], iE6yhOq3SF -[orig_h=141.142.220.118, orig_p=50000/tcp, resp_h=208.80.152.3, resp_p=80/tcp], P654jzLoe3a -[orig_h=141.142.220.118, orig_p=49999/tcp, resp_h=208.80.152.3, resp_p=80/tcp], h5DsfNtYzi1 -[orig_h=141.142.220.118, orig_p=50001/tcp, resp_h=208.80.152.3, resp_p=80/tcp], GvmoxJFXdTa -[orig_h=141.142.220.118, orig_p=35642/tcp, resp_h=208.80.152.2, resp_p=80/tcp], UfGkYA2HI2g -[orig_h=141.142.220.235, orig_p=6705/tcp, resp_h=173.192.163.128, resp_p=80/tcp], i2rO3KD1Syg -[orig_h=141.142.220.235, orig_p=6705/tcp, resp_h=173.192.163.128, resp_p=80/tcp], i2rO3KD1Syg -[orig_h=141.142.220.44, orig_p=5353/udp, resp_h=224.0.0.251, resp_p=5353/udp], 2cx26uAvUPl -[orig_h=141.142.220.226, orig_p=137/udp, resp_h=141.142.220.255, resp_p=137/udp], BWaU4aSuwkc -[orig_h=141.142.220.226, orig_p=55131/udp, resp_h=224.0.0.252, resp_p=5355/udp], 10XodEwRycf -[orig_h=141.142.220.226, orig_p=55671/udp, resp_h=224.0.0.252, resp_p=5355/udp], zno26fFZkrh -[orig_h=141.142.220.238, orig_p=56641/udp, resp_h=141.142.220.255, resp_p=137/udp], v5rgkJBig5l +[orig_h=fe80::217:f2ff:fed7:cf65, orig_p=5353/udp, resp_h=ff02::fb, resp_p=5353/udp], arKYeMETxOg +[orig_h=141.142.220.50, orig_p=5353/udp, resp_h=224.0.0.251, resp_p=5353/udp], k6kgXLOoSKl +[orig_h=141.142.220.118, orig_p=35634/tcp, resp_h=208.80.152.2, resp_p=80/tcp], nQcgTWjvg4c +[orig_h=141.142.220.118, orig_p=48649/tcp, resp_h=208.80.152.118, resp_p=80/tcp], j4u32Pc5bif +[orig_h=141.142.220.118, orig_p=48649/tcp, resp_h=208.80.152.118, resp_p=80/tcp], j4u32Pc5bif +[orig_h=141.142.220.118, orig_p=43927/udp, resp_h=141.142.2.2, resp_p=53/udp], TEfuqmmG4bh +[orig_h=141.142.220.118, orig_p=37676/udp, resp_h=141.142.2.2, resp_p=53/udp], FrJExwHcSal +[orig_h=141.142.220.118, orig_p=40526/udp, resp_h=141.142.2.2, resp_p=53/udp], 5OKnoww6xl4 +[orig_h=141.142.220.118, orig_p=49996/tcp, resp_h=208.80.152.3, resp_p=80/tcp], 3PKsZ2Uye21 +[orig_h=141.142.220.118, orig_p=49997/tcp, resp_h=208.80.152.3, resp_p=80/tcp], VW0XPVINV8a +[orig_h=141.142.220.118, orig_p=32902/udp, resp_h=141.142.2.2, resp_p=53/udp], fRFu0wcOle6 +[orig_h=141.142.220.118, orig_p=59816/udp, resp_h=141.142.2.2, resp_p=53/udp], qSsw6ESzHV4 +[orig_h=141.142.220.118, orig_p=59714/udp, resp_h=141.142.2.2, resp_p=53/udp], iE6yhOq3SF +[orig_h=141.142.220.118, orig_p=49998/tcp, resp_h=208.80.152.3, resp_p=80/tcp], GSxOnSLghOa +[orig_h=141.142.220.118, orig_p=58206/udp, resp_h=141.142.2.2, resp_p=53/udp], qCaWGmzFtM5 +[orig_h=141.142.220.118, orig_p=38911/udp, resp_h=141.142.2.2, resp_p=53/udp], 70MGiRM1Qf4 +[orig_h=141.142.220.118, orig_p=59746/udp, resp_h=141.142.2.2, resp_p=53/udp], h5DsfNtYzi1 +[orig_h=141.142.220.118, orig_p=49999/tcp, resp_h=208.80.152.3, resp_p=80/tcp], P654jzLoe3a +[orig_h=141.142.220.118, orig_p=50000/tcp, resp_h=208.80.152.3, resp_p=80/tcp], Tw8jXtpTGu6 +[orig_h=141.142.220.118, orig_p=45000/udp, resp_h=141.142.2.2, resp_p=53/udp], c4Zw9TmAE05 +[orig_h=141.142.220.118, orig_p=48479/udp, resp_h=141.142.2.2, resp_p=53/udp], EAr0uf4mhq +[orig_h=141.142.220.118, orig_p=48128/udp, resp_h=141.142.2.2, resp_p=53/udp], GvmoxJFXdTa +[orig_h=141.142.220.118, orig_p=50001/tcp, resp_h=208.80.152.3, resp_p=80/tcp], 0Q4FH8sESw5 +[orig_h=141.142.220.118, orig_p=56056/udp, resp_h=141.142.2.2, resp_p=53/udp], slFea8xwSmb +[orig_h=141.142.220.118, orig_p=55092/udp, resp_h=141.142.2.2, resp_p=53/udp], UfGkYA2HI2g +[orig_h=141.142.220.118, orig_p=35642/tcp, resp_h=208.80.152.2, resp_p=80/tcp], i2rO3KD1Syg +[orig_h=141.142.220.118, orig_p=49997/tcp, resp_h=208.80.152.3, resp_p=80/tcp], VW0XPVINV8a +[orig_h=141.142.220.118, orig_p=49996/tcp, resp_h=208.80.152.3, resp_p=80/tcp], 3PKsZ2Uye21 +[orig_h=141.142.220.118, orig_p=49998/tcp, resp_h=208.80.152.3, resp_p=80/tcp], GSxOnSLghOa +[orig_h=141.142.220.118, orig_p=50000/tcp, resp_h=208.80.152.3, resp_p=80/tcp], Tw8jXtpTGu6 +[orig_h=141.142.220.118, orig_p=49999/tcp, resp_h=208.80.152.3, resp_p=80/tcp], P654jzLoe3a +[orig_h=141.142.220.118, orig_p=50001/tcp, resp_h=208.80.152.3, resp_p=80/tcp], 0Q4FH8sESw5 +[orig_h=141.142.220.118, orig_p=35642/tcp, resp_h=208.80.152.2, resp_p=80/tcp], i2rO3KD1Syg +[orig_h=141.142.220.235, orig_p=6705/tcp, resp_h=173.192.163.128, resp_p=80/tcp], 2cx26uAvUPl +[orig_h=141.142.220.235, orig_p=6705/tcp, resp_h=173.192.163.128, resp_p=80/tcp], 2cx26uAvUPl +[orig_h=141.142.220.44, orig_p=5353/udp, resp_h=224.0.0.251, resp_p=5353/udp], BWaU4aSuwkc +[orig_h=141.142.220.226, orig_p=137/udp, resp_h=141.142.220.255, resp_p=137/udp], 10XodEwRycf +[orig_h=fe80::3074:17d5:2052:c324, orig_p=65373/udp, resp_h=ff02::1:3, resp_p=5355/udp], zno26fFZkrh +[orig_h=141.142.220.226, orig_p=55131/udp, resp_h=224.0.0.252, resp_p=5355/udp], v5rgkJBig5l +[orig_h=fe80::3074:17d5:2052:c324, orig_p=54213/udp, resp_h=ff02::1:3, resp_p=5355/udp], eWZCH7OONC1 +[orig_h=141.142.220.226, orig_p=55671/udp, resp_h=224.0.0.252, resp_p=5355/udp], 0Pwk3ntf8O3 +[orig_h=141.142.220.238, orig_p=56641/udp, resp_h=141.142.220.255, resp_p=137/udp], 0HKorjr8Zp7 diff --git a/testing/btest/Baseline/core.conn-uid/output.cc b/testing/btest/Baseline/core.conn-uid/output.cc deleted file mode 100644 index 6f58e7c10c..0000000000 --- a/testing/btest/Baseline/core.conn-uid/output.cc +++ /dev/null @@ -1,40 +0,0 @@ -[orig_h=141.142.220.202, orig_p=5353/udp, resp_h=224.0.0.251, resp_p=5353/udp], UWkUyAuUGXf -[orig_h=141.142.220.50, orig_p=5353/udp, resp_h=224.0.0.251, resp_p=5353/udp], arKYeMETxOg -[orig_h=141.142.220.118, orig_p=35634/tcp, resp_h=208.80.152.2, resp_p=80/tcp], k6kgXLOoSKl -[orig_h=141.142.220.118, orig_p=48649/tcp, resp_h=208.80.152.118, resp_p=80/tcp], nQcgTWjvg4c -[orig_h=141.142.220.118, orig_p=48649/tcp, resp_h=208.80.152.118, resp_p=80/tcp], nQcgTWjvg4c -[orig_h=141.142.220.118, orig_p=43927/udp, resp_h=141.142.2.2, resp_p=53/udp], j4u32Pc5bif -[orig_h=141.142.220.118, orig_p=37676/udp, resp_h=141.142.2.2, resp_p=53/udp], TEfuqmmG4bh -[orig_h=141.142.220.118, orig_p=40526/udp, resp_h=141.142.2.2, resp_p=53/udp], FrJExwHcSal -[orig_h=141.142.220.118, orig_p=49996/tcp, resp_h=208.80.152.3, resp_p=80/tcp], 5OKnoww6xl4 -[orig_h=141.142.220.118, orig_p=49997/tcp, resp_h=208.80.152.3, resp_p=80/tcp], 3PKsZ2Uye21 -[orig_h=141.142.220.118, orig_p=32902/udp, resp_h=141.142.2.2, resp_p=53/udp], VW0XPVINV8a -[orig_h=141.142.220.118, orig_p=59816/udp, resp_h=141.142.2.2, resp_p=53/udp], fRFu0wcOle6 -[orig_h=141.142.220.118, orig_p=59714/udp, resp_h=141.142.2.2, resp_p=53/udp], qSsw6ESzHV4 -[orig_h=141.142.220.118, orig_p=49998/tcp, resp_h=208.80.152.3, resp_p=80/tcp], iE6yhOq3SF -[orig_h=141.142.220.118, orig_p=58206/udp, resp_h=141.142.2.2, resp_p=53/udp], GSxOnSLghOa -[orig_h=141.142.220.118, orig_p=38911/udp, resp_h=141.142.2.2, resp_p=53/udp], qCaWGmzFtM5 -[orig_h=141.142.220.118, orig_p=59746/udp, resp_h=141.142.2.2, resp_p=53/udp], 70MGiRM1Qf4 -[orig_h=141.142.220.118, orig_p=49999/tcp, resp_h=208.80.152.3, resp_p=80/tcp], h5DsfNtYzi1 -[orig_h=141.142.220.118, orig_p=50000/tcp, resp_h=208.80.152.3, resp_p=80/tcp], P654jzLoe3a -[orig_h=141.142.220.118, orig_p=45000/udp, resp_h=141.142.2.2, resp_p=53/udp], Tw8jXtpTGu6 -[orig_h=141.142.220.118, orig_p=48479/udp, resp_h=141.142.2.2, resp_p=53/udp], c4Zw9TmAE05 -[orig_h=141.142.220.118, orig_p=48128/udp, resp_h=141.142.2.2, resp_p=53/udp], EAr0uf4mhq -[orig_h=141.142.220.118, orig_p=50001/tcp, resp_h=208.80.152.3, resp_p=80/tcp], GvmoxJFXdTa -[orig_h=141.142.220.118, orig_p=56056/udp, resp_h=141.142.2.2, resp_p=53/udp], 0Q4FH8sESw5 -[orig_h=141.142.220.118, orig_p=55092/udp, resp_h=141.142.2.2, resp_p=53/udp], slFea8xwSmb -[orig_h=141.142.220.118, orig_p=35642/tcp, resp_h=208.80.152.2, resp_p=80/tcp], UfGkYA2HI2g -[orig_h=141.142.220.118, orig_p=49997/tcp, resp_h=208.80.152.3, resp_p=80/tcp], 3PKsZ2Uye21 -[orig_h=141.142.220.118, orig_p=49996/tcp, resp_h=208.80.152.3, resp_p=80/tcp], 5OKnoww6xl4 -[orig_h=141.142.220.118, orig_p=49998/tcp, resp_h=208.80.152.3, resp_p=80/tcp], iE6yhOq3SF -[orig_h=141.142.220.118, orig_p=50000/tcp, resp_h=208.80.152.3, resp_p=80/tcp], P654jzLoe3a -[orig_h=141.142.220.118, orig_p=49999/tcp, resp_h=208.80.152.3, resp_p=80/tcp], h5DsfNtYzi1 -[orig_h=141.142.220.118, orig_p=50001/tcp, resp_h=208.80.152.3, resp_p=80/tcp], GvmoxJFXdTa -[orig_h=141.142.220.118, orig_p=35642/tcp, resp_h=208.80.152.2, resp_p=80/tcp], UfGkYA2HI2g -[orig_h=141.142.220.235, orig_p=6705/tcp, resp_h=173.192.163.128, resp_p=80/tcp], i2rO3KD1Syg -[orig_h=141.142.220.235, orig_p=6705/tcp, resp_h=173.192.163.128, resp_p=80/tcp], i2rO3KD1Syg -[orig_h=141.142.220.44, orig_p=5353/udp, resp_h=224.0.0.251, resp_p=5353/udp], 2cx26uAvUPl -[orig_h=141.142.220.226, orig_p=137/udp, resp_h=141.142.220.255, resp_p=137/udp], BWaU4aSuwkc -[orig_h=141.142.220.226, orig_p=55131/udp, resp_h=224.0.0.252, resp_p=5355/udp], 10XodEwRycf -[orig_h=141.142.220.226, orig_p=55671/udp, resp_h=224.0.0.252, resp_p=5355/udp], zno26fFZkrh -[orig_h=141.142.220.238, orig_p=56641/udp, resp_h=141.142.220.255, resp_p=137/udp], v5rgkJBig5l diff --git a/testing/btest/Baseline/core.conn-uid/output.cc2 b/testing/btest/Baseline/core.conn-uid/output.cc2 deleted file mode 100644 index 6f58e7c10c..0000000000 --- a/testing/btest/Baseline/core.conn-uid/output.cc2 +++ /dev/null @@ -1,40 +0,0 @@ -[orig_h=141.142.220.202, orig_p=5353/udp, resp_h=224.0.0.251, resp_p=5353/udp], UWkUyAuUGXf -[orig_h=141.142.220.50, orig_p=5353/udp, resp_h=224.0.0.251, resp_p=5353/udp], arKYeMETxOg -[orig_h=141.142.220.118, orig_p=35634/tcp, resp_h=208.80.152.2, resp_p=80/tcp], k6kgXLOoSKl -[orig_h=141.142.220.118, orig_p=48649/tcp, resp_h=208.80.152.118, resp_p=80/tcp], nQcgTWjvg4c -[orig_h=141.142.220.118, orig_p=48649/tcp, resp_h=208.80.152.118, resp_p=80/tcp], nQcgTWjvg4c -[orig_h=141.142.220.118, orig_p=43927/udp, resp_h=141.142.2.2, resp_p=53/udp], j4u32Pc5bif -[orig_h=141.142.220.118, orig_p=37676/udp, resp_h=141.142.2.2, resp_p=53/udp], TEfuqmmG4bh -[orig_h=141.142.220.118, orig_p=40526/udp, resp_h=141.142.2.2, resp_p=53/udp], FrJExwHcSal -[orig_h=141.142.220.118, orig_p=49996/tcp, resp_h=208.80.152.3, resp_p=80/tcp], 5OKnoww6xl4 -[orig_h=141.142.220.118, orig_p=49997/tcp, resp_h=208.80.152.3, resp_p=80/tcp], 3PKsZ2Uye21 -[orig_h=141.142.220.118, orig_p=32902/udp, resp_h=141.142.2.2, resp_p=53/udp], VW0XPVINV8a -[orig_h=141.142.220.118, orig_p=59816/udp, resp_h=141.142.2.2, resp_p=53/udp], fRFu0wcOle6 -[orig_h=141.142.220.118, orig_p=59714/udp, resp_h=141.142.2.2, resp_p=53/udp], qSsw6ESzHV4 -[orig_h=141.142.220.118, orig_p=49998/tcp, resp_h=208.80.152.3, resp_p=80/tcp], iE6yhOq3SF -[orig_h=141.142.220.118, orig_p=58206/udp, resp_h=141.142.2.2, resp_p=53/udp], GSxOnSLghOa -[orig_h=141.142.220.118, orig_p=38911/udp, resp_h=141.142.2.2, resp_p=53/udp], qCaWGmzFtM5 -[orig_h=141.142.220.118, orig_p=59746/udp, resp_h=141.142.2.2, resp_p=53/udp], 70MGiRM1Qf4 -[orig_h=141.142.220.118, orig_p=49999/tcp, resp_h=208.80.152.3, resp_p=80/tcp], h5DsfNtYzi1 -[orig_h=141.142.220.118, orig_p=50000/tcp, resp_h=208.80.152.3, resp_p=80/tcp], P654jzLoe3a -[orig_h=141.142.220.118, orig_p=45000/udp, resp_h=141.142.2.2, resp_p=53/udp], Tw8jXtpTGu6 -[orig_h=141.142.220.118, orig_p=48479/udp, resp_h=141.142.2.2, resp_p=53/udp], c4Zw9TmAE05 -[orig_h=141.142.220.118, orig_p=48128/udp, resp_h=141.142.2.2, resp_p=53/udp], EAr0uf4mhq -[orig_h=141.142.220.118, orig_p=50001/tcp, resp_h=208.80.152.3, resp_p=80/tcp], GvmoxJFXdTa -[orig_h=141.142.220.118, orig_p=56056/udp, resp_h=141.142.2.2, resp_p=53/udp], 0Q4FH8sESw5 -[orig_h=141.142.220.118, orig_p=55092/udp, resp_h=141.142.2.2, resp_p=53/udp], slFea8xwSmb -[orig_h=141.142.220.118, orig_p=35642/tcp, resp_h=208.80.152.2, resp_p=80/tcp], UfGkYA2HI2g -[orig_h=141.142.220.118, orig_p=49997/tcp, resp_h=208.80.152.3, resp_p=80/tcp], 3PKsZ2Uye21 -[orig_h=141.142.220.118, orig_p=49996/tcp, resp_h=208.80.152.3, resp_p=80/tcp], 5OKnoww6xl4 -[orig_h=141.142.220.118, orig_p=49998/tcp, resp_h=208.80.152.3, resp_p=80/tcp], iE6yhOq3SF -[orig_h=141.142.220.118, orig_p=50000/tcp, resp_h=208.80.152.3, resp_p=80/tcp], P654jzLoe3a -[orig_h=141.142.220.118, orig_p=49999/tcp, resp_h=208.80.152.3, resp_p=80/tcp], h5DsfNtYzi1 -[orig_h=141.142.220.118, orig_p=50001/tcp, resp_h=208.80.152.3, resp_p=80/tcp], GvmoxJFXdTa -[orig_h=141.142.220.118, orig_p=35642/tcp, resp_h=208.80.152.2, resp_p=80/tcp], UfGkYA2HI2g -[orig_h=141.142.220.235, orig_p=6705/tcp, resp_h=173.192.163.128, resp_p=80/tcp], i2rO3KD1Syg -[orig_h=141.142.220.235, orig_p=6705/tcp, resp_h=173.192.163.128, resp_p=80/tcp], i2rO3KD1Syg -[orig_h=141.142.220.44, orig_p=5353/udp, resp_h=224.0.0.251, resp_p=5353/udp], 2cx26uAvUPl -[orig_h=141.142.220.226, orig_p=137/udp, resp_h=141.142.220.255, resp_p=137/udp], BWaU4aSuwkc -[orig_h=141.142.220.226, orig_p=55131/udp, resp_h=224.0.0.252, resp_p=5355/udp], 10XodEwRycf -[orig_h=141.142.220.226, orig_p=55671/udp, resp_h=224.0.0.252, resp_p=5355/udp], zno26fFZkrh -[orig_h=141.142.220.238, orig_p=56641/udp, resp_h=141.142.220.255, resp_p=137/udp], v5rgkJBig5l diff --git a/testing/btest/Baseline/core.disable-mobile-ipv6/output b/testing/btest/Baseline/core.disable-mobile-ipv6/output new file mode 100644 index 0000000000..b156353f74 --- /dev/null +++ b/testing/btest/Baseline/core.disable-mobile-ipv6/output @@ -0,0 +1 @@ +1333663011.602839 weird: unknown_protocol_135 diff --git a/testing/btest/Baseline/core.discarder/output b/testing/btest/Baseline/core.discarder/output new file mode 100644 index 0000000000..82b4b3e622 --- /dev/null +++ b/testing/btest/Baseline/core.discarder/output @@ -0,0 +1,24 @@ +################ IP Discarder ################ +[orig_h=141.142.220.118, orig_p=35634/tcp, resp_h=208.80.152.2, resp_p=80/tcp] +[orig_h=141.142.220.118, orig_p=35634/tcp, resp_h=208.80.152.2, resp_p=80/tcp] +[orig_h=141.142.220.118, orig_p=35642/tcp, resp_h=208.80.152.2, resp_p=80/tcp] +[orig_h=141.142.220.118, orig_p=35642/tcp, resp_h=208.80.152.2, resp_p=80/tcp] +[orig_h=141.142.220.118, orig_p=35642/tcp, resp_h=208.80.152.2, resp_p=80/tcp] +[orig_h=141.142.220.118, orig_p=35642/tcp, resp_h=208.80.152.2, resp_p=80/tcp] +################ TCP Discarder ################ +[orig_h=141.142.220.118, orig_p=48649/tcp, resp_h=208.80.152.118, resp_p=80/tcp] +[orig_h=141.142.220.118, orig_p=49996/tcp, resp_h=208.80.152.3, resp_p=80/tcp] +[orig_h=141.142.220.118, orig_p=49997/tcp, resp_h=208.80.152.3, resp_p=80/tcp] +[orig_h=141.142.220.118, orig_p=49998/tcp, resp_h=208.80.152.3, resp_p=80/tcp] +[orig_h=141.142.220.118, orig_p=49999/tcp, resp_h=208.80.152.3, resp_p=80/tcp] +[orig_h=141.142.220.118, orig_p=50000/tcp, resp_h=208.80.152.3, resp_p=80/tcp] +[orig_h=141.142.220.118, orig_p=50001/tcp, resp_h=208.80.152.3, resp_p=80/tcp] +[orig_h=141.142.220.118, orig_p=35642/tcp, resp_h=208.80.152.2, resp_p=80/tcp] +################ UDP Discarder ################ +[orig_h=fe80::217:f2ff:fed7:cf65, orig_p=5353/udp, resp_h=ff02::fb, resp_p=5353/udp] +[orig_h=fe80::3074:17d5:2052:c324, orig_p=65373/udp, resp_h=ff02::1:3, resp_p=5355/udp] +[orig_h=fe80::3074:17d5:2052:c324, orig_p=65373/udp, resp_h=ff02::1:3, resp_p=5355/udp] +[orig_h=fe80::3074:17d5:2052:c324, orig_p=54213/udp, resp_h=ff02::1:3, resp_p=5355/udp] +[orig_h=fe80::3074:17d5:2052:c324, orig_p=54213/udp, resp_h=ff02::1:3, resp_p=5355/udp] +################ ICMP Discarder ################ +Discard icmp packet: [icmp_type=3] diff --git a/testing/btest/Baseline/core.icmp.icmp-context/output b/testing/btest/Baseline/core.icmp.icmp-context/output new file mode 100644 index 0000000000..40dc778d8b --- /dev/null +++ b/testing/btest/Baseline/core.icmp.icmp-context/output @@ -0,0 +1,12 @@ +icmp_unreachable (code=0) + conn_id: [orig_h=10.0.0.1, orig_p=3/icmp, resp_h=10.0.0.2, resp_p=0/icmp] + icmp_conn: [orig_h=10.0.0.1, resp_h=10.0.0.2, itype=3, icode=0, len=0, hlim=64, v6=F] + icmp_context: [id=[orig_h=::, orig_p=0/unknown, resp_h=::, resp_p=0/unknown], len=0, proto=0, frag_offset=0, bad_hdr_len=T, bad_checksum=F, MF=F, DF=F] +icmp_unreachable (code=0) + conn_id: [orig_h=10.0.0.1, orig_p=3/icmp, resp_h=10.0.0.2, resp_p=0/icmp] + icmp_conn: [orig_h=10.0.0.1, resp_h=10.0.0.2, itype=3, icode=0, len=20, hlim=64, v6=F] + icmp_context: [id=[orig_h=10.0.0.2, orig_p=0/unknown, resp_h=10.0.0.1, resp_p=0/unknown], len=20, proto=0, frag_offset=0, bad_hdr_len=T, bad_checksum=F, MF=F, DF=F] +icmp_unreachable (code=3) + conn_id: [orig_h=192.168.1.102, orig_p=3/icmp, resp_h=192.168.1.1, resp_p=3/icmp] + icmp_conn: [orig_h=192.168.1.102, resp_h=192.168.1.1, itype=3, icode=3, len=148, hlim=128, v6=F] + icmp_context: [id=[orig_h=192.168.1.1, orig_p=53/udp, resp_h=192.168.1.102, resp_p=59207/udp], len=163, proto=2, frag_offset=0, bad_hdr_len=F, bad_checksum=F, MF=F, DF=F] diff --git a/testing/btest/Baseline/core.icmp.icmp-events/output b/testing/btest/Baseline/core.icmp.icmp-events/output new file mode 100644 index 0000000000..c8c8eb317f --- /dev/null +++ b/testing/btest/Baseline/core.icmp.icmp-events/output @@ -0,0 +1,20 @@ +icmp_unreachable (code=3) + conn_id: [orig_h=192.168.1.102, orig_p=3/icmp, resp_h=192.168.1.1, resp_p=3/icmp] + icmp_conn: [orig_h=192.168.1.102, resp_h=192.168.1.1, itype=3, icode=3, len=148, hlim=128, v6=F] + icmp_context: [id=[orig_h=192.168.1.1, orig_p=53/udp, resp_h=192.168.1.102, resp_p=59207/udp], len=163, proto=2, frag_offset=0, bad_hdr_len=F, bad_checksum=F, MF=F, DF=F] +icmp_time_exceeded (code=0) + conn_id: [orig_h=10.0.0.1, orig_p=11/icmp, resp_h=10.0.0.2, resp_p=0/icmp] + icmp_conn: [orig_h=10.0.0.1, resp_h=10.0.0.2, itype=11, icode=0, len=32, hlim=64, v6=F] + icmp_context: [id=[orig_h=10.0.0.2, orig_p=30000/udp, resp_h=10.0.0.1, resp_p=13000/udp], len=32, proto=2, frag_offset=0, bad_hdr_len=F, bad_checksum=F, MF=F, DF=F] +icmp_echo_request (id=34844, seq=0, payload=O\x85\xe0C\0^N\xeb\xff^H^I^J^K^L^M^N^O^P^Q^R^S^T^U^V^W^X^Y^Z\x1b\x1c\x1d\x1e\x1f !"#$%&'()*+,-./01234567) + conn_id: [orig_h=10.0.0.1, orig_p=8/icmp, resp_h=74.125.225.99, resp_p=0/icmp] + icmp_conn: [orig_h=10.0.0.1, resp_h=74.125.225.99, itype=8, icode=0, len=56, hlim=64, v6=F] +icmp_echo_reply (id=34844, seq=0, payload=O\x85\xe0C\0^N\xeb\xff^H^I^J^K^L^M^N^O^P^Q^R^S^T^U^V^W^X^Y^Z\x1b\x1c\x1d\x1e\x1f !"#$%&'()*+,-./01234567) + conn_id: [orig_h=10.0.0.1, orig_p=8/icmp, resp_h=74.125.225.99, resp_p=0/icmp] + icmp_conn: [orig_h=10.0.0.1, resp_h=74.125.225.99, itype=8, icode=0, len=56, hlim=64, v6=F] +icmp_echo_request (id=34844, seq=1, payload=O\x85\xe0D\0^N\xf0}^H^I^J^K^L^M^N^O^P^Q^R^S^T^U^V^W^X^Y^Z\x1b\x1c\x1d\x1e\x1f !"#$%&'()*+,-./01234567) + conn_id: [orig_h=10.0.0.1, orig_p=8/icmp, resp_h=74.125.225.99, resp_p=0/icmp] + icmp_conn: [orig_h=10.0.0.1, resp_h=74.125.225.99, itype=8, icode=0, len=56, hlim=64, v6=F] +icmp_echo_reply (id=34844, seq=1, payload=O\x85\xe0D\0^N\xf0}^H^I^J^K^L^M^N^O^P^Q^R^S^T^U^V^W^X^Y^Z\x1b\x1c\x1d\x1e\x1f !"#$%&'()*+,-./01234567) + conn_id: [orig_h=10.0.0.1, orig_p=8/icmp, resp_h=74.125.225.99, resp_p=0/icmp] + icmp_conn: [orig_h=10.0.0.1, resp_h=74.125.225.99, itype=8, icode=0, len=56, hlim=64, v6=F] diff --git a/testing/btest/Baseline/core.icmp.icmp6-context/output b/testing/btest/Baseline/core.icmp.icmp6-context/output new file mode 100644 index 0000000000..7a83679018 --- /dev/null +++ b/testing/btest/Baseline/core.icmp.icmp6-context/output @@ -0,0 +1,16 @@ +icmp_unreachable (code=0) + conn_id: [orig_h=fe80::dead, orig_p=1/icmp, resp_h=fe80::beef, resp_p=0/icmp] + icmp_conn: [orig_h=fe80::dead, resp_h=fe80::beef, itype=1, icode=0, len=0, hlim=64, v6=T] + icmp_context: [id=[orig_h=::, orig_p=0/unknown, resp_h=::, resp_p=0/unknown], len=0, proto=0, frag_offset=0, bad_hdr_len=T, bad_checksum=F, MF=F, DF=F] +icmp_unreachable (code=0) + conn_id: [orig_h=fe80::dead, orig_p=1/icmp, resp_h=fe80::beef, resp_p=0/icmp] + icmp_conn: [orig_h=fe80::dead, resp_h=fe80::beef, itype=1, icode=0, len=40, hlim=64, v6=T] + icmp_context: [id=[orig_h=fe80::beef, orig_p=0/unknown, resp_h=fe80::dead, resp_p=0/unknown], len=48, proto=0, frag_offset=0, bad_hdr_len=T, bad_checksum=F, MF=F, DF=F] +icmp_unreachable (code=0) + conn_id: [orig_h=fe80::dead, orig_p=1/icmp, resp_h=fe80::beef, resp_p=0/icmp] + icmp_conn: [orig_h=fe80::dead, resp_h=fe80::beef, itype=1, icode=0, len=60, hlim=64, v6=T] + icmp_context: [id=[orig_h=fe80::beef, orig_p=30000/udp, resp_h=fe80::dead, resp_p=13000/udp], len=60, proto=2, frag_offset=0, bad_hdr_len=F, bad_checksum=F, MF=F, DF=F] +icmp_unreachable (code=0) + conn_id: [orig_h=fe80::dead, orig_p=1/icmp, resp_h=fe80::beef, resp_p=0/icmp] + icmp_conn: [orig_h=fe80::dead, resp_h=fe80::beef, itype=1, icode=0, len=48, hlim=64, v6=T] + icmp_context: [id=[orig_h=fe80::beef, orig_p=0/unknown, resp_h=fe80::dead, resp_p=0/unknown], len=48, proto=0, frag_offset=0, bad_hdr_len=T, bad_checksum=F, MF=F, DF=F] diff --git a/testing/btest/Baseline/core.icmp.icmp6-events/output b/testing/btest/Baseline/core.icmp.icmp6-events/output new file mode 100644 index 0000000000..81075b716a --- /dev/null +++ b/testing/btest/Baseline/core.icmp.icmp6-events/output @@ -0,0 +1,68 @@ +icmp_unreachable (code=0) + conn_id: [orig_h=fe80::dead, orig_p=1/icmp, resp_h=fe80::beef, resp_p=0/icmp] + icmp_conn: [orig_h=fe80::dead, resp_h=fe80::beef, itype=1, icode=0, len=60, hlim=64, v6=T] + icmp_context: [id=[orig_h=fe80::beef, orig_p=30000/udp, resp_h=fe80::dead, resp_p=13000/udp], len=60, proto=2, frag_offset=0, bad_hdr_len=F, bad_checksum=F, MF=F, DF=F] +icmp_packet_too_big (code=0) + conn_id: [orig_h=fe80::dead, orig_p=2/icmp, resp_h=fe80::beef, resp_p=0/icmp] + icmp_conn: [orig_h=fe80::dead, resp_h=fe80::beef, itype=2, icode=0, len=52, hlim=64, v6=T] + icmp_context: [id=[orig_h=fe80::beef, orig_p=30000/udp, resp_h=fe80::dead, resp_p=13000/udp], len=52, proto=2, frag_offset=0, bad_hdr_len=F, bad_checksum=F, MF=F, DF=F] +icmp_time_exceeded (code=0) + conn_id: [orig_h=fe80::dead, orig_p=3/icmp, resp_h=fe80::beef, resp_p=0/icmp] + icmp_conn: [orig_h=fe80::dead, resp_h=fe80::beef, itype=3, icode=0, len=52, hlim=64, v6=T] + icmp_context: [id=[orig_h=fe80::beef, orig_p=30000/udp, resp_h=fe80::dead, resp_p=13000/udp], len=52, proto=2, frag_offset=0, bad_hdr_len=F, bad_checksum=F, MF=F, DF=F] +icmp_parameter_problem (code=0) + conn_id: [orig_h=fe80::dead, orig_p=4/icmp, resp_h=fe80::beef, resp_p=0/icmp] + icmp_conn: [orig_h=fe80::dead, resp_h=fe80::beef, itype=4, icode=0, len=52, hlim=64, v6=T] + icmp_context: [id=[orig_h=fe80::beef, orig_p=30000/udp, resp_h=fe80::dead, resp_p=13000/udp], len=52, proto=2, frag_offset=0, bad_hdr_len=F, bad_checksum=F, MF=F, DF=F] +icmp_echo_request (id=1, seq=3, payload=abcdefghijklmnopqrstuvwabcdefghi) + conn_id: [orig_h=2620:0:e00:400e:d1d:db37:beb:5aac, orig_p=128/icmp, resp_h=2001:4860:8006::63, resp_p=129/icmp] + icmp_conn: [orig_h=2620:0:e00:400e:d1d:db37:beb:5aac, resp_h=2001:4860:8006::63, itype=128, icode=0, len=32, hlim=128, v6=T] +icmp_echo_reply (id=1, seq=3, payload=abcdefghijklmnopqrstuvwabcdefghi) + conn_id: [orig_h=2620:0:e00:400e:d1d:db37:beb:5aac, orig_p=128/icmp, resp_h=2001:4860:8006::63, resp_p=129/icmp] + icmp_conn: [orig_h=2620:0:e00:400e:d1d:db37:beb:5aac, resp_h=2001:4860:8006::63, itype=128, icode=0, len=32, hlim=128, v6=T] +icmp_echo_request (id=1, seq=4, payload=abcdefghijklmnopqrstuvwabcdefghi) + conn_id: [orig_h=2620:0:e00:400e:d1d:db37:beb:5aac, orig_p=128/icmp, resp_h=2001:4860:8006::63, resp_p=129/icmp] + icmp_conn: [orig_h=2620:0:e00:400e:d1d:db37:beb:5aac, resp_h=2001:4860:8006::63, itype=128, icode=0, len=32, hlim=128, v6=T] +icmp_echo_reply (id=1, seq=4, payload=abcdefghijklmnopqrstuvwabcdefghi) + conn_id: [orig_h=2620:0:e00:400e:d1d:db37:beb:5aac, orig_p=128/icmp, resp_h=2001:4860:8006::63, resp_p=129/icmp] + icmp_conn: [orig_h=2620:0:e00:400e:d1d:db37:beb:5aac, resp_h=2001:4860:8006::63, itype=128, icode=0, len=32, hlim=128, v6=T] +icmp_echo_request (id=1, seq=5, payload=abcdefghijklmnopqrstuvwabcdefghi) + conn_id: [orig_h=2620:0:e00:400e:d1d:db37:beb:5aac, orig_p=128/icmp, resp_h=2001:4860:8006::63, resp_p=129/icmp] + icmp_conn: [orig_h=2620:0:e00:400e:d1d:db37:beb:5aac, resp_h=2001:4860:8006::63, itype=128, icode=0, len=32, hlim=128, v6=T] +icmp_echo_reply (id=1, seq=5, payload=abcdefghijklmnopqrstuvwabcdefghi) + conn_id: [orig_h=2620:0:e00:400e:d1d:db37:beb:5aac, orig_p=128/icmp, resp_h=2001:4860:8006::63, resp_p=129/icmp] + icmp_conn: [orig_h=2620:0:e00:400e:d1d:db37:beb:5aac, resp_h=2001:4860:8006::63, itype=128, icode=0, len=32, hlim=128, v6=T] +icmp_echo_request (id=1, seq=6, payload=abcdefghijklmnopqrstuvwabcdefghi) + conn_id: [orig_h=2620:0:e00:400e:d1d:db37:beb:5aac, orig_p=128/icmp, resp_h=2001:4860:8006::63, resp_p=129/icmp] + icmp_conn: [orig_h=2620:0:e00:400e:d1d:db37:beb:5aac, resp_h=2001:4860:8006::63, itype=128, icode=0, len=32, hlim=128, v6=T] +icmp_echo_reply (id=1, seq=6, payload=abcdefghijklmnopqrstuvwabcdefghi) + conn_id: [orig_h=2620:0:e00:400e:d1d:db37:beb:5aac, orig_p=128/icmp, resp_h=2001:4860:8006::63, resp_p=129/icmp] + icmp_conn: [orig_h=2620:0:e00:400e:d1d:db37:beb:5aac, resp_h=2001:4860:8006::63, itype=128, icode=0, len=32, hlim=128, v6=T] +icmp_redirect (tgt=fe80::cafe, dest=fe80::babe) + conn_id: [orig_h=fe80::dead, orig_p=137/icmp, resp_h=fe80::beef, resp_p=0/icmp] + icmp_conn: [orig_h=fe80::dead, resp_h=fe80::beef, itype=137, icode=0, len=32, hlim=255, v6=T] +icmp_router_advertisement + cur_hop_limit=13 + managed=T + other=F + home_agent=T + pref=3 + proxy=F + rsv=0 + router_lifetime=30.0 mins + reachable_time=3.0 secs 700.0 msecs + retrans_timer=1.0 sec 300.0 msecs + conn_id: [orig_h=fe80::dead, orig_p=134/icmp, resp_h=fe80::beef, resp_p=133/icmp] + icmp_conn: [orig_h=fe80::dead, resp_h=fe80::beef, itype=134, icode=0, len=8, hlim=255, v6=T] +icmp_neighbor_advertisement (tgt=fe80::babe) + router=T + solicited=F + override=T + conn_id: [orig_h=fe80::dead, orig_p=136/icmp, resp_h=fe80::beef, resp_p=135/icmp] + icmp_conn: [orig_h=fe80::dead, resp_h=fe80::beef, itype=136, icode=0, len=16, hlim=255, v6=T] +icmp_router_solicitation + conn_id: [orig_h=fe80::dead, orig_p=133/icmp, resp_h=fe80::beef, resp_p=134/icmp] + icmp_conn: [orig_h=fe80::dead, resp_h=fe80::beef, itype=133, icode=0, len=0, hlim=255, v6=T] +icmp_neighbor_solicitation (tgt=fe80::babe) + conn_id: [orig_h=fe80::dead, orig_p=135/icmp, resp_h=fe80::beef, resp_p=136/icmp] + icmp_conn: [orig_h=fe80::dead, resp_h=fe80::beef, itype=135, icode=0, len=16, hlim=255, v6=T] diff --git a/testing/btest/Baseline/core.ipv6-atomic-frag/output b/testing/btest/Baseline/core.ipv6-atomic-frag/output new file mode 100644 index 0000000000..4a628a4bdc --- /dev/null +++ b/testing/btest/Baseline/core.ipv6-atomic-frag/output @@ -0,0 +1,4 @@ +[orig_h=2001:db8:1::2, orig_p=36951/tcp, resp_h=2001:db8:1::1, resp_p=80/tcp] +[orig_h=2001:db8:1::2, orig_p=59694/tcp, resp_h=2001:db8:1::1, resp_p=80/tcp] +[orig_h=2001:db8:1::2, orig_p=27393/tcp, resp_h=2001:db8:1::1, resp_p=80/tcp] +[orig_h=2001:db8:1::2, orig_p=45805/tcp, resp_h=2001:db8:1::1, resp_p=80/tcp] diff --git a/testing/btest/Baseline/core.ipv6-frag/dns.log b/testing/btest/Baseline/core.ipv6-frag/dns.log new file mode 100644 index 0000000000..251f35d789 --- /dev/null +++ b/testing/btest/Baseline/core.ipv6-frag/dns.log @@ -0,0 +1,9 @@ +#separator \x09 +#set_separator , +#empty_field (empty) +#unset_field - +#path dns +#fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p proto trans_id query qclass qclass_name qtype qtype_name rcode rcode_name AA TC RD RA Z answers TTLs +#types time string addr port addr port enum count string count string count string count string bool bool bool bool count vector[string] vector[interval] +1331084278.438444 UWkUyAuUGXf 2001:470:1f11:81f:d138:5f55:6d4:1fe2 51850 2607:f740:b::f93 53 udp 3903 txtpadding_323.n1.netalyzr.icsi.berkeley.edu 1 C_INTERNET 16 TXT 0 NOERROR T F T F 0 This TXT record should be ignored 1.000000 +1331084293.592245 arKYeMETxOg 2001:470:1f11:81f:d138:5f55:6d4:1fe2 51851 2607:f740:b::f93 53 udp 40849 txtpadding_3230.n1.netalyzr.icsi.berkeley.edu 1 C_INTERNET 16 TXT 0 NOERROR T F T F 0 This TXT record should be ignored 1.000000 diff --git a/testing/btest/Baseline/core.ipv6-frag/output b/testing/btest/Baseline/core.ipv6-frag/output new file mode 100644 index 0000000000..12dfc3a841 --- /dev/null +++ b/testing/btest/Baseline/core.ipv6-frag/output @@ -0,0 +1,5 @@ +ip6=[class=0, flow=0, len=81, nxt=17, hlim=64, src=2001:470:1f11:81f:d138:5f55:6d4:1fe2, dst=2607:f740:b::f93, exts=[]], udp = [sport=51850/udp, dport=53/udp, ulen=81] +ip6=[class=0, flow=0, len=331, nxt=17, hlim=53, src=2607:f740:b::f93, dst=2001:470:1f11:81f:d138:5f55:6d4:1fe2, exts=[]], udp = [sport=53/udp, dport=51850/udp, ulen=331] +ip6=[class=0, flow=0, len=82, nxt=17, hlim=64, src=2001:470:1f11:81f:d138:5f55:6d4:1fe2, dst=2607:f740:b::f93, exts=[]], udp = [sport=51851/udp, dport=53/udp, ulen=82] +ip6=[class=0, flow=0, len=82, nxt=17, hlim=64, src=2001:470:1f11:81f:d138:5f55:6d4:1fe2, dst=2607:f740:b::f93, exts=[]], udp = [sport=51851/udp, dport=53/udp, ulen=82] +ip6=[class=0, flow=0, len=3238, nxt=17, hlim=53, src=2607:f740:b::f93, dst=2001:470:1f11:81f:d138:5f55:6d4:1fe2, exts=[]], udp = [sport=53/udp, dport=51851/udp, ulen=3238] diff --git a/testing/btest/Baseline/core.ipv6_esp/output b/testing/btest/Baseline/core.ipv6_esp/output new file mode 100644 index 0000000000..02fb7e154f --- /dev/null +++ b/testing/btest/Baseline/core.ipv6_esp/output @@ -0,0 +1,120 @@ +[class=0, flow=0, len=116, nxt=50, hlim=64, src=3ffe::1, dst=3ffe::2, exts=[[id=50, hopopts=, dstopts=, routing=, fragment=, ah=, esp=[spi=10, seq=1], mobility=]]] +[class=0, flow=0, len=116, nxt=50, hlim=64, src=3ffe::1, dst=3ffe::2, exts=[[id=50, hopopts=, dstopts=, routing=, fragment=, ah=, esp=[spi=10, seq=2], mobility=]]] +[class=0, flow=0, len=116, nxt=50, hlim=64, src=3ffe::1, dst=3ffe::2, exts=[[id=50, hopopts=, dstopts=, routing=, fragment=, ah=, esp=[spi=10, seq=3], mobility=]]] +[class=0, flow=0, len=116, nxt=50, hlim=64, src=3ffe::1, dst=3ffe::2, exts=[[id=50, hopopts=, dstopts=, routing=, fragment=, ah=, esp=[spi=10, seq=4], mobility=]]] +[class=0, flow=0, len=116, nxt=50, hlim=64, src=3ffe::1, dst=3ffe::2, exts=[[id=50, hopopts=, dstopts=, routing=, fragment=, ah=, esp=[spi=10, seq=5], mobility=]]] +[class=0, flow=0, len=116, nxt=50, hlim=64, src=3ffe::1, dst=3ffe::2, exts=[[id=50, hopopts=, dstopts=, routing=, fragment=, ah=, esp=[spi=10, seq=6], mobility=]]] +[class=0, flow=0, len=116, nxt=50, hlim=64, src=3ffe::1, dst=3ffe::2, exts=[[id=50, hopopts=, dstopts=, routing=, fragment=, ah=, esp=[spi=10, seq=7], mobility=]]] +[class=0, flow=0, len=116, nxt=50, hlim=64, src=3ffe::1, dst=3ffe::2, exts=[[id=50, hopopts=, dstopts=, routing=, fragment=, ah=, esp=[spi=10, seq=8], mobility=]]] +[class=0, flow=0, len=116, nxt=50, hlim=64, src=3ffe::1, dst=3ffe::2, exts=[[id=50, hopopts=, dstopts=, routing=, fragment=, ah=, esp=[spi=10, seq=9], mobility=]]] +[class=0, flow=0, len=116, nxt=50, hlim=64, src=3ffe::1, dst=3ffe::2, exts=[[id=50, hopopts=, dstopts=, routing=, fragment=, ah=, esp=[spi=10, seq=10], mobility=]]] +[class=0, flow=0, len=100, nxt=50, hlim=64, src=3ffe::1, dst=3ffe::3, exts=[[id=50, hopopts=, dstopts=, routing=, fragment=, ah=, esp=[spi=11, seq=1], mobility=]]] +[class=0, flow=0, len=100, nxt=50, hlim=64, src=3ffe::1, dst=3ffe::3, exts=[[id=50, hopopts=, dstopts=, routing=, fragment=, ah=, esp=[spi=11, seq=2], mobility=]]] +[class=0, flow=0, len=100, nxt=50, hlim=64, src=3ffe::1, dst=3ffe::3, exts=[[id=50, hopopts=, dstopts=, routing=, fragment=, ah=, esp=[spi=11, seq=3], mobility=]]] +[class=0, flow=0, len=100, nxt=50, hlim=64, src=3ffe::1, dst=3ffe::3, exts=[[id=50, hopopts=, dstopts=, routing=, fragment=, ah=, esp=[spi=11, seq=4], mobility=]]] +[class=0, flow=0, len=100, nxt=50, hlim=64, src=3ffe::1, dst=3ffe::3, exts=[[id=50, hopopts=, dstopts=, routing=, fragment=, ah=, esp=[spi=11, seq=5], mobility=]]] +[class=0, flow=0, len=100, nxt=50, hlim=64, src=3ffe::1, dst=3ffe::3, exts=[[id=50, hopopts=, dstopts=, routing=, fragment=, ah=, esp=[spi=11, seq=6], mobility=]]] +[class=0, flow=0, len=100, nxt=50, hlim=64, src=3ffe::1, dst=3ffe::3, exts=[[id=50, hopopts=, dstopts=, routing=, fragment=, ah=, esp=[spi=11, seq=7], mobility=]]] +[class=0, flow=0, len=100, nxt=50, hlim=64, src=3ffe::1, dst=3ffe::3, exts=[[id=50, hopopts=, dstopts=, routing=, fragment=, ah=, esp=[spi=11, seq=8], mobility=]]] +[class=0, flow=0, len=100, nxt=50, hlim=64, src=3ffe::1, dst=3ffe::3, exts=[[id=50, hopopts=, dstopts=, routing=, fragment=, ah=, esp=[spi=11, seq=9], mobility=]]] +[class=0, flow=0, len=100, nxt=50, hlim=64, src=3ffe::1, dst=3ffe::3, exts=[[id=50, hopopts=, dstopts=, routing=, fragment=, ah=, esp=[spi=11, seq=10], mobility=]]] +[class=0, flow=0, len=100, nxt=50, hlim=64, src=3ffe::1, dst=3ffe::4, exts=[[id=50, hopopts=, dstopts=, routing=, fragment=, ah=, esp=[spi=12, seq=1], mobility=]]] +[class=0, flow=0, len=100, nxt=50, hlim=64, src=3ffe::1, dst=3ffe::4, exts=[[id=50, hopopts=, dstopts=, routing=, fragment=, ah=, esp=[spi=12, seq=2], mobility=]]] +[class=0, flow=0, len=100, nxt=50, hlim=64, src=3ffe::1, dst=3ffe::4, exts=[[id=50, hopopts=, dstopts=, routing=, fragment=, ah=, esp=[spi=12, seq=3], mobility=]]] +[class=0, flow=0, len=100, nxt=50, hlim=64, src=3ffe::1, dst=3ffe::4, exts=[[id=50, hopopts=, dstopts=, routing=, fragment=, ah=, esp=[spi=12, seq=4], mobility=]]] +[class=0, flow=0, len=100, nxt=50, hlim=64, src=3ffe::1, dst=3ffe::4, exts=[[id=50, hopopts=, dstopts=, routing=, fragment=, ah=, esp=[spi=12, seq=5], mobility=]]] +[class=0, flow=0, len=100, nxt=50, hlim=64, src=3ffe::1, dst=3ffe::4, exts=[[id=50, hopopts=, dstopts=, routing=, fragment=, ah=, esp=[spi=12, seq=6], mobility=]]] +[class=0, flow=0, len=100, nxt=50, hlim=64, src=3ffe::1, dst=3ffe::4, exts=[[id=50, hopopts=, dstopts=, routing=, fragment=, ah=, esp=[spi=12, seq=7], mobility=]]] +[class=0, flow=0, len=100, nxt=50, hlim=64, src=3ffe::1, dst=3ffe::4, exts=[[id=50, hopopts=, dstopts=, routing=, fragment=, ah=, esp=[spi=12, seq=8], mobility=]]] +[class=0, flow=0, len=100, nxt=50, hlim=64, src=3ffe::1, dst=3ffe::4, exts=[[id=50, hopopts=, dstopts=, routing=, fragment=, ah=, esp=[spi=12, seq=9], mobility=]]] +[class=0, flow=0, len=100, nxt=50, hlim=64, src=3ffe::1, dst=3ffe::4, exts=[[id=50, hopopts=, dstopts=, routing=, fragment=, ah=, esp=[spi=12, seq=10], mobility=]]] +[class=0, flow=0, len=88, nxt=50, hlim=64, src=3ffe::1, dst=3ffe::5, exts=[[id=50, hopopts=, dstopts=, routing=, fragment=, ah=, esp=[spi=13, seq=1], mobility=]]] +[class=0, flow=0, len=88, nxt=50, hlim=64, src=3ffe::1, dst=3ffe::5, exts=[[id=50, hopopts=, dstopts=, routing=, fragment=, ah=, esp=[spi=13, seq=2], mobility=]]] +[class=0, flow=0, len=88, nxt=50, hlim=64, src=3ffe::1, dst=3ffe::5, exts=[[id=50, hopopts=, dstopts=, routing=, fragment=, ah=, esp=[spi=13, seq=3], mobility=]]] +[class=0, flow=0, len=88, nxt=50, hlim=64, src=3ffe::1, dst=3ffe::5, exts=[[id=50, hopopts=, dstopts=, routing=, fragment=, ah=, esp=[spi=13, seq=4], mobility=]]] +[class=0, flow=0, len=88, nxt=50, hlim=64, src=3ffe::1, dst=3ffe::5, exts=[[id=50, hopopts=, dstopts=, routing=, fragment=, ah=, esp=[spi=13, seq=5], mobility=]]] +[class=0, flow=0, len=88, nxt=50, hlim=64, src=3ffe::1, dst=3ffe::5, exts=[[id=50, hopopts=, dstopts=, routing=, fragment=, ah=, esp=[spi=13, seq=6], mobility=]]] +[class=0, flow=0, len=88, nxt=50, hlim=64, src=3ffe::1, dst=3ffe::5, exts=[[id=50, hopopts=, dstopts=, routing=, fragment=, ah=, esp=[spi=13, seq=7], mobility=]]] +[class=0, flow=0, len=88, nxt=50, hlim=64, src=3ffe::1, dst=3ffe::5, exts=[[id=50, hopopts=, dstopts=, routing=, fragment=, ah=, esp=[spi=13, seq=8], mobility=]]] +[class=0, flow=0, len=88, nxt=50, hlim=64, src=3ffe::1, dst=3ffe::5, exts=[[id=50, hopopts=, dstopts=, routing=, fragment=, ah=, esp=[spi=13, seq=9], mobility=]]] +[class=0, flow=0, len=88, nxt=50, hlim=64, src=3ffe::1, dst=3ffe::5, exts=[[id=50, hopopts=, dstopts=, routing=, fragment=, ah=, esp=[spi=13, seq=10], mobility=]]] +[class=0, flow=0, len=116, nxt=50, hlim=64, src=3ffe::1, dst=3ffe::12, exts=[[id=50, hopopts=, dstopts=, routing=, fragment=, ah=, esp=[spi=10, seq=1], mobility=]]] +[class=0, flow=0, len=116, nxt=50, hlim=64, src=3ffe::1, dst=3ffe::12, exts=[[id=50, hopopts=, dstopts=, routing=, fragment=, ah=, esp=[spi=10, seq=2], mobility=]]] +[class=0, flow=0, len=116, nxt=50, hlim=64, src=3ffe::1, dst=3ffe::12, exts=[[id=50, hopopts=, dstopts=, routing=, fragment=, ah=, esp=[spi=10, seq=3], mobility=]]] +[class=0, flow=0, len=116, nxt=50, hlim=64, src=3ffe::1, dst=3ffe::12, exts=[[id=50, hopopts=, dstopts=, routing=, fragment=, ah=, esp=[spi=10, seq=4], mobility=]]] +[class=0, flow=0, len=116, nxt=50, hlim=64, src=3ffe::1, dst=3ffe::12, exts=[[id=50, hopopts=, dstopts=, routing=, fragment=, ah=, esp=[spi=10, seq=5], mobility=]]] +[class=0, flow=0, len=116, nxt=50, hlim=64, src=3ffe::1, dst=3ffe::12, exts=[[id=50, hopopts=, dstopts=, routing=, fragment=, ah=, esp=[spi=10, seq=6], mobility=]]] +[class=0, flow=0, len=116, nxt=50, hlim=64, src=3ffe::1, dst=3ffe::12, exts=[[id=50, hopopts=, dstopts=, routing=, fragment=, ah=, esp=[spi=10, seq=7], mobility=]]] +[class=0, flow=0, len=116, nxt=50, hlim=64, src=3ffe::1, dst=3ffe::12, exts=[[id=50, hopopts=, dstopts=, routing=, fragment=, ah=, esp=[spi=10, seq=8], mobility=]]] +[class=0, flow=0, len=116, nxt=50, hlim=64, src=3ffe::1, dst=3ffe::12, exts=[[id=50, hopopts=, dstopts=, routing=, fragment=, ah=, esp=[spi=10, seq=9], mobility=]]] +[class=0, flow=0, len=116, nxt=50, hlim=64, src=3ffe::1, dst=3ffe::12, exts=[[id=50, hopopts=, dstopts=, routing=, fragment=, ah=, esp=[spi=10, seq=10], mobility=]]] +[class=0, flow=0, len=100, nxt=50, hlim=64, src=3ffe::1, dst=3ffe::13, exts=[[id=50, hopopts=, dstopts=, routing=, fragment=, ah=, esp=[spi=11, seq=1], mobility=]]] +[class=0, flow=0, len=100, nxt=50, hlim=64, src=3ffe::1, dst=3ffe::13, exts=[[id=50, hopopts=, dstopts=, routing=, fragment=, ah=, esp=[spi=11, seq=2], mobility=]]] +[class=0, flow=0, len=100, nxt=50, hlim=64, src=3ffe::1, dst=3ffe::13, exts=[[id=50, hopopts=, dstopts=, routing=, fragment=, ah=, esp=[spi=11, seq=3], mobility=]]] +[class=0, flow=0, len=100, nxt=50, hlim=64, src=3ffe::1, dst=3ffe::13, exts=[[id=50, hopopts=, dstopts=, routing=, fragment=, ah=, esp=[spi=11, seq=4], mobility=]]] +[class=0, flow=0, len=100, nxt=50, hlim=64, src=3ffe::1, dst=3ffe::13, exts=[[id=50, hopopts=, dstopts=, routing=, fragment=, ah=, esp=[spi=11, seq=5], mobility=]]] +[class=0, flow=0, len=100, nxt=50, hlim=64, src=3ffe::1, dst=3ffe::13, exts=[[id=50, hopopts=, dstopts=, routing=, fragment=, ah=, esp=[spi=11, seq=6], mobility=]]] +[class=0, flow=0, len=100, nxt=50, hlim=64, src=3ffe::1, dst=3ffe::13, exts=[[id=50, hopopts=, dstopts=, routing=, fragment=, ah=, esp=[spi=11, seq=7], mobility=]]] +[class=0, flow=0, len=100, nxt=50, hlim=64, src=3ffe::1, dst=3ffe::13, exts=[[id=50, hopopts=, dstopts=, routing=, fragment=, ah=, esp=[spi=11, seq=8], mobility=]]] +[class=0, flow=0, len=100, nxt=50, hlim=64, src=3ffe::1, dst=3ffe::13, exts=[[id=50, hopopts=, dstopts=, routing=, fragment=, ah=, esp=[spi=11, seq=9], mobility=]]] +[class=0, flow=0, len=100, nxt=50, hlim=64, src=3ffe::1, dst=3ffe::13, exts=[[id=50, hopopts=, dstopts=, routing=, fragment=, ah=, esp=[spi=11, seq=10], mobility=]]] +[class=0, flow=0, len=100, nxt=50, hlim=64, src=3ffe::1, dst=3ffe::14, exts=[[id=50, hopopts=, dstopts=, routing=, fragment=, ah=, esp=[spi=12, seq=1], mobility=]]] +[class=0, flow=0, len=100, nxt=50, hlim=64, src=3ffe::1, dst=3ffe::14, exts=[[id=50, hopopts=, dstopts=, routing=, fragment=, ah=, esp=[spi=12, seq=2], mobility=]]] +[class=0, flow=0, len=100, nxt=50, hlim=64, src=3ffe::1, dst=3ffe::14, exts=[[id=50, hopopts=, dstopts=, routing=, fragment=, ah=, esp=[spi=12, seq=3], mobility=]]] +[class=0, flow=0, len=100, nxt=50, hlim=64, src=3ffe::1, dst=3ffe::14, exts=[[id=50, hopopts=, dstopts=, routing=, fragment=, ah=, esp=[spi=12, seq=4], mobility=]]] +[class=0, flow=0, len=100, nxt=50, hlim=64, src=3ffe::1, dst=3ffe::14, exts=[[id=50, hopopts=, dstopts=, routing=, fragment=, ah=, esp=[spi=12, seq=5], mobility=]]] +[class=0, flow=0, len=100, nxt=50, hlim=64, src=3ffe::1, dst=3ffe::14, exts=[[id=50, hopopts=, dstopts=, routing=, fragment=, ah=, esp=[spi=12, seq=6], mobility=]]] +[class=0, flow=0, len=100, nxt=50, hlim=64, src=3ffe::1, dst=3ffe::14, exts=[[id=50, hopopts=, dstopts=, routing=, fragment=, ah=, esp=[spi=12, seq=7], mobility=]]] +[class=0, flow=0, len=100, nxt=50, hlim=64, src=3ffe::1, dst=3ffe::14, exts=[[id=50, hopopts=, dstopts=, routing=, fragment=, ah=, esp=[spi=12, seq=8], mobility=]]] +[class=0, flow=0, len=100, nxt=50, hlim=64, src=3ffe::1, dst=3ffe::14, exts=[[id=50, hopopts=, dstopts=, routing=, fragment=, ah=, esp=[spi=12, seq=9], mobility=]]] +[class=0, flow=0, len=100, nxt=50, hlim=64, src=3ffe::1, dst=3ffe::14, exts=[[id=50, hopopts=, dstopts=, routing=, fragment=, ah=, esp=[spi=12, seq=10], mobility=]]] +[class=0, flow=0, len=88, nxt=50, hlim=64, src=3ffe::1, dst=3ffe::15, exts=[[id=50, hopopts=, dstopts=, routing=, fragment=, ah=, esp=[spi=13, seq=1], mobility=]]] +[class=0, flow=0, len=88, nxt=50, hlim=64, src=3ffe::1, dst=3ffe::15, exts=[[id=50, hopopts=, dstopts=, routing=, fragment=, ah=, esp=[spi=13, seq=2], mobility=]]] +[class=0, flow=0, len=88, nxt=50, hlim=64, src=3ffe::1, dst=3ffe::15, exts=[[id=50, hopopts=, dstopts=, routing=, fragment=, ah=, esp=[spi=13, seq=3], mobility=]]] +[class=0, flow=0, len=88, nxt=50, hlim=64, src=3ffe::1, dst=3ffe::15, exts=[[id=50, hopopts=, dstopts=, routing=, fragment=, ah=, esp=[spi=13, seq=4], mobility=]]] +[class=0, flow=0, len=88, nxt=50, hlim=64, src=3ffe::1, dst=3ffe::15, exts=[[id=50, hopopts=, dstopts=, routing=, fragment=, ah=, esp=[spi=13, seq=5], mobility=]]] +[class=0, flow=0, len=88, nxt=50, hlim=64, src=3ffe::1, dst=3ffe::15, exts=[[id=50, hopopts=, dstopts=, routing=, fragment=, ah=, esp=[spi=13, seq=6], mobility=]]] +[class=0, flow=0, len=88, nxt=50, hlim=64, src=3ffe::1, dst=3ffe::15, exts=[[id=50, hopopts=, dstopts=, routing=, fragment=, ah=, esp=[spi=13, seq=7], mobility=]]] +[class=0, flow=0, len=88, nxt=50, hlim=64, src=3ffe::1, dst=3ffe::15, exts=[[id=50, hopopts=, dstopts=, routing=, fragment=, ah=, esp=[spi=13, seq=8], mobility=]]] +[class=0, flow=0, len=88, nxt=50, hlim=64, src=3ffe::1, dst=3ffe::15, exts=[[id=50, hopopts=, dstopts=, routing=, fragment=, ah=, esp=[spi=13, seq=9], mobility=]]] +[class=0, flow=0, len=88, nxt=50, hlim=64, src=3ffe::1, dst=3ffe::15, exts=[[id=50, hopopts=, dstopts=, routing=, fragment=, ah=, esp=[spi=13, seq=10], mobility=]]] +[class=0, flow=0, len=104, nxt=50, hlim=64, src=3ffe::1, dst=3ffe::22, exts=[[id=50, hopopts=, dstopts=, routing=, fragment=, ah=, esp=[spi=20, seq=1], mobility=]]] +[class=0, flow=0, len=104, nxt=50, hlim=64, src=3ffe::1, dst=3ffe::22, exts=[[id=50, hopopts=, dstopts=, routing=, fragment=, ah=, esp=[spi=20, seq=2], mobility=]]] +[class=0, flow=0, len=104, nxt=50, hlim=64, src=3ffe::1, dst=3ffe::22, exts=[[id=50, hopopts=, dstopts=, routing=, fragment=, ah=, esp=[spi=20, seq=3], mobility=]]] +[class=0, flow=0, len=104, nxt=50, hlim=64, src=3ffe::1, dst=3ffe::22, exts=[[id=50, hopopts=, dstopts=, routing=, fragment=, ah=, esp=[spi=20, seq=4], mobility=]]] +[class=0, flow=0, len=104, nxt=50, hlim=64, src=3ffe::1, dst=3ffe::22, exts=[[id=50, hopopts=, dstopts=, routing=, fragment=, ah=, esp=[spi=20, seq=5], mobility=]]] +[class=0, flow=0, len=104, nxt=50, hlim=64, src=3ffe::1, dst=3ffe::22, exts=[[id=50, hopopts=, dstopts=, routing=, fragment=, ah=, esp=[spi=20, seq=6], mobility=]]] +[class=0, flow=0, len=104, nxt=50, hlim=64, src=3ffe::1, dst=3ffe::22, exts=[[id=50, hopopts=, dstopts=, routing=, fragment=, ah=, esp=[spi=20, seq=7], mobility=]]] +[class=0, flow=0, len=104, nxt=50, hlim=64, src=3ffe::1, dst=3ffe::22, exts=[[id=50, hopopts=, dstopts=, routing=, fragment=, ah=, esp=[spi=20, seq=8], mobility=]]] +[class=0, flow=0, len=104, nxt=50, hlim=64, src=3ffe::1, dst=3ffe::22, exts=[[id=50, hopopts=, dstopts=, routing=, fragment=, ah=, esp=[spi=20, seq=9], mobility=]]] +[class=0, flow=0, len=104, nxt=50, hlim=64, src=3ffe::1, dst=3ffe::22, exts=[[id=50, hopopts=, dstopts=, routing=, fragment=, ah=, esp=[spi=20, seq=10], mobility=]]] +[class=0, flow=0, len=88, nxt=50, hlim=64, src=3ffe::1, dst=3ffe::23, exts=[[id=50, hopopts=, dstopts=, routing=, fragment=, ah=, esp=[spi=21, seq=1], mobility=]]] +[class=0, flow=0, len=88, nxt=50, hlim=64, src=3ffe::1, dst=3ffe::23, exts=[[id=50, hopopts=, dstopts=, routing=, fragment=, ah=, esp=[spi=21, seq=2], mobility=]]] +[class=0, flow=0, len=88, nxt=50, hlim=64, src=3ffe::1, dst=3ffe::23, exts=[[id=50, hopopts=, dstopts=, routing=, fragment=, ah=, esp=[spi=21, seq=3], mobility=]]] +[class=0, flow=0, len=88, nxt=50, hlim=64, src=3ffe::1, dst=3ffe::23, exts=[[id=50, hopopts=, dstopts=, routing=, fragment=, ah=, esp=[spi=21, seq=4], mobility=]]] +[class=0, flow=0, len=88, nxt=50, hlim=64, src=3ffe::1, dst=3ffe::23, exts=[[id=50, hopopts=, dstopts=, routing=, fragment=, ah=, esp=[spi=21, seq=5], mobility=]]] +[class=0, flow=0, len=88, nxt=50, hlim=64, src=3ffe::1, dst=3ffe::23, exts=[[id=50, hopopts=, dstopts=, routing=, fragment=, ah=, esp=[spi=21, seq=6], mobility=]]] +[class=0, flow=0, len=88, nxt=50, hlim=64, src=3ffe::1, dst=3ffe::23, exts=[[id=50, hopopts=, dstopts=, routing=, fragment=, ah=, esp=[spi=21, seq=7], mobility=]]] +[class=0, flow=0, len=88, nxt=50, hlim=64, src=3ffe::1, dst=3ffe::23, exts=[[id=50, hopopts=, dstopts=, routing=, fragment=, ah=, esp=[spi=21, seq=8], mobility=]]] +[class=0, flow=0, len=88, nxt=50, hlim=64, src=3ffe::1, dst=3ffe::23, exts=[[id=50, hopopts=, dstopts=, routing=, fragment=, ah=, esp=[spi=21, seq=9], mobility=]]] +[class=0, flow=0, len=88, nxt=50, hlim=64, src=3ffe::1, dst=3ffe::23, exts=[[id=50, hopopts=, dstopts=, routing=, fragment=, ah=, esp=[spi=21, seq=10], mobility=]]] +[class=0, flow=0, len=88, nxt=50, hlim=64, src=3ffe::1, dst=3ffe::24, exts=[[id=50, hopopts=, dstopts=, routing=, fragment=, ah=, esp=[spi=22, seq=1], mobility=]]] +[class=0, flow=0, len=88, nxt=50, hlim=64, src=3ffe::1, dst=3ffe::24, exts=[[id=50, hopopts=, dstopts=, routing=, fragment=, ah=, esp=[spi=22, seq=2], mobility=]]] +[class=0, flow=0, len=88, nxt=50, hlim=64, src=3ffe::1, dst=3ffe::24, exts=[[id=50, hopopts=, dstopts=, routing=, fragment=, ah=, esp=[spi=22, seq=3], mobility=]]] +[class=0, flow=0, len=88, nxt=50, hlim=64, src=3ffe::1, dst=3ffe::24, exts=[[id=50, hopopts=, dstopts=, routing=, fragment=, ah=, esp=[spi=22, seq=4], mobility=]]] +[class=0, flow=0, len=88, nxt=50, hlim=64, src=3ffe::1, dst=3ffe::24, exts=[[id=50, hopopts=, dstopts=, routing=, fragment=, ah=, esp=[spi=22, seq=5], mobility=]]] +[class=0, flow=0, len=88, nxt=50, hlim=64, src=3ffe::1, dst=3ffe::24, exts=[[id=50, hopopts=, dstopts=, routing=, fragment=, ah=, esp=[spi=22, seq=6], mobility=]]] +[class=0, flow=0, len=88, nxt=50, hlim=64, src=3ffe::1, dst=3ffe::24, exts=[[id=50, hopopts=, dstopts=, routing=, fragment=, ah=, esp=[spi=22, seq=7], mobility=]]] +[class=0, flow=0, len=88, nxt=50, hlim=64, src=3ffe::1, dst=3ffe::24, exts=[[id=50, hopopts=, dstopts=, routing=, fragment=, ah=, esp=[spi=22, seq=8], mobility=]]] +[class=0, flow=0, len=88, nxt=50, hlim=64, src=3ffe::1, dst=3ffe::24, exts=[[id=50, hopopts=, dstopts=, routing=, fragment=, ah=, esp=[spi=22, seq=9], mobility=]]] +[class=0, flow=0, len=88, nxt=50, hlim=64, src=3ffe::1, dst=3ffe::24, exts=[[id=50, hopopts=, dstopts=, routing=, fragment=, ah=, esp=[spi=22, seq=10], mobility=]]] +[class=0, flow=0, len=76, nxt=50, hlim=64, src=3ffe::1, dst=3ffe::25, exts=[[id=50, hopopts=, dstopts=, routing=, fragment=, ah=, esp=[spi=23, seq=1], mobility=]]] +[class=0, flow=0, len=76, nxt=50, hlim=64, src=3ffe::1, dst=3ffe::25, exts=[[id=50, hopopts=, dstopts=, routing=, fragment=, ah=, esp=[spi=23, seq=2], mobility=]]] +[class=0, flow=0, len=76, nxt=50, hlim=64, src=3ffe::1, dst=3ffe::25, exts=[[id=50, hopopts=, dstopts=, routing=, fragment=, ah=, esp=[spi=23, seq=3], mobility=]]] +[class=0, flow=0, len=76, nxt=50, hlim=64, src=3ffe::1, dst=3ffe::25, exts=[[id=50, hopopts=, dstopts=, routing=, fragment=, ah=, esp=[spi=23, seq=4], mobility=]]] +[class=0, flow=0, len=76, nxt=50, hlim=64, src=3ffe::1, dst=3ffe::25, exts=[[id=50, hopopts=, dstopts=, routing=, fragment=, ah=, esp=[spi=23, seq=5], mobility=]]] +[class=0, flow=0, len=76, nxt=50, hlim=64, src=3ffe::1, dst=3ffe::25, exts=[[id=50, hopopts=, dstopts=, routing=, fragment=, ah=, esp=[spi=23, seq=6], mobility=]]] +[class=0, flow=0, len=76, nxt=50, hlim=64, src=3ffe::1, dst=3ffe::25, exts=[[id=50, hopopts=, dstopts=, routing=, fragment=, ah=, esp=[spi=23, seq=7], mobility=]]] +[class=0, flow=0, len=76, nxt=50, hlim=64, src=3ffe::1, dst=3ffe::25, exts=[[id=50, hopopts=, dstopts=, routing=, fragment=, ah=, esp=[spi=23, seq=8], mobility=]]] +[class=0, flow=0, len=76, nxt=50, hlim=64, src=3ffe::1, dst=3ffe::25, exts=[[id=50, hopopts=, dstopts=, routing=, fragment=, ah=, esp=[spi=23, seq=9], mobility=]]] +[class=0, flow=0, len=76, nxt=50, hlim=64, src=3ffe::1, dst=3ffe::25, exts=[[id=50, hopopts=, dstopts=, routing=, fragment=, ah=, esp=[spi=23, seq=10], mobility=]]] diff --git a/testing/btest/Baseline/core.ipv6_ext_headers/output b/testing/btest/Baseline/core.ipv6_ext_headers/output new file mode 100644 index 0000000000..b4cd249371 --- /dev/null +++ b/testing/btest/Baseline/core.ipv6_ext_headers/output @@ -0,0 +1,3 @@ +weird routing0_hdr from 2001:4f8:4:7:2e0:81ff:fe52:ffff to 2001:78:1:32::2 +[orig_h=2001:4f8:4:7:2e0:81ff:fe52:ffff, orig_p=53/udp, resp_h=2001:78:1:32::2, resp_p=53/udp] +[ip=, ip6=[class=0, flow=0, len=59, nxt=0, hlim=64, src=2001:4f8:4:7:2e0:81ff:fe52:ffff, dst=2001:4f8:4:7:2e0:81ff:fe52:9a6b, exts=[[id=0, hopopts=[nxt=43, len=0, options=[[otype=1, len=4, data=\0\0\0\0]]], dstopts=, routing=, fragment=, ah=, esp=, mobility=], [id=43, hopopts=, dstopts=, routing=[nxt=17, len=4, rtype=0, segleft=2, data=\0\0\0\0 ^A\0x\0^A\02\0\0\0\0\0\0\0^A ^A\0x\0^A\02\0\0\0\0\0\0\0^B], fragment=, ah=, esp=, mobility=]]], tcp=, udp=[sport=53/udp, dport=53/udp, ulen=11], icmp=] diff --git a/testing/btest/Baseline/core.leaks.basic-cluster/manager-1.metrics.log b/testing/btest/Baseline/core.leaks.basic-cluster/manager-1.metrics.log new file mode 100644 index 0000000000..42fcd6a526 --- /dev/null +++ b/testing/btest/Baseline/core.leaks.basic-cluster/manager-1.metrics.log @@ -0,0 +1,10 @@ +#separator \x09 +#set_separator , +#empty_field (empty) +#unset_field - +#path metrics +#fields ts metric_id filter_name index.host index.str index.network value +#types time enum string addr string subnet count +1331256494.591966 TEST_METRIC foo-bar 6.5.4.3 - - 4 +1331256494.591966 TEST_METRIC foo-bar 7.2.1.5 - - 2 +1331256494.591966 TEST_METRIC foo-bar 1.2.3.4 - - 6 diff --git a/testing/btest/Baseline/core.leaks.remote/sender.test.failure.log b/testing/btest/Baseline/core.leaks.remote/sender.test.failure.log new file mode 100644 index 0000000000..5a26f322f4 --- /dev/null +++ b/testing/btest/Baseline/core.leaks.remote/sender.test.failure.log @@ -0,0 +1,10 @@ +#separator \x09 +#set_separator , +#empty_field (empty) +#unset_field - +#path test.failure +#fields t id.orig_h id.orig_p id.resp_h id.resp_p status country +#types time addr port addr port string string +1331256472.375609 1.2.3.4 1234 2.3.4.5 80 failure US +1331256472.375609 1.2.3.4 1234 2.3.4.5 80 failure UK +1331256472.375609 1.2.3.4 1234 2.3.4.5 80 failure MX diff --git a/testing/btest/Baseline/core.leaks.remote/sender.test.log b/testing/btest/Baseline/core.leaks.remote/sender.test.log new file mode 100644 index 0000000000..9d2ba26f48 --- /dev/null +++ b/testing/btest/Baseline/core.leaks.remote/sender.test.log @@ -0,0 +1,12 @@ +#separator \x09 +#set_separator , +#empty_field (empty) +#unset_field - +#path test +#fields t id.orig_h id.orig_p id.resp_h id.resp_p status country +#types time addr port addr port string string +1331256472.375609 1.2.3.4 1234 2.3.4.5 80 success unknown +1331256472.375609 1.2.3.4 1234 2.3.4.5 80 failure US +1331256472.375609 1.2.3.4 1234 2.3.4.5 80 failure UK +1331256472.375609 1.2.3.4 1234 2.3.4.5 80 success BR +1331256472.375609 1.2.3.4 1234 2.3.4.5 80 failure MX diff --git a/testing/btest/Baseline/core.leaks.remote/sender.test.success.log b/testing/btest/Baseline/core.leaks.remote/sender.test.success.log new file mode 100644 index 0000000000..1b2ed452a0 --- /dev/null +++ b/testing/btest/Baseline/core.leaks.remote/sender.test.success.log @@ -0,0 +1,9 @@ +#separator \x09 +#set_separator , +#empty_field (empty) +#unset_field - +#path test.success +#fields t id.orig_h id.orig_p id.resp_h id.resp_p status country +#types time addr port addr port string string +1331256472.375609 1.2.3.4 1234 2.3.4.5 80 success unknown +1331256472.375609 1.2.3.4 1234 2.3.4.5 80 success BR diff --git a/testing/btest/Baseline/core.mobile-ipv6-home-addr/output b/testing/btest/Baseline/core.mobile-ipv6-home-addr/output new file mode 100644 index 0000000000..88cbe0cb16 --- /dev/null +++ b/testing/btest/Baseline/core.mobile-ipv6-home-addr/output @@ -0,0 +1,2 @@ +[orig_h=2001:78:1:32::1, orig_p=30000/udp, resp_h=2001:4f8:4:7:2e0:81ff:fe52:9a6b, resp_p=13000/udp] +[ip=, ip6=[class=0, flow=0, len=36, nxt=60, hlim=64, src=2001:4f8:4:7:2e0:81ff:fe52:ffff, dst=2001:4f8:4:7:2e0:81ff:fe52:9a6b, exts=[[id=60, hopopts=, dstopts=[nxt=17, len=2, options=[[otype=1, len=2, data=\0\0], [otype=201, len=16, data= ^A\0x\0^A\02\0\0\0\0\0\0\0^A]]], routing=, fragment=, ah=, esp=, mobility=]]], tcp=, udp=[sport=30000/udp, dport=13000/udp, ulen=12], icmp=] diff --git a/testing/btest/Baseline/core.mobile-ipv6-routing/output b/testing/btest/Baseline/core.mobile-ipv6-routing/output new file mode 100644 index 0000000000..04292caaa7 --- /dev/null +++ b/testing/btest/Baseline/core.mobile-ipv6-routing/output @@ -0,0 +1,2 @@ +[orig_h=2001:4f8:4:7:2e0:81ff:fe52:ffff, orig_p=30000/udp, resp_h=2001:78:1:32::1, resp_p=13000/udp] +[ip=, ip6=[class=0, flow=0, len=36, nxt=43, hlim=64, src=2001:4f8:4:7:2e0:81ff:fe52:ffff, dst=2001:4f8:4:7:2e0:81ff:fe52:9a6b, exts=[[id=43, hopopts=, dstopts=, routing=[nxt=17, len=2, rtype=2, segleft=1, data=\0\0\0\0 ^A\0x\0^A\02\0\0\0\0\0\0\0^A], fragment=, ah=, esp=, mobility=]]], tcp=, udp=[sport=30000/udp, dport=13000/udp, ulen=12], icmp=] diff --git a/testing/btest/Baseline/core.mobility-checksums/bad.out b/testing/btest/Baseline/core.mobility-checksums/bad.out new file mode 100644 index 0000000000..6ea9955402 --- /dev/null +++ b/testing/btest/Baseline/core.mobility-checksums/bad.out @@ -0,0 +1,3 @@ +1333988844.893456 weird: bad_MH_checksum +1333995733.276730 weird: bad_TCP_checksum +1333995701.656496 weird: bad_UDP_checksum diff --git a/testing/btest/Baseline/core.mobility-checksums/good.out b/testing/btest/Baseline/core.mobility-checksums/good.out new file mode 100644 index 0000000000..e69de29bb2 diff --git a/testing/btest/Baseline/core.mobility_msg/output b/testing/btest/Baseline/core.mobility_msg/output new file mode 100644 index 0000000000..6f8d6a1699 --- /dev/null +++ b/testing/btest/Baseline/core.mobility_msg/output @@ -0,0 +1,16 @@ +Binding ACK: +[class=0, flow=0, len=16, nxt=135, hlim=64, src=2001:4f8:4:7:2e0:81ff:fe52:ffff, dst=2001:4f8:4:7:2e0:81ff:fe52:9a6b, exts=[[id=135, hopopts=, dstopts=, routing=, fragment=, ah=, esp=, mobility=[nxt=59, len=1, mh_type=6, rsv=0, chksum=53722, msg=[id=6, brr=, hoti=, coti=, hot=, cot=, bu=, back=[status=0, k=T, seq=42, life=8, options=[[otype=1, len=2, data=\0\0]]], be=]]]]] +Binding Error: +[class=0, flow=0, len=24, nxt=135, hlim=64, src=2001:4f8:4:7:2e0:81ff:fe52:ffff, dst=2001:4f8:4:7:2e0:81ff:fe52:9a6b, exts=[[id=135, hopopts=, dstopts=, routing=, fragment=, ah=, esp=, mobility=[nxt=59, len=2, mh_type=7, rsv=0, chksum=45272, msg=[id=7, brr=, hoti=, coti=, hot=, cot=, bu=, back=, be=[status=1, hoa=2001:78:1:32::1, options=[]]]]]]] +Binding Refresh Request: +[class=0, flow=0, len=8, nxt=135, hlim=64, src=2001:4f8:4:7:2e0:81ff:fe52:ffff, dst=2001:4f8:4:7:2e0:81ff:fe52:9a6b, exts=[[id=135, hopopts=, dstopts=, routing=, fragment=, ah=, esp=, mobility=[nxt=59, len=0, mh_type=0, rsv=0, chksum=55703, msg=[id=0, brr=[rsv=0, options=[]], hoti=, coti=, hot=, cot=, bu=, back=, be=]]]]] +Binding Update: +[class=0, flow=0, len=16, nxt=135, hlim=64, src=2001:4f8:4:7:2e0:81ff:fe52:ffff, dst=2001:4f8:4:7:2e0:81ff:fe52:9a6b, exts=[[id=135, hopopts=, dstopts=, routing=, fragment=, ah=, esp=, mobility=[nxt=59, len=1, mh_type=5, rsv=0, chksum=868, msg=[id=5, brr=, hoti=, coti=, hot=, cot=, bu=[seq=37, a=T, h=T, l=F, k=T, life=3, options=[[otype=1, len=2, data=\0\0]]], back=, be=]]]]] +Care-of Test: +[class=0, flow=0, len=24, nxt=135, hlim=64, src=2001:4f8:4:7:2e0:81ff:fe52:ffff, dst=2001:4f8:4:7:2e0:81ff:fe52:9a6b, exts=[[id=135, hopopts=, dstopts=, routing=, fragment=, ah=, esp=, mobility=[nxt=59, len=2, mh_type=4, rsv=0, chksum=54378, msg=[id=4, brr=, hoti=, coti=, hot=, cot=[nonce_idx=13, cookie=15, token=255, options=[]], bu=, back=, be=]]]]] +Care-of Test Init: +[class=0, flow=0, len=16, nxt=135, hlim=64, src=2001:4f8:4:7:2e0:81ff:fe52:ffff, dst=2001:4f8:4:7:2e0:81ff:fe52:9a6b, exts=[[id=135, hopopts=, dstopts=, routing=, fragment=, ah=, esp=, mobility=[nxt=59, len=1, mh_type=2, rsv=0, chksum=55181, msg=[id=2, brr=, hoti=, coti=[rsv=0, cookie=1, options=[]], hot=, cot=, bu=, back=, be=]]]]] +Home Test: +[class=0, flow=0, len=24, nxt=135, hlim=64, src=2001:4f8:4:7:2e0:81ff:fe52:ffff, dst=2001:4f8:4:7:2e0:81ff:fe52:9a6b, exts=[[id=135, hopopts=, dstopts=, routing=, fragment=, ah=, esp=, mobility=[nxt=59, len=2, mh_type=3, rsv=0, chksum=54634, msg=[id=3, brr=, hoti=, coti=, hot=[nonce_idx=13, cookie=15, token=255, options=[]], cot=, bu=, back=, be=]]]]] +Home Test Init: +[class=0, flow=0, len=16, nxt=135, hlim=64, src=2001:4f8:4:7:2e0:81ff:fe52:ffff, dst=2001:4f8:4:7:2e0:81ff:fe52:9a6b, exts=[[id=135, hopopts=, dstopts=, routing=, fragment=, ah=, esp=, mobility=[nxt=59, len=1, mh_type=1, rsv=0, chksum=55437, msg=[id=1, brr=, hoti=[rsv=0, cookie=1, options=[]], coti=, hot=, cot=, bu=, back=, be=]]]]] diff --git a/testing/btest/Baseline/core.print-bpf-filters-ipv4/output b/testing/btest/Baseline/core.print-bpf-filters-ipv4/output deleted file mode 100644 index d7ff523927..0000000000 --- a/testing/btest/Baseline/core.print-bpf-filters-ipv4/output +++ /dev/null @@ -1,32 +0,0 @@ -#separator \x09 -#set_separator , -#empty_field (empty) -#unset_field - -#path packet_filter -#fields ts node filter init success -#types time string string bool bool -1324314285.981347 - not ip6 T T -#separator \x09 -#set_separator , -#empty_field (empty) -#unset_field - -#path packet_filter -#fields ts node filter init success -#types time string string bool bool -1324314286.168294 - (((((((((((((((((((((((((port 53) or (tcp port 989)) or (tcp port 443)) or (port 6669)) or (udp and port 5353)) or (port 6668)) or (udp and port 5355)) or (tcp port 22)) or (tcp port 995)) or (port 21)) or (tcp port 25 or tcp port 587)) or (port 6667)) or (tcp port 614)) or (tcp port 990)) or (udp port 137)) or (tcp port 993)) or (tcp port 5223)) or (port 514)) or (tcp port 585)) or (tcp port 992)) or (tcp port 563)) or (tcp port 994)) or (tcp port 636)) or (tcp and port (80 or 81 or 631 or 1080 or 3138 or 8000 or 8080 or 8888))) or (port 6666)) and (not ip6) T T -#separator \x09 -#set_separator , -#empty_field (empty) -#unset_field - -#path packet_filter -#fields ts node filter init success -#types time string string bool bool -1324314286.350780 - port 42 T T -#separator \x09 -#set_separator , -#empty_field (empty) -#unset_field - -#path packet_filter -#fields ts node filter init success -#types time string string bool bool -1324314286.530768 - port 56730 T T diff --git a/testing/btest/Baseline/core.print-bpf-filters-ipv6/conn.log b/testing/btest/Baseline/core.print-bpf-filters-ipv6/conn.log deleted file mode 100644 index e71eff9d57..0000000000 --- a/testing/btest/Baseline/core.print-bpf-filters-ipv6/conn.log +++ /dev/null @@ -1,2 +0,0 @@ -# ts uid id.orig_h id.orig_p id.resp_h id.resp_p proto service duration orig_bytes resp_bytes conn_state local_orig missed_bytes history notice_tags -1128727435.4509 UWkUyAuUGXf 141.42.64.125 56730 125.190.109.199 80 tcp - 1.73330307006836 98 9417 SF - 0 ShADdFaf - diff --git a/testing/btest/Baseline/core.print-bpf-filters-ipv6/output b/testing/btest/Baseline/core.print-bpf-filters-ipv6/output deleted file mode 100644 index 0065fabfd3..0000000000 --- a/testing/btest/Baseline/core.print-bpf-filters-ipv6/output +++ /dev/null @@ -1,8 +0,0 @@ -# ts node filter init success -1308603220.46822 - ip or not ip F T -# ts node filter init success -1308603220.51607 - tcp port 22 F T -# ts node filter init success -1308603220.55432 - port 42 F T -# ts node filter init success -1308603220.59452 - port 56730 T T diff --git a/testing/btest/Baseline/core.print-bpf-filters-ipv4/conn.log b/testing/btest/Baseline/core.print-bpf-filters/conn.log similarity index 100% rename from testing/btest/Baseline/core.print-bpf-filters-ipv4/conn.log rename to testing/btest/Baseline/core.print-bpf-filters/conn.log diff --git a/testing/btest/Baseline/core.print-bpf-filters/output b/testing/btest/Baseline/core.print-bpf-filters/output new file mode 100644 index 0000000000..a2bf430fb4 --- /dev/null +++ b/testing/btest/Baseline/core.print-bpf-filters/output @@ -0,0 +1,32 @@ +#separator \x09 +#set_separator , +#empty_field (empty) +#unset_field - +#path packet_filter +#fields ts node filter init success +#types time string string bool bool +1328294052.330721 - ip or not ip T T +#separator \x09 +#set_separator , +#empty_field (empty) +#unset_field - +#path packet_filter +#fields ts node filter init success +#types time string string bool bool +1328294052.542418 - ((((((((((((((((((((((((port 53) or (tcp port 989)) or (tcp port 443)) or (port 6669)) or (udp and port 5353)) or (port 6668)) or (udp and port 5355)) or (tcp port 22)) or (tcp port 995)) or (port 21)) or (tcp port 25 or tcp port 587)) or (port 6667)) or (tcp port 614)) or (tcp port 990)) or (udp port 137)) or (tcp port 993)) or (tcp port 5223)) or (port 514)) or (tcp port 585)) or (tcp port 992)) or (tcp port 563)) or (tcp port 994)) or (tcp port 636)) or (tcp and port (80 or 81 or 631 or 1080 or 3138 or 8000 or 8080 or 8888))) or (port 6666) T T +#separator \x09 +#set_separator , +#empty_field (empty) +#unset_field - +#path packet_filter +#fields ts node filter init success +#types time string string bool bool +1328294052.748480 - port 42 T T +#separator \x09 +#set_separator , +#empty_field (empty) +#unset_field - +#path packet_filter +#fields ts node filter init success +#types time string string bool bool +1328294052.952845 - port 56730 T T diff --git a/testing/btest/Baseline/core.truncation/output b/testing/btest/Baseline/core.truncation/output new file mode 100644 index 0000000000..ba8d3eedee --- /dev/null +++ b/testing/btest/Baseline/core.truncation/output @@ -0,0 +1,3 @@ +1334160095.895421 weird: truncated_IP +1334156241.519125 weird: truncated_IP +1334094648.590126 weird: truncated_IP diff --git a/testing/btest/Baseline/istate.broccoli-ipv6/bro..stdout b/testing/btest/Baseline/istate.broccoli-ipv6/bro..stdout new file mode 100644 index 0000000000..0a7bac52c5 --- /dev/null +++ b/testing/btest/Baseline/istate.broccoli-ipv6/bro..stdout @@ -0,0 +1,9 @@ +handshake done with peer +bro_addr(1.2.3.4) +bro_subnet(10.0.0.0/16) +bro_addr(2607:f8b0:4009:802::1014) +bro_subnet(2607:f8b0::/32) +broccoli_addr(1.2.3.4) +broccoli_subnet(10.0.0.0/16) +broccoli_addr(2607:f8b0:4009:802::1014) +broccoli_subnet(2607:f8b0::/32) diff --git a/testing/btest/Baseline/istate.broccoli-ipv6/broccoli..stdout b/testing/btest/Baseline/istate.broccoli-ipv6/broccoli..stdout new file mode 100644 index 0000000000..481778c98a --- /dev/null +++ b/testing/btest/Baseline/istate.broccoli-ipv6/broccoli..stdout @@ -0,0 +1,6 @@ +Connected to Bro instance at: localhost:47757 +Received bro_addr(1.2.3.4) +Received bro_subnet(10.0.0.0/16) +Received bro_addr(2607:f8b0:4009:802::1014) +Received bro_subnet(2607:f8b0::/32) +Terminating diff --git a/testing/btest/Baseline/istate.pybroccoli/bro..stdout b/testing/btest/Baseline/istate.pybroccoli/bro..stdout index 1e70711932..70ca69dd98 100644 --- a/testing/btest/Baseline/istate.pybroccoli/bro..stdout +++ b/testing/btest/Baseline/istate.pybroccoli/bro..stdout @@ -1,14 +1,16 @@ ==== atomic -10 2 -1313624487.48817 +1330035434.516896 2.0 mins F 1.5 Servus 5555/tcp 6.7.6.5 +2001:db8:85a3::8a2e:370:7334 192.168.0.0/16 +2001:db8:85a3::/48 ==== record [a=42, b=6.6.7.7] 42, 6.6.7.7 diff --git a/testing/btest/Baseline/istate.pybroccoli/python..stdout.filtered b/testing/btest/Baseline/istate.pybroccoli/python..stdout.filtered index 864a4eb627..5d98e2d759 100644 --- a/testing/btest/Baseline/istate.pybroccoli/python..stdout.filtered +++ b/testing/btest/Baseline/istate.pybroccoli/python..stdout.filtered @@ -1,7 +1,7 @@ ==== atomic a 1 ==== -4L -4 42 42 -1313624487.4889 +1330035434.5180 60.0 True True 3.14 @@ -9,10 +9,12 @@ True True '12345/udp' 12345/udp '1.2.3.4' 1.2.3.4 '22.33.44.0/24' 22.33.44.0/24 +'2607:f8b0:4009:802::1014' 2607:f8b0:4009:802::1014 +'2607:f8b0::/32' 2607:f8b0::/32 ==== atomic a 2 ==== -10L -10 2 2 -1313624487.4882 +1330035434.5169 120.0 False False 1.5 @@ -20,10 +22,12 @@ False False '5555/tcp' 5555/tcp '6.7.6.5' 6.7.6.5 '192.168.0.0/16' 192.168.0.0/16 +'2001:db8:85a3::8a2e:370:7334' 2001:db8:85a3::8a2e:370:7334 +'2001:db8:85a3::/48' 2001:db8:85a3::/48 ==== atomic b 2 ==== -10L -10 2 - 1313624487.4882 + 1330035434.5169 120.0 False False 1.5 @@ -31,6 +35,8 @@ False False 5555/tcp 6.7.6.5 192.168.0.0/16 + 2001:db8:85a3::8a2e:370:7334 + 2001:db8:85a3::/48 ==== record 1 ==== 42L 42 diff --git a/testing/btest/Baseline/language.cross-product-init/output b/testing/btest/Baseline/language.cross-product-init/output index 95794f9179..fcb5e43e67 100644 --- a/testing/btest/Baseline/language.cross-product-init/output +++ b/testing/btest/Baseline/language.cross-product-init/output @@ -1,6 +1,6 @@ { -[bar, 1.2.0.0/19] , -[foo, 5.6.0.0/21] , +[foo, 1.2.0.0/19] , [bar, 5.6.0.0/21] , -[foo, 1.2.0.0/19] +[foo, 5.6.0.0/21] , +[bar, 1.2.0.0/19] } diff --git a/testing/btest/Baseline/language.expire_func/output b/testing/btest/Baseline/language.expire_func/output new file mode 100644 index 0000000000..91cd2bad16 --- /dev/null +++ b/testing/btest/Baseline/language.expire_func/output @@ -0,0 +1,378 @@ +{ +[orig_h=172.16.238.1, orig_p=49656/tcp, resp_h=172.16.238.131, resp_p=22/tcp], +i, +here, +am +} +{ +[orig_h=172.16.238.1, orig_p=49656/tcp, resp_h=172.16.238.131, resp_p=22/tcp], +i, +[orig_h=172.16.238.131, orig_p=37975/udp, resp_h=172.16.238.2, resp_p=53/udp], +here, +am +} +{ +[orig_h=172.16.238.1, orig_p=49656/tcp, resp_h=172.16.238.131, resp_p=22/tcp], +i, +[orig_h=172.16.238.131, orig_p=37975/udp, resp_h=172.16.238.2, resp_p=53/udp], +here, +[orig_h=fe80::20c:29ff:febd:6f01, orig_p=5353/udp, resp_h=ff02::fb, resp_p=5353/udp], +am +} +{ +[orig_h=172.16.238.131, orig_p=5353/udp, resp_h=224.0.0.251, resp_p=5353/udp], +[orig_h=172.16.238.1, orig_p=49656/tcp, resp_h=172.16.238.131, resp_p=22/tcp], +i, +[orig_h=172.16.238.131, orig_p=37975/udp, resp_h=172.16.238.2, resp_p=53/udp], +here, +[orig_h=fe80::20c:29ff:febd:6f01, orig_p=5353/udp, resp_h=ff02::fb, resp_p=5353/udp], +am +} +{ +[orig_h=172.16.238.131, orig_p=5353/udp, resp_h=224.0.0.251, resp_p=5353/udp], +[orig_h=172.16.238.1, orig_p=49656/tcp, resp_h=172.16.238.131, resp_p=22/tcp], +i, +[orig_h=172.16.238.131, orig_p=37975/udp, resp_h=172.16.238.2, resp_p=53/udp], +here, +[orig_h=fe80::20c:29ff:febd:6f01, orig_p=5353/udp, resp_h=ff02::fb, resp_p=5353/udp], +[orig_h=172.16.238.1, orig_p=5353/udp, resp_h=224.0.0.251, resp_p=5353/udp], +am +} +{ +[orig_h=172.16.238.131, orig_p=5353/udp, resp_h=224.0.0.251, resp_p=5353/udp], +[orig_h=172.16.238.1, orig_p=49656/tcp, resp_h=172.16.238.131, resp_p=22/tcp], +i, +[orig_h=172.16.238.131, orig_p=37975/udp, resp_h=172.16.238.2, resp_p=53/udp], +here, +[orig_h=172.16.238.1, orig_p=49657/tcp, resp_h=172.16.238.131, resp_p=80/tcp], +[orig_h=fe80::20c:29ff:febd:6f01, orig_p=5353/udp, resp_h=ff02::fb, resp_p=5353/udp], +[orig_h=172.16.238.1, orig_p=5353/udp, resp_h=224.0.0.251, resp_p=5353/udp], +am +} +{ +[orig_h=172.16.238.131, orig_p=5353/udp, resp_h=224.0.0.251, resp_p=5353/udp], +[orig_h=172.16.238.1, orig_p=49656/tcp, resp_h=172.16.238.131, resp_p=22/tcp], +i, +[orig_h=172.16.238.131, orig_p=37975/udp, resp_h=172.16.238.2, resp_p=53/udp], +here, +[orig_h=172.16.238.1, orig_p=49657/tcp, resp_h=172.16.238.131, resp_p=80/tcp], +[orig_h=172.16.238.1, orig_p=49658/tcp, resp_h=172.16.238.131, resp_p=80/tcp], +[orig_h=fe80::20c:29ff:febd:6f01, orig_p=5353/udp, resp_h=ff02::fb, resp_p=5353/udp], +[orig_h=172.16.238.1, orig_p=5353/udp, resp_h=224.0.0.251, resp_p=5353/udp], +am +} +{ +[orig_h=172.16.238.131, orig_p=5353/udp, resp_h=224.0.0.251, resp_p=5353/udp], +[orig_h=172.16.238.1, orig_p=17500/udp, resp_h=172.16.238.255, resp_p=17500/udp], +[orig_h=172.16.238.1, orig_p=49656/tcp, resp_h=172.16.238.131, resp_p=22/tcp], +i, +[orig_h=172.16.238.131, orig_p=37975/udp, resp_h=172.16.238.2, resp_p=53/udp], +here, +[orig_h=172.16.238.1, orig_p=49657/tcp, resp_h=172.16.238.131, resp_p=80/tcp], +[orig_h=172.16.238.1, orig_p=49658/tcp, resp_h=172.16.238.131, resp_p=80/tcp], +[orig_h=fe80::20c:29ff:febd:6f01, orig_p=5353/udp, resp_h=ff02::fb, resp_p=5353/udp], +[orig_h=172.16.238.1, orig_p=5353/udp, resp_h=224.0.0.251, resp_p=5353/udp], +am +} +expired [orig_h=172.16.238.131, orig_p=5353/udp, resp_h=224.0.0.251, resp_p=5353/udp] +expired [orig_h=172.16.238.1, orig_p=17500/udp, resp_h=172.16.238.255, resp_p=17500/udp] +expired [orig_h=172.16.238.1, orig_p=49656/tcp, resp_h=172.16.238.131, resp_p=22/tcp] +expired i +expired [orig_h=172.16.238.131, orig_p=37975/udp, resp_h=172.16.238.2, resp_p=53/udp] +expired here +expired [orig_h=172.16.238.1, orig_p=49657/tcp, resp_h=172.16.238.131, resp_p=80/tcp] +expired [orig_h=172.16.238.1, orig_p=49658/tcp, resp_h=172.16.238.131, resp_p=80/tcp] +expired [orig_h=fe80::20c:29ff:febd:6f01, orig_p=5353/udp, resp_h=ff02::fb, resp_p=5353/udp] +expired [orig_h=172.16.238.1, orig_p=5353/udp, resp_h=224.0.0.251, resp_p=5353/udp] +expired am +{ +[orig_h=172.16.238.1, orig_p=49659/tcp, resp_h=172.16.238.131, resp_p=21/tcp] +} +{ +[orig_h=172.16.238.1, orig_p=49659/tcp, resp_h=172.16.238.131, resp_p=21/tcp], +[orig_h=172.16.238.131, orig_p=45126/udp, resp_h=172.16.238.2, resp_p=53/udp] +} +expired [orig_h=172.16.238.1, orig_p=49659/tcp, resp_h=172.16.238.131, resp_p=21/tcp] +expired [orig_h=172.16.238.131, orig_p=45126/udp, resp_h=172.16.238.2, resp_p=53/udp] +{ +[orig_h=172.16.238.131, orig_p=55515/tcp, resp_h=74.125.225.81, resp_p=80/tcp] +} +{ +[orig_h=172.16.238.131, orig_p=37846/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=55515/tcp, resp_h=74.125.225.81, resp_p=80/tcp] +} +{ +[orig_h=172.16.238.131, orig_p=37846/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=55515/tcp, resp_h=74.125.225.81, resp_p=80/tcp], +[orig_h=172.16.238.131, orig_p=51970/udp, resp_h=172.16.238.2, resp_p=53/udp] +} +{ +[orig_h=172.16.238.131, orig_p=37846/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=55515/tcp, resp_h=74.125.225.81, resp_p=80/tcp], +[orig_h=172.16.238.131, orig_p=51970/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=54304/udp, resp_h=172.16.238.2, resp_p=53/udp] +} +{ +[orig_h=172.16.238.131, orig_p=37846/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=55515/tcp, resp_h=74.125.225.81, resp_p=80/tcp], +[orig_h=172.16.238.131, orig_p=44555/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=51970/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=54304/udp, resp_h=172.16.238.2, resp_p=53/udp] +} +{ +[orig_h=172.16.238.131, orig_p=37846/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=55515/tcp, resp_h=74.125.225.81, resp_p=80/tcp], +[orig_h=172.16.238.131, orig_p=44555/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=33109/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=51970/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=54304/udp, resp_h=172.16.238.2, resp_p=53/udp] +} +{ +[orig_h=172.16.238.131, orig_p=37846/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=55515/tcp, resp_h=74.125.225.81, resp_p=80/tcp], +[orig_h=172.16.238.131, orig_p=44555/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=50205/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=33109/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=51970/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=54304/udp, resp_h=172.16.238.2, resp_p=53/udp] +} +{ +[orig_h=172.16.238.131, orig_p=37846/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=57272/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=55515/tcp, resp_h=74.125.225.81, resp_p=80/tcp], +[orig_h=172.16.238.131, orig_p=44555/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=50205/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=33109/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=51970/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=54304/udp, resp_h=172.16.238.2, resp_p=53/udp] +} +{ +[orig_h=172.16.238.131, orig_p=37846/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=57272/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=55515/tcp, resp_h=74.125.225.81, resp_p=80/tcp], +[orig_h=172.16.238.131, orig_p=44555/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=50205/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=33818/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=33109/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=51970/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=54304/udp, resp_h=172.16.238.2, resp_p=53/udp] +} +{ +[orig_h=172.16.238.131, orig_p=37846/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=57272/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=55515/tcp, resp_h=74.125.225.81, resp_p=80/tcp], +[orig_h=172.16.238.131, orig_p=44555/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=50205/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=33818/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=33109/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=45140/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=51970/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=54304/udp, resp_h=172.16.238.2, resp_p=53/udp] +} +{ +[orig_h=172.16.238.131, orig_p=37846/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=57272/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=55515/tcp, resp_h=74.125.225.81, resp_p=80/tcp], +[orig_h=172.16.238.131, orig_p=44555/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=55368/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=50205/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=33818/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=33109/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=45140/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=51970/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=54304/udp, resp_h=172.16.238.2, resp_p=53/udp] +} +{ +[orig_h=172.16.238.131, orig_p=37846/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=57272/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=55515/tcp, resp_h=74.125.225.81, resp_p=80/tcp], +[orig_h=172.16.238.131, orig_p=44555/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=55368/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=50205/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=33818/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=33109/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=45140/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=53102/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=51970/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=54304/udp, resp_h=172.16.238.2, resp_p=53/udp] +} +{ +[orig_h=172.16.238.131, orig_p=37846/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=57272/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=55515/tcp, resp_h=74.125.225.81, resp_p=80/tcp], +[orig_h=172.16.238.131, orig_p=44555/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=55368/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=50205/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=59573/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=33818/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=33109/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=45140/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=53102/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=51970/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=54304/udp, resp_h=172.16.238.2, resp_p=53/udp] +} +{ +[orig_h=172.16.238.131, orig_p=37846/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=57272/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=55515/tcp, resp_h=74.125.225.81, resp_p=80/tcp], +[orig_h=172.16.238.131, orig_p=44555/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=55368/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=50205/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=59573/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=33818/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=33109/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=52952/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=45140/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=53102/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=51970/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=54304/udp, resp_h=172.16.238.2, resp_p=53/udp] +} +{ +[orig_h=172.16.238.131, orig_p=48621/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=37846/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=57272/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=55515/tcp, resp_h=74.125.225.81, resp_p=80/tcp], +[orig_h=172.16.238.131, orig_p=44555/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=55368/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=50205/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=59573/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=33818/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=33109/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=52952/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=45140/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=53102/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=51970/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=54304/udp, resp_h=172.16.238.2, resp_p=53/udp] +} +expired [orig_h=172.16.238.131, orig_p=48621/udp, resp_h=172.16.238.2, resp_p=53/udp] +expired [orig_h=172.16.238.131, orig_p=37846/udp, resp_h=172.16.238.2, resp_p=53/udp] +expired [orig_h=172.16.238.131, orig_p=57272/udp, resp_h=172.16.238.2, resp_p=53/udp] +expired [orig_h=172.16.238.131, orig_p=55515/tcp, resp_h=74.125.225.81, resp_p=80/tcp] +expired [orig_h=172.16.238.131, orig_p=44555/udp, resp_h=172.16.238.2, resp_p=53/udp] +expired [orig_h=172.16.238.131, orig_p=55368/udp, resp_h=172.16.238.2, resp_p=53/udp] +expired [orig_h=172.16.238.131, orig_p=50205/udp, resp_h=172.16.238.2, resp_p=53/udp] +expired [orig_h=172.16.238.131, orig_p=59573/udp, resp_h=172.16.238.2, resp_p=53/udp] +expired [orig_h=172.16.238.131, orig_p=33818/udp, resp_h=172.16.238.2, resp_p=53/udp] +expired [orig_h=172.16.238.131, orig_p=33109/udp, resp_h=172.16.238.2, resp_p=53/udp] +expired [orig_h=172.16.238.131, orig_p=52952/udp, resp_h=172.16.238.2, resp_p=53/udp] +expired [orig_h=172.16.238.131, orig_p=45140/udp, resp_h=172.16.238.2, resp_p=53/udp] +expired [orig_h=172.16.238.131, orig_p=53102/udp, resp_h=172.16.238.2, resp_p=53/udp] +expired [orig_h=172.16.238.131, orig_p=51970/udp, resp_h=172.16.238.2, resp_p=53/udp] +expired [orig_h=172.16.238.131, orig_p=54304/udp, resp_h=172.16.238.2, resp_p=53/udp] +{ +[orig_h=172.16.238.131, orig_p=54935/udp, resp_h=172.16.238.2, resp_p=53/udp] +} +{ +[orig_h=172.16.238.131, orig_p=54935/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=33624/udp, resp_h=172.16.238.2, resp_p=53/udp] +} +{ +[orig_h=172.16.238.131, orig_p=54935/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=33624/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=45908/tcp, resp_h=141.142.192.39, resp_p=22/tcp] +} +{ +[orig_h=172.16.238.131, orig_p=54935/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=56214/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=33624/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=45908/tcp, resp_h=141.142.192.39, resp_p=22/tcp] +} +{ +[orig_h=172.16.238.131, orig_p=54935/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=56214/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=38118/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=33624/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=45908/tcp, resp_h=141.142.192.39, resp_p=22/tcp] +} +{ +[orig_h=172.16.238.131, orig_p=54935/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=56214/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=38118/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=33624/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=37934/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=45908/tcp, resp_h=141.142.192.39, resp_p=22/tcp] +} +{ +[orig_h=172.16.238.131, orig_p=54935/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=56214/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=38118/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=33624/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=37934/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=45908/tcp, resp_h=141.142.192.39, resp_p=22/tcp], +[orig_h=172.16.238.131, orig_p=36682/udp, resp_h=172.16.238.2, resp_p=53/udp] +} +{ +[orig_h=172.16.238.131, orig_p=54935/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=56214/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=38118/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=46552/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=33624/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=37934/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=45908/tcp, resp_h=141.142.192.39, resp_p=22/tcp], +[orig_h=172.16.238.131, orig_p=36682/udp, resp_h=172.16.238.2, resp_p=53/udp] +} +{ +[orig_h=172.16.238.131, orig_p=54935/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=58367/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=56214/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=38118/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=46552/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=33624/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=37934/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=45908/tcp, resp_h=141.142.192.39, resp_p=22/tcp], +[orig_h=172.16.238.131, orig_p=36682/udp, resp_h=172.16.238.2, resp_p=53/udp] +} +{ +[orig_h=172.16.238.131, orig_p=54935/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=58367/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=56214/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=38118/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=46552/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=42269/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=33624/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=37934/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=45908/tcp, resp_h=141.142.192.39, resp_p=22/tcp], +[orig_h=172.16.238.131, orig_p=36682/udp, resp_h=172.16.238.2, resp_p=53/udp] +} +{ +[orig_h=172.16.238.131, orig_p=54935/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=58367/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=56214/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=38118/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=56485/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=46552/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=42269/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=33624/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=37934/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=45908/tcp, resp_h=141.142.192.39, resp_p=22/tcp], +[orig_h=172.16.238.131, orig_p=36682/udp, resp_h=172.16.238.2, resp_p=53/udp] +} +{ +[orig_h=172.16.238.131, orig_p=54935/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=58367/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=56214/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=38118/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=56485/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=46552/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=42269/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=33624/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=37934/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=39723/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=45908/tcp, resp_h=141.142.192.39, resp_p=22/tcp], +[orig_h=172.16.238.131, orig_p=36682/udp, resp_h=172.16.238.2, resp_p=53/udp] +} +{ +[orig_h=172.16.238.131, orig_p=54935/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=58367/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=56214/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=123/udp, resp_h=69.50.219.51, resp_p=123/udp], +[orig_h=172.16.238.131, orig_p=38118/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=56485/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=46552/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=42269/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=33624/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=37934/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=39723/udp, resp_h=172.16.238.2, resp_p=53/udp], +[orig_h=172.16.238.131, orig_p=45908/tcp, resp_h=141.142.192.39, resp_p=22/tcp], +[orig_h=172.16.238.131, orig_p=36682/udp, resp_h=172.16.238.2, resp_p=53/udp] +} diff --git a/testing/btest/Baseline/language.ipv6-literals/output b/testing/btest/Baseline/language.ipv6-literals/output new file mode 100644 index 0000000000..8542af7f91 --- /dev/null +++ b/testing/btest/Baseline/language.ipv6-literals/output @@ -0,0 +1,24 @@ +::1 +::ffff +::255.255.255.255 +::10.10.255.255 +1::1 +1::a +1::1:1 +1::1:a +a::a +a::1 +a::a:a +a::a:1 +a:a::a +aaaa::ffff +192.168.1.100 +ffff::c0a8:164 +::192.168.1.100 +::ffff:0:192.168.1.100 +805b:2d9d:dc28::fc57:d4c8:1fff +aaaa::bbbb +aaaa:bbbb:cccc:dddd:eeee:ffff:1111:2222 +aaaa:bbbb:cccc:dddd:eeee:ffff:1:2222 +aaaa:bbbb:cccc:dddd:eeee:ffff:0:2222 +aaaa:bbbb:cccc:dddd:eeee::2222 diff --git a/testing/btest/Baseline/language.match-test/output b/testing/btest/Baseline/language.match-test/output deleted file mode 100644 index 5ee7ba029d..0000000000 --- a/testing/btest/Baseline/language.match-test/output +++ /dev/null @@ -1,3 +0,0 @@ -default -it's big -it's really big diff --git a/testing/btest/Baseline/language.match-test2/output b/testing/btest/Baseline/language.match-test2/output deleted file mode 100644 index 0cfbf08886..0000000000 --- a/testing/btest/Baseline/language.match-test2/output +++ /dev/null @@ -1 +0,0 @@ -2 diff --git a/testing/btest/Baseline/language.sizeof/output b/testing/btest/Baseline/language.sizeof/output index 737a999292..43cb73f763 100644 --- a/testing/btest/Baseline/language.sizeof/output +++ b/testing/btest/Baseline/language.sizeof/output @@ -1,4 +1,5 @@ -Address 1.2.3.4: 16909060 +IPv4 Address 1.2.3.4: 32 +IPv6 Address ::1: 128 Boolean T: 1 Count 10: 10 Double -1.23: 1.230000 diff --git a/testing/btest/Baseline/scripts.base.frameworks.logging.path-func-column-demote/local.log b/testing/btest/Baseline/scripts.base.frameworks.logging.path-func-column-demote/local.log index e2b3da6efd..291909b80a 100644 --- a/testing/btest/Baseline/scripts.base.frameworks.logging.path-func-column-demote/local.log +++ b/testing/btest/Baseline/scripts.base.frameworks.logging.path-func-column-demote/local.log @@ -5,33 +5,33 @@ #path local #fields ts id.orig_h #types time addr -1300475168.652003 141.142.220.118 -1300475168.724007 141.142.220.118 -1300475168.859163 141.142.220.118 -1300475168.902635 141.142.220.118 -1300475168.892936 141.142.220.118 -1300475168.892913 141.142.220.118 -1300475168.855305 141.142.220.118 1300475168.855330 141.142.220.118 +1300475168.652003 141.142.220.118 1300475168.895267 141.142.220.118 -1300475168.853899 141.142.220.118 -1300475168.893988 141.142.220.118 -1300475168.894787 141.142.220.118 -1300475173.117362 141.142.220.226 -1300475173.153679 141.142.220.238 -1300475168.857956 141.142.220.118 -1300475168.854378 141.142.220.118 -1300475168.854837 141.142.220.118 -1300475167.099816 141.142.220.50 +1300475168.855305 141.142.220.118 +1300475168.859163 141.142.220.118 +1300475168.892913 141.142.220.118 +1300475168.724007 141.142.220.118 +1300475168.892936 141.142.220.118 +1300475168.902635 141.142.220.118 1300475168.891644 141.142.220.118 -1300475168.892037 141.142.220.118 -1300475171.677081 141.142.220.226 -1300475168.894422 141.142.220.118 -1300475167.096535 141.142.220.202 -1300475168.858713 141.142.220.118 -1300475168.902195 141.142.220.118 -1300475169.899438 141.142.220.44 -1300475168.892414 141.142.220.118 -1300475168.858306 141.142.220.118 -1300475168.901749 141.142.220.118 1300475170.862384 141.142.220.226 +1300475168.853899 141.142.220.118 +1300475168.854378 141.142.220.118 +1300475168.857956 141.142.220.118 +1300475173.117362 141.142.220.226 +1300475168.858713 141.142.220.118 +1300475167.096535 141.142.220.202 +1300475167.099816 141.142.220.50 +1300475168.892037 141.142.220.118 +1300475168.893988 141.142.220.118 +1300475168.854837 141.142.220.118 +1300475169.899438 141.142.220.44 +1300475168.858306 141.142.220.118 +1300475168.894422 141.142.220.118 +1300475173.153679 141.142.220.238 +1300475168.892414 141.142.220.118 +1300475171.677081 141.142.220.226 +1300475168.902195 141.142.220.118 +1300475168.894787 141.142.220.118 +1300475168.901749 141.142.220.118 diff --git a/testing/btest/Baseline/scripts.base.frameworks.logging.path-func-column-demote/remote.log b/testing/btest/Baseline/scripts.base.frameworks.logging.path-func-column-demote/remote.log index 1ac18ff5f7..b396c3fc2d 100644 --- a/testing/btest/Baseline/scripts.base.frameworks.logging.path-func-column-demote/remote.log +++ b/testing/btest/Baseline/scripts.base.frameworks.logging.path-func-column-demote/remote.log @@ -6,3 +6,6 @@ #fields ts id.orig_h #types time addr 1300475169.780331 173.192.163.128 +1300475167.097012 fe80::217:f2ff:fed7:cf65 +1300475171.675372 fe80::3074:17d5:2052:c324 +1300475173.116749 fe80::3074:17d5:2052:c324 diff --git a/testing/btest/Baseline/scripts.base.frameworks.metrics.basic-cluster/manager-1.metrics.log b/testing/btest/Baseline/scripts.base.frameworks.metrics.basic-cluster/manager-1.metrics.log index a278bdc56a..a22deb26e4 100644 --- a/testing/btest/Baseline/scripts.base.frameworks.metrics.basic-cluster/manager-1.metrics.log +++ b/testing/btest/Baseline/scripts.base.frameworks.metrics.basic-cluster/manager-1.metrics.log @@ -5,6 +5,6 @@ #path metrics #fields ts metric_id filter_name index.host index.str index.network value #types time enum string addr string subnet count -1324314335.570789 TEST_METRIC foo-bar 6.5.4.3 - - 4 -1324314335.570789 TEST_METRIC foo-bar 1.2.3.4 - - 6 -1324314335.570789 TEST_METRIC foo-bar 7.2.1.5 - - 2 +1328303679.867377 TEST_METRIC foo-bar 6.5.4.3 - - 4 +1328303679.867377 TEST_METRIC foo-bar 7.2.1.5 - - 2 +1328303679.867377 TEST_METRIC foo-bar 1.2.3.4 - - 6 diff --git a/testing/btest/Baseline/scripts.base.frameworks.metrics.basic/metrics.log b/testing/btest/Baseline/scripts.base.frameworks.metrics.basic/metrics.log index 8ee19c255b..4bfb6964ea 100644 --- a/testing/btest/Baseline/scripts.base.frameworks.metrics.basic/metrics.log +++ b/testing/btest/Baseline/scripts.base.frameworks.metrics.basic/metrics.log @@ -5,6 +5,6 @@ #path metrics #fields ts metric_id filter_name index.host index.str index.network value #types time enum string addr string subnet count -1324314344.807073 TEST_METRIC foo-bar 6.5.4.3 - - 2 -1324314344.807073 TEST_METRIC foo-bar 1.2.3.4 - - 3 -1324314344.807073 TEST_METRIC foo-bar 7.2.1.5 - - 1 +1328303763.333948 TEST_METRIC foo-bar 6.5.4.3 - - 2 +1328303763.333948 TEST_METRIC foo-bar 7.2.1.5 - - 1 +1328303763.333948 TEST_METRIC foo-bar 1.2.3.4 - - 3 diff --git a/testing/btest/Baseline/scripts.base.protocols.ftp.ftp-ipv4/conn.log b/testing/btest/Baseline/scripts.base.protocols.ftp.ftp-ipv4/conn.log new file mode 100644 index 0000000000..bcb05ef415 --- /dev/null +++ b/testing/btest/Baseline/scripts.base.protocols.ftp.ftp-ipv4/conn.log @@ -0,0 +1,12 @@ +#separator \x09 +#set_separator , +#empty_field (empty) +#unset_field - +#path conn +#fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p proto service duration orig_bytes resp_bytes conn_state local_orig missed_bytes history orig_pkts orig_ip_bytes resp_pkts resp_ip_bytes +#types time string addr port addr port enum string interval count count string bool count string count count count count +1329843175.736107 arKYeMETxOg 141.142.220.235 37604 199.233.217.249 56666 tcp ftp-data 0.112432 0 342 SF - 0 ShAdfFa 4 216 4 562 +1329843179.871641 k6kgXLOoSKl 141.142.220.235 59378 199.233.217.249 56667 tcp ftp-data 0.111218 0 77 SF - 0 ShAdfFa 4 216 4 297 +1329843194.151526 nQcgTWjvg4c 199.233.217.249 61920 141.142.220.235 33582 tcp ftp-data 0.056211 342 0 SF - 0 ShADaFf 5 614 3 164 +1329843197.783443 j4u32Pc5bif 199.233.217.249 61918 141.142.220.235 37835 tcp ftp-data 0.056005 77 0 SF - 0 ShADaFf 5 349 3 164 +1329843161.968492 UWkUyAuUGXf 141.142.220.235 50003 199.233.217.249 21 tcp ftp 38.055625 180 3146 SF - 0 ShAdDfFa 38 2164 25 4458 diff --git a/testing/btest/Baseline/scripts.base.protocols.ftp.ftp-ipv4/ftp.log b/testing/btest/Baseline/scripts.base.protocols.ftp.ftp-ipv4/ftp.log new file mode 100644 index 0000000000..debc093771 --- /dev/null +++ b/testing/btest/Baseline/scripts.base.protocols.ftp.ftp-ipv4/ftp.log @@ -0,0 +1,9 @@ +#separator \x09 +#set_separator , +#empty_field (empty) +#unset_field - +#path ftp +#fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p user password command arg mime_type mime_desc file_size reply_code reply_msg tags extraction_file +#types time string addr port addr port string string string string string string count count string table[string] file +1329843179.926563 UWkUyAuUGXf 141.142.220.235 50003 199.233.217.249 21 anonymous test RETR ftp://199.233.217.249/./robots.txt text/plain ASCII text 77 226 Transfer complete. - - +1329843197.727769 UWkUyAuUGXf 141.142.220.235 50003 199.233.217.249 21 anonymous test RETR ftp://199.233.217.249/./robots.txt text/plain ASCII text, with CRLF line terminators 77 226 Transfer complete. - - diff --git a/testing/btest/Baseline/scripts.base.protocols.ftp.ftp-ipv6/conn.log b/testing/btest/Baseline/scripts.base.protocols.ftp.ftp-ipv6/conn.log new file mode 100644 index 0000000000..c4a515710d --- /dev/null +++ b/testing/btest/Baseline/scripts.base.protocols.ftp.ftp-ipv6/conn.log @@ -0,0 +1,13 @@ +#separator \x09 +#set_separator , +#empty_field (empty) +#unset_field - +#path conn +#fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p proto service duration orig_bytes resp_bytes conn_state local_orig missed_bytes history orig_pkts orig_ip_bytes resp_pkts resp_ip_bytes +#types time string addr port addr port enum string interval count count string bool count string count count count count +1329327783.316897 arKYeMETxOg 2001:470:1f11:81f:c999:d94:aa7c:2e3e 49186 2001:470:4867:99::21 57086 tcp ftp-data 0.219721 0 342 SF - 0 ShAdfFa 5 372 4 642 +1329327786.524332 k6kgXLOoSKl 2001:470:1f11:81f:c999:d94:aa7c:2e3e 49187 2001:470:4867:99::21 57087 tcp ftp-data 0.217501 0 43 SF - 0 ShAdfFa 5 372 4 343 +1329327787.289095 nQcgTWjvg4c 2001:470:1f11:81f:c999:d94:aa7c:2e3e 49188 2001:470:4867:99::21 57088 tcp ftp-data 0.217941 0 77 SF - 0 ShAdfFa 5 372 4 377 +1329327795.571921 j4u32Pc5bif 2001:470:4867:99::21 55785 2001:470:1f11:81f:c999:d94:aa7c:2e3e 49189 tcp ftp-data 0.109813 77 0 SF - 0 ShADFaf 5 449 4 300 +1329327777.822004 UWkUyAuUGXf 2001:470:1f11:81f:c999:d94:aa7c:2e3e 49185 2001:470:4867:99::21 21 tcp ftp 26.658219 310 3448 SF - 0 ShAdDfFa 57 4426 34 5908 +1329327800.017649 TEfuqmmG4bh 2001:470:4867:99::21 55647 2001:470:1f11:81f:c999:d94:aa7c:2e3e 49190 tcp ftp-data 0.109181 342 0 SF - 0 ShADFaf 5 714 4 300 diff --git a/testing/btest/Baseline/scripts.base.protocols.ftp.ftp-ipv6/ftp.log b/testing/btest/Baseline/scripts.base.protocols.ftp.ftp-ipv6/ftp.log new file mode 100644 index 0000000000..8bc2ef2cb7 --- /dev/null +++ b/testing/btest/Baseline/scripts.base.protocols.ftp.ftp-ipv6/ftp.log @@ -0,0 +1,9 @@ +#separator \x09 +#set_separator , +#empty_field (empty) +#unset_field - +#path ftp +#fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p user password command arg mime_type mime_desc file_size reply_code reply_msg tags extraction_file +#types time string addr port addr port string string string string string string count count string table[string] file +1329327787.396984 UWkUyAuUGXf 2001:470:1f11:81f:c999:d94:aa7c:2e3e 49185 2001:470:4867:99::21 21 anonymous test RETR ftp://[2001:470:4867:99::21]/robots.txt - - 77 226 Transfer complete. - - +1329327795.463946 UWkUyAuUGXf 2001:470:1f11:81f:c999:d94:aa7c:2e3e 49185 2001:470:4867:99::21 21 anonymous test RETR ftp://[2001:470:4867:99::21]/robots.txt - - 77 226 Transfer complete. - - diff --git a/testing/btest/Baseline/scripts.policy.protocols.dns.event-priority/dns.log b/testing/btest/Baseline/scripts.policy.protocols.dns.event-priority/dns.log index 9d80898e0f..f636093677 100644 --- a/testing/btest/Baseline/scripts.policy.protocols.dns.event-priority/dns.log +++ b/testing/btest/Baseline/scripts.policy.protocols.dns.event-priority/dns.log @@ -3,6 +3,6 @@ #empty_field (empty) #unset_field - #path dns -#fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p proto trans_id query qclass qclass_name qtype qtype_name rcode rcode_name QR AA TC RD RA Z answers TTLs auth addl -#types time string addr port addr port enum count string count string count string count string bool bool bool bool bool count vector[string] vector[interval] table[string] table[string] -930613226.529070 UWkUyAuUGXf 212.180.42.100 25000 131.243.64.3 53 tcp 34798 - - - - - 0 NOERROR F F F F T 0 4.3.2.1 31337.000000 - - +#fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p proto trans_id query qclass qclass_name qtype qtype_name rcode rcode_name AA TC RD RA Z answers TTLs auth addl +#types time string addr port addr port enum count string count string count string count string bool bool bool bool count vector[string] vector[interval] table[string] table[string] +930613226.529070 UWkUyAuUGXf 212.180.42.100 25000 131.243.64.3 53 tcp 34798 - - - - - 0 NOERROR F F F T 0 4.3.2.1 31337.000000 - - diff --git a/testing/btest/Makefile b/testing/btest/Makefile index a2ca30609a..257146daa0 100644 --- a/testing/btest/Makefile +++ b/testing/btest/Makefile @@ -2,16 +2,23 @@ DIAG=diag.log BTEST=../../aux/btest/btest -all: cleanup - # Showing all tests. - @$(BTEST) -f $(DIAG) - @../scripts/coverage-calc ".tmp/script-coverage*" coverage.log `pwd`/../../scripts +all: cleanup btest-verbose coverage -brief: cleanup - # Brief output showing only failed tests. - @$(BTEST) -b -f $(DIAG) +# Showing all tests. +btest-verbose: + @$(BTEST) -j 5 -f $(DIAG) + +brief: cleanup btest-brief coverage + +# Brief output showing only failed tests. +btest-brief: + @$(BTEST) -j 5 -b -f $(DIAG) + +coverage: @../scripts/coverage-calc ".tmp/script-coverage*" coverage.log `pwd`/../../scripts cleanup: @rm -f $(DIAG) @rm -f .tmp/script-coverage* + +.PHONY: all btest-verbose brief btest-brief coverage cleanup diff --git a/testing/btest/Traces/chksums/ip4-bad-chksum.pcap b/testing/btest/Traces/chksums/ip4-bad-chksum.pcap new file mode 100644 index 0000000000..6d8b9dd27d Binary files /dev/null and b/testing/btest/Traces/chksums/ip4-bad-chksum.pcap differ diff --git a/testing/btest/Traces/chksums/ip4-icmp-bad-chksum.pcap b/testing/btest/Traces/chksums/ip4-icmp-bad-chksum.pcap new file mode 100644 index 0000000000..cc60d879c4 Binary files /dev/null and b/testing/btest/Traces/chksums/ip4-icmp-bad-chksum.pcap differ diff --git a/testing/btest/Traces/chksums/ip4-icmp-good-chksum.pcap b/testing/btest/Traces/chksums/ip4-icmp-good-chksum.pcap new file mode 100644 index 0000000000..2b07326eab Binary files /dev/null and b/testing/btest/Traces/chksums/ip4-icmp-good-chksum.pcap differ diff --git a/testing/btest/Traces/chksums/ip4-tcp-bad-chksum.pcap b/testing/btest/Traces/chksums/ip4-tcp-bad-chksum.pcap new file mode 100644 index 0000000000..b9ccd9e6b2 Binary files /dev/null and b/testing/btest/Traces/chksums/ip4-tcp-bad-chksum.pcap differ diff --git a/testing/btest/Traces/chksums/ip4-tcp-good-chksum.pcap b/testing/btest/Traces/chksums/ip4-tcp-good-chksum.pcap new file mode 100644 index 0000000000..ff3f011884 Binary files /dev/null and b/testing/btest/Traces/chksums/ip4-tcp-good-chksum.pcap differ diff --git a/testing/btest/Traces/chksums/ip4-udp-bad-chksum.pcap b/testing/btest/Traces/chksums/ip4-udp-bad-chksum.pcap new file mode 100644 index 0000000000..f3998c7e1c Binary files /dev/null and b/testing/btest/Traces/chksums/ip4-udp-bad-chksum.pcap differ diff --git a/testing/btest/Traces/chksums/ip4-udp-good-chksum.pcap b/testing/btest/Traces/chksums/ip4-udp-good-chksum.pcap new file mode 100644 index 0000000000..3aec507329 Binary files /dev/null and b/testing/btest/Traces/chksums/ip4-udp-good-chksum.pcap differ diff --git a/testing/btest/Traces/chksums/ip6-hoa-tcp-bad-chksum.pcap b/testing/btest/Traces/chksums/ip6-hoa-tcp-bad-chksum.pcap new file mode 100644 index 0000000000..3aa4bd21fa Binary files /dev/null and b/testing/btest/Traces/chksums/ip6-hoa-tcp-bad-chksum.pcap differ diff --git a/testing/btest/Traces/chksums/ip6-hoa-tcp-good-chksum.pcap b/testing/btest/Traces/chksums/ip6-hoa-tcp-good-chksum.pcap new file mode 100644 index 0000000000..a6fc9cb017 Binary files /dev/null and b/testing/btest/Traces/chksums/ip6-hoa-tcp-good-chksum.pcap differ diff --git a/testing/btest/Traces/chksums/ip6-hoa-udp-bad-chksum.pcap b/testing/btest/Traces/chksums/ip6-hoa-udp-bad-chksum.pcap new file mode 100644 index 0000000000..d2434dea80 Binary files /dev/null and b/testing/btest/Traces/chksums/ip6-hoa-udp-bad-chksum.pcap differ diff --git a/testing/btest/Traces/chksums/ip6-hoa-udp-good-chksum.pcap b/testing/btest/Traces/chksums/ip6-hoa-udp-good-chksum.pcap new file mode 100644 index 0000000000..f3e9d632c3 Binary files /dev/null and b/testing/btest/Traces/chksums/ip6-hoa-udp-good-chksum.pcap differ diff --git a/testing/btest/Traces/chksums/ip6-icmp6-bad-chksum.pcap b/testing/btest/Traces/chksums/ip6-icmp6-bad-chksum.pcap new file mode 100644 index 0000000000..ce1dfa547a Binary files /dev/null and b/testing/btest/Traces/chksums/ip6-icmp6-bad-chksum.pcap differ diff --git a/testing/btest/Traces/chksums/ip6-icmp6-good-chksum.pcap b/testing/btest/Traces/chksums/ip6-icmp6-good-chksum.pcap new file mode 100644 index 0000000000..4051fa5bc5 Binary files /dev/null and b/testing/btest/Traces/chksums/ip6-icmp6-good-chksum.pcap differ diff --git a/testing/btest/Traces/chksums/ip6-route0-icmp6-bad-chksum.pcap b/testing/btest/Traces/chksums/ip6-route0-icmp6-bad-chksum.pcap new file mode 100644 index 0000000000..15e11ed326 Binary files /dev/null and b/testing/btest/Traces/chksums/ip6-route0-icmp6-bad-chksum.pcap differ diff --git a/testing/btest/Traces/chksums/ip6-route0-icmp6-good-chksum.pcap b/testing/btest/Traces/chksums/ip6-route0-icmp6-good-chksum.pcap new file mode 100644 index 0000000000..b7924cab6f Binary files /dev/null and b/testing/btest/Traces/chksums/ip6-route0-icmp6-good-chksum.pcap differ diff --git a/testing/btest/Traces/chksums/ip6-route0-tcp-bad-chksum.pcap b/testing/btest/Traces/chksums/ip6-route0-tcp-bad-chksum.pcap new file mode 100644 index 0000000000..0f5711fe2e Binary files /dev/null and b/testing/btest/Traces/chksums/ip6-route0-tcp-bad-chksum.pcap differ diff --git a/testing/btest/Traces/chksums/ip6-route0-tcp-good-chksum.pcap b/testing/btest/Traces/chksums/ip6-route0-tcp-good-chksum.pcap new file mode 100644 index 0000000000..18f9a366c6 Binary files /dev/null and b/testing/btest/Traces/chksums/ip6-route0-tcp-good-chksum.pcap differ diff --git a/testing/btest/Traces/chksums/ip6-route0-udp-bad-chksum.pcap b/testing/btest/Traces/chksums/ip6-route0-udp-bad-chksum.pcap new file mode 100644 index 0000000000..b4eecae7a2 Binary files /dev/null and b/testing/btest/Traces/chksums/ip6-route0-udp-bad-chksum.pcap differ diff --git a/testing/btest/Traces/chksums/ip6-route0-udp-good-chksum.pcap b/testing/btest/Traces/chksums/ip6-route0-udp-good-chksum.pcap new file mode 100644 index 0000000000..deb1310107 Binary files /dev/null and b/testing/btest/Traces/chksums/ip6-route0-udp-good-chksum.pcap differ diff --git a/testing/btest/Traces/chksums/ip6-tcp-bad-chksum.pcap b/testing/btest/Traces/chksums/ip6-tcp-bad-chksum.pcap new file mode 100644 index 0000000000..38d8abf18f Binary files /dev/null and b/testing/btest/Traces/chksums/ip6-tcp-bad-chksum.pcap differ diff --git a/testing/btest/Traces/chksums/ip6-tcp-good-chksum.pcap b/testing/btest/Traces/chksums/ip6-tcp-good-chksum.pcap new file mode 100644 index 0000000000..9ab19b0ad8 Binary files /dev/null and b/testing/btest/Traces/chksums/ip6-tcp-good-chksum.pcap differ diff --git a/testing/btest/Traces/chksums/ip6-udp-bad-chksum.pcap b/testing/btest/Traces/chksums/ip6-udp-bad-chksum.pcap new file mode 100644 index 0000000000..25aa3fc2c3 Binary files /dev/null and b/testing/btest/Traces/chksums/ip6-udp-bad-chksum.pcap differ diff --git a/testing/btest/Traces/chksums/ip6-udp-good-chksum.pcap b/testing/btest/Traces/chksums/ip6-udp-good-chksum.pcap new file mode 100644 index 0000000000..b72b86678e Binary files /dev/null and b/testing/btest/Traces/chksums/ip6-udp-good-chksum.pcap differ diff --git a/testing/btest/Traces/chksums/mip6-bad-mh-chksum.pcap b/testing/btest/Traces/chksums/mip6-bad-mh-chksum.pcap new file mode 100644 index 0000000000..9a2437baef Binary files /dev/null and b/testing/btest/Traces/chksums/mip6-bad-mh-chksum.pcap differ diff --git a/testing/btest/Traces/chksums/mip6-good-mh-chksum.pcap b/testing/btest/Traces/chksums/mip6-good-mh-chksum.pcap new file mode 100644 index 0000000000..6183fd9cb1 Binary files /dev/null and b/testing/btest/Traces/chksums/mip6-good-mh-chksum.pcap differ diff --git a/testing/btest/Traces/ftp-ipv4.trace b/testing/btest/Traces/ftp-ipv4.trace new file mode 100644 index 0000000000..02cac6f464 Binary files /dev/null and b/testing/btest/Traces/ftp-ipv4.trace differ diff --git a/testing/btest/Traces/icmp/icmp-destunreach-ip.pcap b/testing/btest/Traces/icmp/icmp-destunreach-ip.pcap new file mode 100644 index 0000000000..982f2e4734 Binary files /dev/null and b/testing/btest/Traces/icmp/icmp-destunreach-ip.pcap differ diff --git a/testing/btest/Traces/icmp/icmp-destunreach-no-context.pcap b/testing/btest/Traces/icmp/icmp-destunreach-no-context.pcap new file mode 100644 index 0000000000..1f904e3d91 Binary files /dev/null and b/testing/btest/Traces/icmp/icmp-destunreach-no-context.pcap differ diff --git a/testing/btest/Traces/icmp/icmp-destunreach-udp.pcap b/testing/btest/Traces/icmp/icmp-destunreach-udp.pcap new file mode 100644 index 0000000000..60137bb6fe Binary files /dev/null and b/testing/btest/Traces/icmp/icmp-destunreach-udp.pcap differ diff --git a/testing/btest/Traces/icmp/icmp-ping.pcap b/testing/btest/Traces/icmp/icmp-ping.pcap new file mode 100644 index 0000000000..499769b280 Binary files /dev/null and b/testing/btest/Traces/icmp/icmp-ping.pcap differ diff --git a/testing/btest/Traces/icmp/icmp-timeexceeded.pcap b/testing/btest/Traces/icmp/icmp-timeexceeded.pcap new file mode 100644 index 0000000000..27804b5559 Binary files /dev/null and b/testing/btest/Traces/icmp/icmp-timeexceeded.pcap differ diff --git a/testing/btest/Traces/icmp/icmp6-destunreach-ip6ext-trunc.pcap b/testing/btest/Traces/icmp/icmp6-destunreach-ip6ext-trunc.pcap new file mode 100644 index 0000000000..bd0e0ccd42 Binary files /dev/null and b/testing/btest/Traces/icmp/icmp6-destunreach-ip6ext-trunc.pcap differ diff --git a/testing/btest/Traces/icmp/icmp6-destunreach-ip6ext-udp.pcap b/testing/btest/Traces/icmp/icmp6-destunreach-ip6ext-udp.pcap new file mode 100644 index 0000000000..5aca9af1b5 Binary files /dev/null and b/testing/btest/Traces/icmp/icmp6-destunreach-ip6ext-udp.pcap differ diff --git a/testing/btest/Traces/icmp/icmp6-destunreach-ip6ext.pcap b/testing/btest/Traces/icmp/icmp6-destunreach-ip6ext.pcap new file mode 100644 index 0000000000..996048e5ab Binary files /dev/null and b/testing/btest/Traces/icmp/icmp6-destunreach-ip6ext.pcap differ diff --git a/testing/btest/Traces/icmp/icmp6-destunreach-no-context.pcap b/testing/btest/Traces/icmp/icmp6-destunreach-no-context.pcap new file mode 100644 index 0000000000..cf15a7cf65 Binary files /dev/null and b/testing/btest/Traces/icmp/icmp6-destunreach-no-context.pcap differ diff --git a/testing/btest/Traces/icmp/icmp6-neighbor-advert.pcap b/testing/btest/Traces/icmp/icmp6-neighbor-advert.pcap new file mode 100644 index 0000000000..0a06329fb5 Binary files /dev/null and b/testing/btest/Traces/icmp/icmp6-neighbor-advert.pcap differ diff --git a/testing/btest/Traces/icmp/icmp6-neighbor-solicit.pcap b/testing/btest/Traces/icmp/icmp6-neighbor-solicit.pcap new file mode 100644 index 0000000000..248bbae4ea Binary files /dev/null and b/testing/btest/Traces/icmp/icmp6-neighbor-solicit.pcap differ diff --git a/testing/btest/Traces/icmp/icmp6-paramprob.pcap b/testing/btest/Traces/icmp/icmp6-paramprob.pcap new file mode 100644 index 0000000000..ab2d41cd3a Binary files /dev/null and b/testing/btest/Traces/icmp/icmp6-paramprob.pcap differ diff --git a/testing/btest/Traces/icmp/icmp6-ping.pcap b/testing/btest/Traces/icmp/icmp6-ping.pcap new file mode 100644 index 0000000000..1638ca0822 Binary files /dev/null and b/testing/btest/Traces/icmp/icmp6-ping.pcap differ diff --git a/testing/btest/Traces/icmp/icmp6-redirect.pcap b/testing/btest/Traces/icmp/icmp6-redirect.pcap new file mode 100644 index 0000000000..f8ae7ed96b Binary files /dev/null and b/testing/btest/Traces/icmp/icmp6-redirect.pcap differ diff --git a/testing/btest/Traces/icmp/icmp6-router-advert.pcap b/testing/btest/Traces/icmp/icmp6-router-advert.pcap new file mode 100644 index 0000000000..38de434c2f Binary files /dev/null and b/testing/btest/Traces/icmp/icmp6-router-advert.pcap differ diff --git a/testing/btest/Traces/icmp/icmp6-router-solicit.pcap b/testing/btest/Traces/icmp/icmp6-router-solicit.pcap new file mode 100644 index 0000000000..b33495aa8d Binary files /dev/null and b/testing/btest/Traces/icmp/icmp6-router-solicit.pcap differ diff --git a/testing/btest/Traces/icmp/icmp6-timeexceeded.pcap b/testing/btest/Traces/icmp/icmp6-timeexceeded.pcap new file mode 100644 index 0000000000..b32fc4ab68 Binary files /dev/null and b/testing/btest/Traces/icmp/icmp6-timeexceeded.pcap differ diff --git a/testing/btest/Traces/icmp/icmp6-toobig.pcap b/testing/btest/Traces/icmp/icmp6-toobig.pcap new file mode 100644 index 0000000000..92bf50f240 Binary files /dev/null and b/testing/btest/Traces/icmp/icmp6-toobig.pcap differ diff --git a/testing/btest/Traces/ip6_esp.trace b/testing/btest/Traces/ip6_esp.trace new file mode 100644 index 0000000000..8b3b19a99a Binary files /dev/null and b/testing/btest/Traces/ip6_esp.trace differ diff --git a/testing/btest/Traces/ipv6-fragmented-dns.trace b/testing/btest/Traces/ipv6-fragmented-dns.trace new file mode 100755 index 0000000000..9dda47a8a9 Binary files /dev/null and b/testing/btest/Traces/ipv6-fragmented-dns.trace differ diff --git a/testing/btest/Traces/ipv6-ftp.trace b/testing/btest/Traces/ipv6-ftp.trace new file mode 100644 index 0000000000..81313fac11 Binary files /dev/null and b/testing/btest/Traces/ipv6-ftp.trace differ diff --git a/testing/btest/Traces/ipv6-hbh-routing0.trace b/testing/btest/Traces/ipv6-hbh-routing0.trace new file mode 100644 index 0000000000..2a294ed58e Binary files /dev/null and b/testing/btest/Traces/ipv6-hbh-routing0.trace differ diff --git a/testing/btest/Traces/ipv6-http-atomic-frag.trace b/testing/btest/Traces/ipv6-http-atomic-frag.trace new file mode 100644 index 0000000000..d5d9db276c Binary files /dev/null and b/testing/btest/Traces/ipv6-http-atomic-frag.trace differ diff --git a/testing/btest/Traces/mobile-ipv6/ipv6-mobile-hoa.trace b/testing/btest/Traces/mobile-ipv6/ipv6-mobile-hoa.trace new file mode 100644 index 0000000000..f3e9d632c3 Binary files /dev/null and b/testing/btest/Traces/mobile-ipv6/ipv6-mobile-hoa.trace differ diff --git a/testing/btest/Traces/mobile-ipv6/ipv6-mobile-routing.trace b/testing/btest/Traces/mobile-ipv6/ipv6-mobile-routing.trace new file mode 100644 index 0000000000..6289f268e3 Binary files /dev/null and b/testing/btest/Traces/mobile-ipv6/ipv6-mobile-routing.trace differ diff --git a/testing/btest/Traces/mobile-ipv6/mip6_back.trace b/testing/btest/Traces/mobile-ipv6/mip6_back.trace new file mode 100644 index 0000000000..9b97186979 Binary files /dev/null and b/testing/btest/Traces/mobile-ipv6/mip6_back.trace differ diff --git a/testing/btest/Traces/mobile-ipv6/mip6_be.trace b/testing/btest/Traces/mobile-ipv6/mip6_be.trace new file mode 100644 index 0000000000..19862ee4be Binary files /dev/null and b/testing/btest/Traces/mobile-ipv6/mip6_be.trace differ diff --git a/testing/btest/Traces/mobile-ipv6/mip6_brr.trace b/testing/btest/Traces/mobile-ipv6/mip6_brr.trace new file mode 100644 index 0000000000..4020ae8b14 Binary files /dev/null and b/testing/btest/Traces/mobile-ipv6/mip6_brr.trace differ diff --git a/testing/btest/Traces/mobile-ipv6/mip6_bu.trace b/testing/btest/Traces/mobile-ipv6/mip6_bu.trace new file mode 100644 index 0000000000..1c8c61e09d Binary files /dev/null and b/testing/btest/Traces/mobile-ipv6/mip6_bu.trace differ diff --git a/testing/btest/Traces/mobile-ipv6/mip6_cot.trace b/testing/btest/Traces/mobile-ipv6/mip6_cot.trace new file mode 100644 index 0000000000..2d8d215a41 Binary files /dev/null and b/testing/btest/Traces/mobile-ipv6/mip6_cot.trace differ diff --git a/testing/btest/Traces/mobile-ipv6/mip6_coti.trace b/testing/btest/Traces/mobile-ipv6/mip6_coti.trace new file mode 100644 index 0000000000..2a5790cc7c Binary files /dev/null and b/testing/btest/Traces/mobile-ipv6/mip6_coti.trace differ diff --git a/testing/btest/Traces/mobile-ipv6/mip6_hot.trace b/testing/btest/Traces/mobile-ipv6/mip6_hot.trace new file mode 100644 index 0000000000..0b54c9797d Binary files /dev/null and b/testing/btest/Traces/mobile-ipv6/mip6_hot.trace differ diff --git a/testing/btest/Traces/mobile-ipv6/mip6_hoti.trace b/testing/btest/Traces/mobile-ipv6/mip6_hoti.trace new file mode 100644 index 0000000000..3daaeb2905 Binary files /dev/null and b/testing/btest/Traces/mobile-ipv6/mip6_hoti.trace differ diff --git a/testing/btest/Traces/trunc/ip4-trunc.pcap b/testing/btest/Traces/trunc/ip4-trunc.pcap new file mode 100644 index 0000000000..30df0ea94d Binary files /dev/null and b/testing/btest/Traces/trunc/ip4-trunc.pcap differ diff --git a/testing/btest/Traces/trunc/ip6-ext-trunc.pcap b/testing/btest/Traces/trunc/ip6-ext-trunc.pcap new file mode 100644 index 0000000000..1de659084e Binary files /dev/null and b/testing/btest/Traces/trunc/ip6-ext-trunc.pcap differ diff --git a/testing/btest/Traces/trunc/ip6-trunc.pcap b/testing/btest/Traces/trunc/ip6-trunc.pcap new file mode 100644 index 0000000000..0111caed0f Binary files /dev/null and b/testing/btest/Traces/trunc/ip6-trunc.pcap differ diff --git a/testing/btest/analyzers/conn-size.bro b/testing/btest/analyzers/conn-size.bro index 0e413cc554..0ba7977cf5 100644 --- a/testing/btest/analyzers/conn-size.bro +++ b/testing/btest/analyzers/conn-size.bro @@ -1,2 +1,2 @@ -# @TEST-EXEC: bro -C -r ${TRACES}/conn-size.trace tcp udp icmp report_conn_size_analyzer=T use_connection_compressor=F +# @TEST-EXEC: bro -C -r ${TRACES}/conn-size.trace tcp udp icmp report_conn_size_analyzer=T # @TEST-EXEC: btest-diff conn.log diff --git a/testing/btest/bifs/addr_count_conversion.bro b/testing/btest/bifs/addr_count_conversion.bro new file mode 100644 index 0000000000..360994a8e5 --- /dev/null +++ b/testing/btest/bifs/addr_count_conversion.bro @@ -0,0 +1,11 @@ +# @TEST-EXEC: bro %INPUT >output +# @TEST-EXEC: btest-diff output + +global v: index_vec; + +v = addr_to_counts([2001:0db8:85a3:0000:0000:8a2e:0370:7334]); +print v; +print counts_to_addr(v); +v = addr_to_counts(1.2.3.4); +print v; +print counts_to_addr(v); diff --git a/testing/btest/bifs/addr_to_ptr_name.bro b/testing/btest/bifs/addr_to_ptr_name.bro new file mode 100644 index 0000000000..b9c831d061 --- /dev/null +++ b/testing/btest/bifs/addr_to_ptr_name.bro @@ -0,0 +1,6 @@ +# @TEST-EXEC: bro %INPUT >output +# @TEST-EXEC: btest-diff output + +print addr_to_ptr_name([2607:f8b0:4009:802::1012]); +print addr_to_ptr_name(74.125.225.52); + diff --git a/testing/btest/bifs/addr_version.bro b/testing/btest/bifs/addr_version.bro new file mode 100644 index 0000000000..3e0123ef42 --- /dev/null +++ b/testing/btest/bifs/addr_version.bro @@ -0,0 +1,7 @@ +# @TEST-EXEC: bro %INPUT >out +# @TEST-EXEC: btest-diff out + +print is_v4_addr(1.2.3.4); +print is_v4_addr([::1]); +print is_v6_addr(1.2.3.4); +print is_v6_addr([::1]); diff --git a/testing/btest/bifs/install_src_addr_filter.test b/testing/btest/bifs/install_src_addr_filter.test new file mode 100644 index 0000000000..5b387832de --- /dev/null +++ b/testing/btest/bifs/install_src_addr_filter.test @@ -0,0 +1,13 @@ +# @TEST-EXEC: bro -C -r $TRACES/wikipedia.trace %INPUT >output +# @TEST-EXEC: btest-diff output + +event bro_init() + { + install_src_addr_filter(141.142.220.118, TH_SYN, 100.0); + } + +event new_packet(c: connection, p: pkt_hdr) + { + if ( p?$tcp && p$ip$src == 141.142.220.118 ) + print c$id; + } diff --git a/testing/btest/bifs/ptr_name_to_addr.bro b/testing/btest/bifs/ptr_name_to_addr.bro new file mode 100644 index 0000000000..89679ba57a --- /dev/null +++ b/testing/btest/bifs/ptr_name_to_addr.bro @@ -0,0 +1,8 @@ +# @TEST-EXEC: bro %INPUT >output +# @TEST-EXEC: btest-diff output + +global v6 = ptr_name_to_addr("2.1.0.1.0.0.0.0.0.0.0.0.0.0.0.0.2.0.8.0.9.0.0.4.0.b.8.f.7.0.6.2.ip6.arpa"); +global v4 = ptr_name_to_addr("52.225.125.74.in-addr.arpa"); + +print v6; +print v4; \ No newline at end of file diff --git a/testing/btest/bifs/remask_addr.bro b/testing/btest/bifs/remask_addr.bro new file mode 100644 index 0000000000..d387667b6a --- /dev/null +++ b/testing/btest/bifs/remask_addr.bro @@ -0,0 +1,10 @@ +# @TEST-EXEC: bro %INPUT >output +# @TEST-EXEC: btest-diff output + +const one_to_32: vector of count = {1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32}; + +for ( i in one_to_32 ) + { + print fmt("%s: %s", one_to_32[i], + remask_addr(0.0.255.255, 255.255.0.0, 96+one_to_32[i])); + } diff --git a/testing/btest/bifs/routing0_data_to_addrs.test b/testing/btest/bifs/routing0_data_to_addrs.test new file mode 100644 index 0000000000..a20bb3bf59 --- /dev/null +++ b/testing/btest/bifs/routing0_data_to_addrs.test @@ -0,0 +1,10 @@ +# @TEST-EXEC: bro -b -r $TRACES/ipv6-hbh-routing0.trace %INPUT >output +# @TEST-EXEC: btest-diff output + +event ipv6_ext_headers(c: connection, p: pkt_hdr) + { + for ( h in p$ip6$exts ) + if ( p$ip6$exts[h]$id == IPPROTO_ROUTING ) + if ( p$ip6$exts[h]$routing$rtype == 0 ) + print routing0_data_to_addrs(p$ip6$exts[h]$routing$data); + } diff --git a/testing/btest/bifs/to_addr.bro b/testing/btest/bifs/to_addr.bro new file mode 100644 index 0000000000..3a43438bb7 --- /dev/null +++ b/testing/btest/bifs/to_addr.bro @@ -0,0 +1,20 @@ +# @TEST-EXEC: bro -b %INPUT >output 2>error +# @TEST-EXEC: btest-diff output +# @TEST-EXEC: btest-diff error + +function test_to_addr(ip: string, expect: addr) + { + local result = to_addr(ip); + print fmt("to_addr(%s) = %s (%s)", ip, result, + result == expect ? "SUCCESS" : "FAILURE"); + } + +test_to_addr("0.0.0.0", 0.0.0.0); +test_to_addr("1.2.3.4", 1.2.3.4); +test_to_addr("01.02.03.04", 1.2.3.4); +test_to_addr("001.002.003.004", 1.2.3.4); +test_to_addr("10.20.30.40", 10.20.30.40); +test_to_addr("100.200.30.40", 100.200.30.40); +test_to_addr("10.0.0.0", 10.0.0.0); +test_to_addr("10.00.00.000", 10.0.0.0); +test_to_addr("not an IP", [::]); diff --git a/testing/btest/bifs/to_subnet.bro b/testing/btest/bifs/to_subnet.bro new file mode 100644 index 0000000000..59064893e1 --- /dev/null +++ b/testing/btest/bifs/to_subnet.bro @@ -0,0 +1,11 @@ +# @TEST-EXEC: bro -b %INPUT >output 2>error +# @TEST-EXEC: btest-diff output +# @TEST-EXEC: btest-diff error + +global sn: subnet; +sn = to_subnet("10.0.0.0/8"); +print sn, sn == 10.0.0.0/8; +sn = to_subnet("2607:f8b0::/32"); +print sn, sn == [2607:f8b0::]/32; +sn = to_subnet("10.0.0.0"); +print sn, sn == [::]/0; diff --git a/testing/btest/btest.cfg b/testing/btest/btest.cfg index 2126e733e7..6afbde1ddb 100644 --- a/testing/btest/btest.cfg +++ b/testing/btest/btest.cfg @@ -18,4 +18,4 @@ DIST=%(testbase)s/../.. BUILD=%(testbase)s/../../build TEST_DIFF_CANONIFIER=$SCRIPTS/diff-canonifier TMPDIR=%(testbase)s/.tmp -BRO_PROFILER_FILE=%(testbase)s/.tmp/script-coverage +BRO_PROFILER_FILE=%(testbase)s/.tmp/script-coverage.XXXXXX diff --git a/testing/btest/core/checksums.test b/testing/btest/core/checksums.test new file mode 100644 index 0000000000..f5b3230686 --- /dev/null +++ b/testing/btest/core/checksums.test @@ -0,0 +1,23 @@ +# @TEST-EXEC: bro -b -r $TRACES/chksums/ip4-bad-chksum.pcap >>bad.out 2>&1 +# @TEST-EXEC: bro -b -r $TRACES/chksums/ip4-tcp-bad-chksum.pcap >>bad.out 2>&1 +# @TEST-EXEC: bro -b -r $TRACES/chksums/ip4-udp-bad-chksum.pcap >>bad.out 2>&1 +# @TEST-EXEC: bro -b -r $TRACES/chksums/ip4-icmp-bad-chksum.pcap >>bad.out 2>&1 +# @TEST-EXEC: bro -b -r $TRACES/chksums/ip6-route0-tcp-bad-chksum.pcap >>bad.out 2>&1 +# @TEST-EXEC: bro -b -r $TRACES/chksums/ip6-route0-udp-bad-chksum.pcap >>bad.out 2>&1 +# @TEST-EXEC: bro -b -r $TRACES/chksums/ip6-route0-icmp6-bad-chksum.pcap >>bad.out 2>&1 +# @TEST-EXEC: bro -b -r $TRACES/chksums/ip6-tcp-bad-chksum.pcap >>bad.out 2>&1 +# @TEST-EXEC: bro -b -r $TRACES/chksums/ip6-udp-bad-chksum.pcap >>bad.out 2>&1 +# @TEST-EXEC: bro -b -r $TRACES/chksums/ip6-icmp6-bad-chksum.pcap >>bad.out 2>&1 + + +# @TEST-EXEC: bro -b -r $TRACES/chksums/ip4-tcp-good-chksum.pcap >>good.out 2>&1 +# @TEST-EXEC: bro -b -r $TRACES/chksums/ip4-udp-good-chksum.pcap >>good.out 2>&1 +# @TEST-EXEC: bro -b -r $TRACES/chksums/ip4-icmp-good-chksum.pcap >>good.out 2>&1 +# @TEST-EXEC: bro -b -r $TRACES/chksums/ip6-route0-tcp-good-chksum.pcap >>good.out 2>&1 +# @TEST-EXEC: bro -b -r $TRACES/chksums/ip6-route0-udp-good-chksum.pcap >>good.out 2>&1 +# @TEST-EXEC: bro -b -r $TRACES/chksums/ip6-route0-icmp6-good-chksum.pcap >>good.out 2>&1 +# @TEST-EXEC: bro -b -r $TRACES/chksums/ip6-tcp-good-chksum.pcap >>good.out 2>&1 +# @TEST-EXEC: bro -b -r $TRACES/chksums/ip6-udp-good-chksum.pcap >>good.out 2>&1 +# @TEST-EXEC: bro -b -r $TRACES/chksums/ip6-icmp6-good-chksum.pcap >>good.out 2>&1 +# @TEST-EXEC: btest-diff bad.out +# @TEST-EXEC: btest-diff good.out diff --git a/testing/btest/core/conn-uid.bro b/testing/btest/core/conn-uid.bro index b2078bc9f5..52ff8fc4d3 100644 --- a/testing/btest/core/conn-uid.bro +++ b/testing/btest/core/conn-uid.bro @@ -9,17 +9,6 @@ # @TEST-EXEC: unset BRO_SEED_FILE && bro -C -r $TRACES/wikipedia.trace %INPUT >output2 # @TEST-EXEC: cat output output2 | sort | uniq -c | wc -l | sed 's/ //g' >counts # @TEST-EXEC: btest-diff counts -# -# Make sure it works without the connection compressor as well. -# -# @TEST-EXEC: bro -C -r $TRACES/wikipedia.trace %INPUT use_connection_compressor=F >output.cc -# @TEST-EXEC: btest-diff output.cc -# -# Make sure it works with the full connection compressor as well. -# -# @TEST-EXEC: bro -C -r $TRACES/wikipedia.trace %INPUT cc_handle_only_syns=F >output.cc2 -# @TEST-EXEC: btest-diff output.cc2 - event new_connection(c: connection) { diff --git a/testing/btest/core/disable-mobile-ipv6.test b/testing/btest/core/disable-mobile-ipv6.test new file mode 100644 index 0000000000..84dc43dae8 --- /dev/null +++ b/testing/btest/core/disable-mobile-ipv6.test @@ -0,0 +1,12 @@ +# @TEST-REQUIRES: grep -q "#undef ENABLE_MOBILE_IPV6" $BUILD/config.h +# @TEST-EXEC: bro -b -r $TRACES/mobile-ipv6/mip6_back.trace %INPUT >output 2>&1 +# @TEST-EXEC: btest-diff output + +event mobile_ipv6_message(p: pkt_hdr) + { + if ( ! p?$ip6 ) return; + + for ( i in p$ip6$exts ) + if ( p$ip6$exts[i]$id == IPPROTO_MOBILITY ) + print p$ip6; + } diff --git a/testing/btest/core/discarder.bro b/testing/btest/core/discarder.bro new file mode 100644 index 0000000000..0c87eece18 --- /dev/null +++ b/testing/btest/core/discarder.bro @@ -0,0 +1,92 @@ +# @TEST-EXEC: bro -C -r $TRACES/wikipedia.trace discarder-ip.bro >output +# @TEST-EXEC: bro -C -r $TRACES/wikipedia.trace discarder-tcp.bro >>output +# @TEST-EXEC: bro -C -r $TRACES/wikipedia.trace discarder-udp.bro >>output +# @TEST-EXEC: bro -C -r $TRACES/icmp/icmp-destunreach-udp.pcap discarder-icmp.bro >>output +# @TEST-EXEC: btest-diff output + +@TEST-START-FILE discarder-ip.bro + +event bro_init() + { + print "################ IP Discarder ################"; + } + +function discarder_check_ip(p: pkt_hdr): bool + { + if ( p?$ip && p$ip$src == 141.142.220.118 && p$ip$dst == 208.80.152.2 ) + return F; + return T; + } + + +event new_packet(c: connection, p: pkt_hdr) + { + print c$id; + } + +@TEST-END-FILE + +@TEST-START-FILE discarder-tcp.bro + +event bro_init() + { + print "################ TCP Discarder ################"; + } + +function discarder_check_tcp(p: pkt_hdr, d: string): bool + { + if ( p$tcp$flags == TH_SYN ) + return F; + return T; + } + +event new_packet(c: connection, p: pkt_hdr) + { + if ( p?$tcp ) + print c$id; + } + +@TEST-END-FILE + +@TEST-START-FILE discarder-udp.bro + +event bro_init() + { + print "################ UDP Discarder ################"; + } + +function discarder_check_udp(p: pkt_hdr, d: string): bool + { + if ( p?$ip6 ) + return F; + return T; + } + +event new_packet(c: connection, p: pkt_hdr) + { + if ( p?$udp ) + print c$id; + } + +@TEST-END-FILE + +@TEST-START-FILE discarder-icmp.bro + +event bro_init() + { + print "################ ICMP Discarder ################"; + } + +function discarder_check_icmp(p: pkt_hdr): bool + { + print fmt("Discard icmp packet: %s", p$icmp); + return T; + } + +event new_packet(c: connection, p: pkt_hdr) + { + if ( p?$icmp ) + print c$id; + } + +@TEST-END-FILE diff --git a/testing/btest/core/icmp/icmp-context.test b/testing/btest/core/icmp/icmp-context.test new file mode 100644 index 0000000000..ca7a34c5aa --- /dev/null +++ b/testing/btest/core/icmp/icmp-context.test @@ -0,0 +1,14 @@ +# These tests all check that IPv6 context packet construction for ICMP6 works. + +# @TEST-EXEC: bro -b -r $TRACES/icmp/icmp-destunreach-no-context.pcap %INPUT >>output 2>&1 +# @TEST-EXEC: bro -b -r $TRACES/icmp/icmp-destunreach-ip.pcap %INPUT >>output 2>&1 +# @TEST-EXEC: bro -b -r $TRACES/icmp/icmp-destunreach-udp.pcap %INPUT >>output 2>&1 +# @TEST-EXEC: btest-diff output + +event icmp_unreachable(c: connection, icmp: icmp_conn, code: count, context: icmp_context) + { + print "icmp_unreachable (code=" + fmt("%d", code) + ")"; + print " conn_id: " + fmt("%s", c$id); + print " icmp_conn: " + fmt("%s", icmp); + print " icmp_context: " + fmt("%s", context); + } diff --git a/testing/btest/core/icmp/icmp-events.test b/testing/btest/core/icmp/icmp-events.test new file mode 100644 index 0000000000..1a54f05fba --- /dev/null +++ b/testing/btest/core/icmp/icmp-events.test @@ -0,0 +1,44 @@ +# These tests all check that ICMP6 events get raised with correct arguments. + +# @TEST-EXEC: bro -b -r $TRACES/icmp/icmp-destunreach-udp.pcap %INPUT >>output 2>&1 +# @TEST-EXEC: bro -b -r $TRACES/icmp/icmp-timeexceeded.pcap %INPUT >>output 2>&1 +# @TEST-EXEC: bro -b -r $TRACES/icmp/icmp-ping.pcap %INPUT >>output 2>&1 + +# @TEST-EXEC: btest-diff output + +event icmp_sent(c: connection, icmp: icmp_conn) + { + print "icmp_sent"; + print " conn_id: " + fmt("%s", c$id); + print " icmp_conn: " + fmt("%s", icmp); + } + +event icmp_echo_request(c: connection, icmp: icmp_conn, id: count, seq: count, payload: string) + { + print "icmp_echo_request (id=" + fmt("%d", id) + ", seq=" + fmt("%d", seq) + ", payload=" + payload + ")"; + print " conn_id: " + fmt("%s", c$id); + print " icmp_conn: " + fmt("%s", icmp); + } + +event icmp_echo_reply(c: connection, icmp: icmp_conn, id: count, seq: count, payload: string) + { + print "icmp_echo_reply (id=" + fmt("%d", id) + ", seq=" + fmt("%d", seq) + ", payload=" + payload + ")"; + print " conn_id: " + fmt("%s", c$id); + print " icmp_conn: " + fmt("%s", icmp); + } + +event icmp_unreachable(c: connection, icmp: icmp_conn, code: count, context: icmp_context) + { + print "icmp_unreachable (code=" + fmt("%d", code) + ")"; + print " conn_id: " + fmt("%s", c$id); + print " icmp_conn: " + fmt("%s", icmp); + print " icmp_context: " + fmt("%s", context); + } + +event icmp_time_exceeded(c: connection, icmp: icmp_conn, code: count, context: icmp_context) + { + print "icmp_time_exceeded (code=" + fmt("%d", code) + ")"; + print " conn_id: " + fmt("%s", c$id); + print " icmp_conn: " + fmt("%s", icmp); + print " icmp_context: " + fmt("%s", context); + } diff --git a/testing/btest/core/icmp/icmp6-context.test b/testing/btest/core/icmp/icmp6-context.test new file mode 100644 index 0000000000..dfa8271cbc --- /dev/null +++ b/testing/btest/core/icmp/icmp6-context.test @@ -0,0 +1,15 @@ +# These tests all check that IPv6 context packet construction for ICMP6 works. + +# @TEST-EXEC: bro -b -r $TRACES/icmp/icmp6-destunreach-no-context.pcap %INPUT >>output 2>&1 +# @TEST-EXEC: bro -b -r $TRACES/icmp/icmp6-destunreach-ip6ext-trunc.pcap %INPUT >>output 2>&1 +# @TEST-EXEC: bro -b -r $TRACES/icmp/icmp6-destunreach-ip6ext-udp.pcap %INPUT >>output 2>&1 +# @TEST-EXEC: bro -b -r $TRACES/icmp/icmp6-destunreach-ip6ext.pcap %INPUT >>output 2>&1 +# @TEST-EXEC: btest-diff output + +event icmp_unreachable(c: connection, icmp: icmp_conn, code: count, context: icmp_context) + { + print "icmp_unreachable (code=" + fmt("%d", code) + ")"; + print " conn_id: " + fmt("%s", c$id); + print " icmp_conn: " + fmt("%s", icmp); + print " icmp_context: " + fmt("%s", context); + } diff --git a/testing/btest/core/icmp/icmp6-events.test b/testing/btest/core/icmp/icmp6-events.test new file mode 100644 index 0000000000..052ba91ee6 --- /dev/null +++ b/testing/btest/core/icmp/icmp6-events.test @@ -0,0 +1,123 @@ +# These tests all check that ICMP6 events get raised with correct arguments. + +# @TEST-EXEC: bro -b -r $TRACES/icmp/icmp6-destunreach-ip6ext-udp.pcap %INPUT >>output 2>&1 +# @TEST-EXEC: bro -b -r $TRACES/icmp/icmp6-toobig.pcap %INPUT >>output 2>&1 +# @TEST-EXEC: bro -b -r $TRACES/icmp/icmp6-timeexceeded.pcap %INPUT >>output 2>&1 +# @TEST-EXEC: bro -b -r $TRACES/icmp/icmp6-paramprob.pcap %INPUT >>output 2>&1 +# @TEST-EXEC: bro -b -r $TRACES/icmp/icmp6-ping.pcap %INPUT >>output 2>&1 +# @TEST-EXEC: bro -b -r $TRACES/icmp/icmp6-redirect.pcap %INPUT >>output 2>&1 +# @TEST-EXEC: bro -b -r $TRACES/icmp/icmp6-router-advert.pcap %INPUT >>output 2>&1 +# @TEST-EXEC: bro -b -r $TRACES/icmp/icmp6-neighbor-advert.pcap %INPUT >>output 2>&1 +# @TEST-EXEC: bro -b -r $TRACES/icmp/icmp6-router-solicit.pcap %INPUT >>output 2>&1 +# @TEST-EXEC: bro -b -r $TRACES/icmp/icmp6-neighbor-solicit.pcap %INPUT >>output 2>&1 + +# @TEST-EXEC: btest-diff output + +event icmp_sent(c: connection, icmp: icmp_conn) + { + print "icmp_sent"; + print " conn_id: " + fmt("%s", c$id); + print " icmp_conn: " + fmt("%s", icmp); + } + +event icmp_echo_request(c: connection, icmp: icmp_conn, id: count, seq: count, payload: string) + { + print "icmp_echo_request (id=" + fmt("%d", id) + ", seq=" + fmt("%d", seq) + ", payload=" + payload + ")"; + print " conn_id: " + fmt("%s", c$id); + print " icmp_conn: " + fmt("%s", icmp); + } + +event icmp_echo_reply(c: connection, icmp: icmp_conn, id: count, seq: count, payload: string) + { + print "icmp_echo_reply (id=" + fmt("%d", id) + ", seq=" + fmt("%d", seq) + ", payload=" + payload + ")"; + print " conn_id: " + fmt("%s", c$id); + print " icmp_conn: " + fmt("%s", icmp); + } + +event icmp_unreachable(c: connection, icmp: icmp_conn, code: count, context: icmp_context) + { + print "icmp_unreachable (code=" + fmt("%d", code) + ")"; + print " conn_id: " + fmt("%s", c$id); + print " icmp_conn: " + fmt("%s", icmp); + print " icmp_context: " + fmt("%s", context); + } + +event icmp_packet_too_big(c: connection, icmp: icmp_conn, code: count, context: icmp_context) + { + print "icmp_packet_too_big (code=" + fmt("%d", code) + ")"; + print " conn_id: " + fmt("%s", c$id); + print " icmp_conn: " + fmt("%s", icmp); + print " icmp_context: " + fmt("%s", context); + } + +event icmp_time_exceeded(c: connection, icmp: icmp_conn, code: count, context: icmp_context) + { + print "icmp_time_exceeded (code=" + fmt("%d", code) + ")"; + print " conn_id: " + fmt("%s", c$id); + print " icmp_conn: " + fmt("%s", icmp); + print " icmp_context: " + fmt("%s", context); + } + +event icmp_parameter_problem(c: connection, icmp: icmp_conn, code: count, context: icmp_context) + { + print "icmp_parameter_problem (code=" + fmt("%d", code) + ")"; + print " conn_id: " + fmt("%s", c$id); + print " icmp_conn: " + fmt("%s", icmp); + print " icmp_context: " + fmt("%s", context); + } + +event icmp_redirect(c: connection, icmp: icmp_conn, tgt: addr, dest: addr) + { + print "icmp_redirect (tgt=" + fmt("%s", tgt) + ", dest=" + fmt("%s", dest) + ")"; + print " conn_id: " + fmt("%s", c$id); + print " icmp_conn: " + fmt("%s", icmp); + } + +event icmp_error_message(c: connection, icmp: icmp_conn, code: count, context: icmp_context) + { + print "icmp_error_message (code=" + fmt("%d", code) + ")"; + print " conn_id: " + fmt("%s", c$id); + print " icmp_conn: " + fmt("%s", icmp); + print " icmp_context: " + fmt("%s", context); + } + +event icmp_neighbor_solicitation(c: connection, icmp: icmp_conn, tgt: addr) + { + print "icmp_neighbor_solicitation (tgt=" + fmt("%s", tgt) + ")"; + print " conn_id: " + fmt("%s", c$id); + print " icmp_conn: " + fmt("%s", icmp); + } + +event icmp_neighbor_advertisement(c: connection, icmp: icmp_conn, router: bool, solicited: bool, override: bool, tgt: addr) + { + print "icmp_neighbor_advertisement (tgt=" + fmt("%s", tgt) + ")"; + print " router=" + fmt("%s", router); + print " solicited=" + fmt("%s", solicited); + print " override=" + fmt("%s", override); + print " conn_id: " + fmt("%s", c$id); + print " icmp_conn: " + fmt("%s", icmp); + } + +event icmp_router_solicitation(c: connection, icmp: icmp_conn) + { + print "icmp_router_solicitation"; + print " conn_id: " + fmt("%s", c$id); + print " icmp_conn: " + fmt("%s", icmp); + } + +event icmp_router_advertisement(c: connection, icmp: icmp_conn, cur_hop_limit: count, managed: bool, other: bool, home_agent: bool, pref: count, proxy: bool, rsv: count, router_lifetime: interval, reachable_time: interval, retrans_timer: interval) + { + print "icmp_router_advertisement"; + print " cur_hop_limit=" + fmt("%s", cur_hop_limit); + print " managed=" + fmt("%s", managed); + print " other=" + fmt("%s", other); + print " home_agent=" + fmt("%s", home_agent); + print " pref=" + fmt("%s", pref); + print " proxy=" + fmt("%s", proxy); + print " rsv=" + fmt("%s", rsv); + print " router_lifetime=" + fmt("%s", router_lifetime); + print " reachable_time=" + fmt("%s", reachable_time); + print " retrans_timer=" + fmt("%s", retrans_timer); + print " conn_id: " + fmt("%s", c$id); + print " icmp_conn: " + fmt("%s", icmp); + } diff --git a/testing/btest/core/ipv6-atomic-frag.test b/testing/btest/core/ipv6-atomic-frag.test new file mode 100644 index 0000000000..8c8fe6ca64 --- /dev/null +++ b/testing/btest/core/ipv6-atomic-frag.test @@ -0,0 +1,8 @@ +# @TEST-EXEC: bro -r $TRACES/ipv6-http-atomic-frag.trace %INPUT >output +# @TEST-EXEC: btest-diff output + +event new_connection(c: connection) + { + if ( c$id$resp_p == 80/tcp ) + print c$id; + } diff --git a/testing/btest/core/ipv6-frag.test b/testing/btest/core/ipv6-frag.test new file mode 100644 index 0000000000..32c7c0a8c1 --- /dev/null +++ b/testing/btest/core/ipv6-frag.test @@ -0,0 +1,9 @@ +# @TEST-EXEC: bro -r $TRACES/ipv6-fragmented-dns.trace %INPUT >output +# @TEST-EXEC: btest-diff output +# @TEST-EXEC: btest-diff dns.log + +event new_packet(c: connection, p: pkt_hdr) + { + if ( p?$ip6 && p?$ udp ) + print fmt("ip6=%s, udp = %s", p$ip6, p$udp); + } diff --git a/testing/btest/core/ipv6_esp.test b/testing/btest/core/ipv6_esp.test new file mode 100644 index 0000000000..8744df0036 --- /dev/null +++ b/testing/btest/core/ipv6_esp.test @@ -0,0 +1,11 @@ +# @TEST-EXEC: bro -r $TRACES/ip6_esp.trace %INPUT >output +# @TEST-EXEC: btest-diff output + +# Just check that the event is raised correctly for a packet containing +# ESP extension headers. + +event esp_packet(p: pkt_hdr) + { + if ( p?$ip6 ) + print p$ip6; + } diff --git a/testing/btest/core/ipv6_ext_headers.test b/testing/btest/core/ipv6_ext_headers.test new file mode 100644 index 0000000000..32a0f5d558 --- /dev/null +++ b/testing/btest/core/ipv6_ext_headers.test @@ -0,0 +1,22 @@ +# @TEST-EXEC: bro -b -r $TRACES/ipv6-hbh-routing0.trace %INPUT >output +# @TEST-EXEC: btest-diff output + +# Just check that the event is raised correctly for a packet containing +# extension headers. +event ipv6_ext_headers(c: connection, p: pkt_hdr) + { + print p; + } + +# Also check the weird for routing type 0 extensions headers +event flow_weird(name: string, src: addr, dst: addr) + { + print fmt("weird %s from %s to %s", name, src, dst); + } + +# And the connection for routing type 0 packets with non-zero segments left +# should use the last address in that extension header. +event new_connection(c: connection) + { + print c$id; + } diff --git a/testing/btest/core/leaks/basic-cluster.bro b/testing/btest/core/leaks/basic-cluster.bro new file mode 100644 index 0000000000..f5b40c1104 --- /dev/null +++ b/testing/btest/core/leaks/basic-cluster.bro @@ -0,0 +1,42 @@ +# Needs perftools support. +# +# @TEST-GROUP: leaks + +# @TEST-REQUIRES: bro --help 2>&1 | grep -q mem-leaks + +# @TEST-EXEC: btest-bg-run manager-1 HEAP_CHECK_DUMP_DIRECTORY=. HEAPCHECK=local BROPATH=$BROPATH:.. CLUSTER_NODE=manager-1 bro -m %INPUT +# @TEST-EXEC: btest-bg-run proxy-1 HEAP_CHECK_DUMP_DIRECTORY=. HEAPCHECK=local BROPATH=$BROPATH:.. CLUSTER_NODE=proxy-1 bro -m %INPUT +# @TEST-EXEC: sleep 1 +# @TEST-EXEC: btest-bg-run worker-1 HEAP_CHECK_DUMP_DIRECTORY=. HEAPCHECK=local BROPATH=$BROPATH:.. CLUSTER_NODE=worker-1 bro -m -r $TRACES/web.trace --pseudo-realtime %INPUT +# @TEST-EXEC: btest-bg-run worker-2 HEAP_CHECK_DUMP_DIRECTORY=. HEAPCHECK=local BROPATH=$BROPATH:.. CLUSTER_NODE=worker-2 bro -m -r $TRACES/web.trace --pseudo-realtime %INPUT +# @TEST-EXEC: btest-bg-wait -k 30 +# @TEST-EXEC: btest-diff manager-1/metrics.log + +@TEST-START-FILE cluster-layout.bro +redef Cluster::nodes = { + ["manager-1"] = [$node_type=Cluster::MANAGER, $ip=127.0.0.1, $p=37757/tcp, $workers=set("worker-1")], + ["proxy-1"] = [$node_type=Cluster::PROXY, $ip=127.0.0.1, $p=37758/tcp, $manager="manager-1", $workers=set("worker-1")], + ["worker-1"] = [$node_type=Cluster::WORKER, $ip=127.0.0.1, $p=37760/tcp, $manager="manager-1", $proxy="proxy-1", $interface="eth0"], + ["worker-2"] = [$node_type=Cluster::WORKER, $ip=127.0.0.1, $p=37761/tcp, $manager="manager-1", $proxy="proxy-1", $interface="eth1"], +}; +@TEST-END-FILE + +redef Log::default_rotation_interval = 0secs; + +redef enum Metrics::ID += { + TEST_METRIC, +}; + +event bro_init() &priority=5 + { + Metrics::add_filter(TEST_METRIC, + [$name="foo-bar", + $break_interval=3secs]); + + if ( Cluster::local_node_type() == Cluster::WORKER ) + { + Metrics::add_data(TEST_METRIC, [$host=1.2.3.4], 3); + Metrics::add_data(TEST_METRIC, [$host=6.5.4.3], 2); + Metrics::add_data(TEST_METRIC, [$host=7.2.1.5], 1); + } + } diff --git a/testing/btest/core/leaks/dns.bro b/testing/btest/core/leaks/dns.bro index 3d3fdc6f09..2816750758 100644 --- a/testing/btest/core/leaks/dns.bro +++ b/testing/btest/core/leaks/dns.bro @@ -1,5 +1,7 @@ # Needs perftools support. # +# @TEST-GROUP: leaks +# # @TEST-REQUIRES: bro --help 2>&1 | grep -q mem-leaks # # @TEST-EXEC: HEAP_CHECK_DUMP_DIRECTORY=. HEAPCHECK=local bro -m -r $TRACES/wikipedia.trace %INPUT diff --git a/testing/btest/core/leaks/remote.bro b/testing/btest/core/leaks/remote.bro new file mode 100644 index 0000000000..f888d8f6ee --- /dev/null +++ b/testing/btest/core/leaks/remote.bro @@ -0,0 +1,81 @@ +# Needs perftools support. +# +# @TEST-GROUP: leaks +# +# @TEST-REQUIRES: bro --help 2>&1 | grep -q mem-leaks +# +# @TEST-EXEC: btest-bg-run sender HEAP_CHECK_DUMP_DIRECTORY=. HEAPCHECK=local bro -m --pseudo-realtime %INPUT ../sender.bro +# @TEST-EXEC: sleep 1 +# @TEST-EXEC: btest-bg-run receiver HEAP_CHECK_DUMP_DIRECTORY=. HEAPCHECK=local bro -m --pseudo-realtime %INPUT ../receiver.bro +# @TEST-EXEC: sleep 1 +# @TEST-EXEC: btest-bg-wait -k 10 +# @TEST-EXEC: btest-diff sender/test.log +# @TEST-EXEC: btest-diff sender/test.failure.log +# @TEST-EXEC: btest-diff sender/test.success.log +# @TEST-EXEC: cmp receiver/test.log sender/test.log +# @TEST-EXEC: cmp receiver/test.failure.log sender/test.failure.log +# @TEST-EXEC: cmp receiver/test.success.log sender/test.success.log + +# This is the common part loaded by both sender and receiver. +module Test; + +export { + # Create a new ID for our log stream + redef enum Log::ID += { LOG }; + + # Define a record with all the columns the log file can have. + # (I'm using a subset of fields from ssh-ext for demonstration.) + type Log: record { + t: time; + id: conn_id; # Will be rolled out into individual columns. + status: string &optional; + country: string &default="unknown"; + } &log; +} + +event bro_init() +{ + Log::create_stream(Test::LOG, [$columns=Log]); + Log::add_filter(Test::LOG, [$name="f1", $path="test.success", $pred=function(rec: Log): bool { return rec$status == "success"; }]); +} + +##### + +@TEST-START-FILE sender.bro + +module Test; + +@load frameworks/communication/listen + +function fail(rec: Log): bool + { + return rec$status != "success"; + } + +event remote_connection_handshake_done(p: event_peer) + { + Log::add_filter(Test::LOG, [$name="f2", $path="test.failure", $pred=fail]); + + local cid = [$orig_h=1.2.3.4, $orig_p=1234/tcp, $resp_h=2.3.4.5, $resp_p=80/tcp]; + + local r: Log = [$t=network_time(), $id=cid, $status="success"]; + + # Log something. + Log::write(Test::LOG, r); + Log::write(Test::LOG, [$t=network_time(), $id=cid, $status="failure", $country="US"]); + Log::write(Test::LOG, [$t=network_time(), $id=cid, $status="failure", $country="UK"]); + Log::write(Test::LOG, [$t=network_time(), $id=cid, $status="success", $country="BR"]); + Log::write(Test::LOG, [$t=network_time(), $id=cid, $status="failure", $country="MX"]); + disconnect(p); + } +@TEST-END-FILE + +@TEST-START-FILE receiver.bro + +##### + +redef Communication::nodes += { + ["foo"] = [$host = 127.0.0.1, $connect=T, $request_logs=T] +}; + +@TEST-END-FILE diff --git a/testing/btest/core/leaks/test-all.bro b/testing/btest/core/leaks/test-all.bro index 6e605372c9..f217cc229c 100644 --- a/testing/btest/core/leaks/test-all.bro +++ b/testing/btest/core/leaks/test-all.bro @@ -1,5 +1,7 @@ # Needs perftools support. # +# @TEST-GROUP: leaks +# # @TEST-REQUIRES: bro --help 2>&1 | grep -q mem-leaks # # @TEST-EXEC: HEAP_CHECK_DUMP_DIRECTORY=. HEAPCHECK=local bro -m -r $TRACES/wikipedia.trace test-all-policy diff --git a/testing/btest/core/mobile-ipv6-home-addr.test b/testing/btest/core/mobile-ipv6-home-addr.test new file mode 100644 index 0000000000..536d381f9b --- /dev/null +++ b/testing/btest/core/mobile-ipv6-home-addr.test @@ -0,0 +1,11 @@ +# @TEST-REQUIRES: grep -q "#define ENABLE_MOBILE_IPV6" $BUILD/config.h +# @TEST-EXEC: bro -b -r $TRACES/mobile-ipv6/ipv6-mobile-hoa.trace %INPUT >output +# @TEST-EXEC: btest-diff output + +# Just check that the orig of the connection is the Home Address, but the +# source in the header is the actual source address. +event new_packet(c: connection, p: pkt_hdr) + { + print c$id; + print p; + } diff --git a/testing/btest/core/mobile-ipv6-routing.test b/testing/btest/core/mobile-ipv6-routing.test new file mode 100644 index 0000000000..6ad5be002d --- /dev/null +++ b/testing/btest/core/mobile-ipv6-routing.test @@ -0,0 +1,11 @@ +# @TEST-REQUIRES: grep -q "#define ENABLE_MOBILE_IPV6" $BUILD/config.h +# @TEST-EXEC: bro -b -r $TRACES/mobile-ipv6/ipv6-mobile-routing.trace %INPUT >output +# @TEST-EXEC: btest-diff output + +# Just check that the responder of the connection is the final routing +# address, but the destination in the header is the actual destination address. +event new_packet(c: connection, p: pkt_hdr) + { + print c$id; + print p; + } diff --git a/testing/btest/core/mobility-checksums.test b/testing/btest/core/mobility-checksums.test new file mode 100644 index 0000000000..1d41daf543 --- /dev/null +++ b/testing/btest/core/mobility-checksums.test @@ -0,0 +1,9 @@ +# @TEST-REQUIRES: grep -q "#define ENABLE_MOBILE_IPV6" $BUILD/config.h +# @TEST-EXEC: bro -b -r $TRACES/chksums/mip6-bad-mh-chksum.pcap >>bad.out 2>&1 +# @TEST-EXEC: bro -b -r $TRACES/chksums/ip6-hoa-tcp-bad-chksum.pcap >>bad.out 2>&1 +# @TEST-EXEC: bro -b -r $TRACES/chksums/ip6-hoa-udp-bad-chksum.pcap >>bad.out 2>&1 +# @TEST-EXEC: bro -b -r $TRACES/chksums/mip6-good-mh-chksum.pcap >>good.out 2>&1 +# @TEST-EXEC: bro -b -r $TRACES/chksums/ip6-hoa-tcp-good-chksum.pcap >>bad.out 2>&1 +# @TEST-EXEC: bro -b -r $TRACES/chksums/ip6-hoa-udp-good-chksum.pcap >>bad.out 2>&1 +# @TEST-EXEC: btest-diff bad.out +# @TEST-EXEC: btest-diff good.out diff --git a/testing/btest/core/mobility_msg.test b/testing/btest/core/mobility_msg.test new file mode 100644 index 0000000000..73461e7944 --- /dev/null +++ b/testing/btest/core/mobility_msg.test @@ -0,0 +1,44 @@ +# @TEST-REQUIRES: grep -q "#define ENABLE_MOBILE_IPV6" $BUILD/config.h +# @TEST-EXEC: bro -b -r $TRACES/mobile-ipv6/mip6_back.trace %INPUT >output +# @TEST-EXEC: bro -b -r $TRACES/mobile-ipv6/mip6_be.trace %INPUT >>output +# @TEST-EXEC: bro -b -r $TRACES/mobile-ipv6/mip6_brr.trace %INPUT >>output +# @TEST-EXEC: bro -b -r $TRACES/mobile-ipv6/mip6_bu.trace %INPUT >>output +# @TEST-EXEC: bro -b -r $TRACES/mobile-ipv6/mip6_cot.trace %INPUT >>output +# @TEST-EXEC: bro -b -r $TRACES/mobile-ipv6/mip6_coti.trace %INPUT >>output +# @TEST-EXEC: bro -b -r $TRACES/mobile-ipv6/mip6_hot.trace %INPUT >>output +# @TEST-EXEC: bro -b -r $TRACES/mobile-ipv6/mip6_hoti.trace %INPUT >>output +# @TEST-EXEC: btest-diff output + +event mobile_ipv6_message(p: pkt_hdr) + { + if ( ! p?$ip6 ) return; + + for ( i in p$ip6$exts ) + { + if ( p$ip6$exts[i]$id == IPPROTO_MOBILITY ) + { + if ( ! p$ip6$exts[i]?$mobility ) + print "ERROR: Mobility extension header uninitialized"; + + if ( p$ip6$exts[i]$mobility$mh_type == 0 ) + print "Binding Refresh Request:"; + else if ( p$ip6$exts[i]$mobility$mh_type == 1 ) + print "Home Test Init:"; + else if ( p$ip6$exts[i]$mobility$mh_type == 2 ) + print "Care-of Test Init:"; + else if ( p$ip6$exts[i]$mobility$mh_type == 3 ) + print "Home Test:"; + else if ( p$ip6$exts[i]$mobility$mh_type == 4 ) + print "Care-of Test:"; + else if ( p$ip6$exts[i]$mobility$mh_type == 5 ) + print "Binding Update:"; + else if ( p$ip6$exts[i]$mobility$mh_type == 6 ) + print "Binding ACK:"; + else if ( p$ip6$exts[i]$mobility$mh_type == 7 ) + print "Binding Error:"; + else + print "Unknown Mobility Header:"; + print p$ip6; + } + } + } diff --git a/testing/btest/core/print-bpf-filters-ipv6.bro b/testing/btest/core/print-bpf-filters-ipv6.bro deleted file mode 100644 index 824c979d70..0000000000 --- a/testing/btest/core/print-bpf-filters-ipv6.bro +++ /dev/null @@ -1,12 +0,0 @@ -# @TEST-REQUIRES: bro -e 'print bro_has_ipv6()' | grep -q T -# -# @TEST-EXEC: bro -r $TRACES/empty.trace -e '' >output -# @TEST-EXEC: cat packet_filter.log >>output -# @TEST-EXEC: bro -r $TRACES/empty.trace PacketFilter::all_packets=F >>output -# @TEST-EXEC: cat packet_filter.log >>output -# @TEST-EXEC: bro -r $TRACES/empty.trace -f "port 42" -e '' >>output -# @TEST-EXEC: cat packet_filter.log >>output -# @TEST-EXEC: bro -r $TRACES/empty.trace -C -f "port 56730" -r $TRACES/mixed-vlan-mpls.trace >>output -# @TEST-EXEC: cat packet_filter.log >>output -# @TEST-EXEC: btest-diff output -# @TEST-EXEC: btest-diff conn.log diff --git a/testing/btest/core/print-bpf-filters-ipv4.bro b/testing/btest/core/print-bpf-filters.bro similarity index 89% rename from testing/btest/core/print-bpf-filters-ipv4.bro rename to testing/btest/core/print-bpf-filters.bro index 5a3b0cf7ce..6d9cef0220 100644 --- a/testing/btest/core/print-bpf-filters-ipv4.bro +++ b/testing/btest/core/print-bpf-filters.bro @@ -1,5 +1,3 @@ -# @TEST-REQUIRES: bro -e 'print bro_has_ipv6()' | grep -q F -# # @TEST-EXEC: bro -r $TRACES/empty.trace -e '' >output # @TEST-EXEC: cat packet_filter.log >>output # @TEST-EXEC: bro -r $TRACES/empty.trace PacketFilter::all_packets=F >>output diff --git a/testing/btest/core/truncation.test b/testing/btest/core/truncation.test new file mode 100644 index 0000000000..16a60fe6db --- /dev/null +++ b/testing/btest/core/truncation.test @@ -0,0 +1,6 @@ +# Truncated IP packet's should not be analyzed, and generate truncated_IP weird + +# @TEST-EXEC: bro -b -r $TRACES/trunc/ip4-trunc.pcap >>output 2>&1 +# @TEST-EXEC: bro -b -r $TRACES/trunc/ip6-trunc.pcap >>output 2>&1 +# @TEST-EXEC: bro -b -r $TRACES/trunc/ip6-ext-trunc.pcap >>output 2>&1 +# @TEST-EXEC: btest-diff output diff --git a/testing/btest/istate/broccoli-ipv6.bro b/testing/btest/istate/broccoli-ipv6.bro new file mode 100644 index 0000000000..b7ab5bdb05 --- /dev/null +++ b/testing/btest/istate/broccoli-ipv6.bro @@ -0,0 +1,15 @@ +# @TEST-GROUP: comm +# +# @TEST-REQUIRES: test -e $BUILD/aux/broccoli/src/libbroccoli.so || test -e $BUILD/aux/broccoli/src/libbroccoli.dylib +# +# @TEST-EXEC: btest-bg-run bro bro %INPUT $DIST/aux/broccoli/test/broccoli-v6addrs.bro +# @TEST-EXEC: btest-bg-run broccoli $BUILD/aux/broccoli/test/broccoli-v6addrs +# @TEST-EXEC: btest-bg-wait -k 20 +# @TEST-EXEC: btest-diff bro/.stdout +# @TEST-EXEC: btest-diff broccoli/.stdout + +event remote_connection_closed(p: event_peer) + { + terminate(); + } + diff --git a/testing/btest/istate/broccoli.bro b/testing/btest/istate/broccoli.bro index 7f97f40585..235ff9119c 100644 --- a/testing/btest/istate/broccoli.bro +++ b/testing/btest/istate/broccoli.bro @@ -1,4 +1,5 @@ -# @TEST-REQUIRES: grep -vq '#define BROv6' $BUILD/config.h +# @TEST-GROUP: comm +# # @TEST-REQUIRES: test -e $BUILD/aux/broccoli/src/libbroccoli.so || test -e $BUILD/aux/broccoli/src/libbroccoli.dylib # # @TEST-EXEC: btest-bg-run bro bro %INPUT $DIST/aux/broccoli/test/broping-record.bro diff --git a/testing/btest/istate/events-ssl.bro b/testing/btest/istate/events-ssl.bro index 5a41e1683b..25aa2dc8fb 100644 --- a/testing/btest/istate/events-ssl.bro +++ b/testing/btest/istate/events-ssl.bro @@ -1,4 +1,5 @@ -# +# @TEST-GROUP: comm +# # @TEST-EXEC: btest-bg-run sender bro -C -r $TRACES/web.trace --pseudo-realtime ../sender.bro # @TEST-EXEC: btest-bg-run receiver bro ../receiver.bro # @TEST-EXEC: btest-bg-wait -k 20 diff --git a/testing/btest/istate/events.bro b/testing/btest/istate/events.bro index a0dc494ced..81a1d765db 100644 --- a/testing/btest/istate/events.bro +++ b/testing/btest/istate/events.bro @@ -1,4 +1,5 @@ -# +# @TEST-GROUP: comm +# # @TEST-EXEC: btest-bg-run sender bro -C -r $TRACES/web.trace --pseudo-realtime ../sender.bro # @TEST-EXEC: btest-bg-run receiver bro ../receiver.bro # @TEST-EXEC: btest-bg-wait -k 20 diff --git a/testing/btest/istate/pybroccoli.py b/testing/btest/istate/pybroccoli.py index b7fb53a955..1a5830b41a 100644 --- a/testing/btest/istate/pybroccoli.py +++ b/testing/btest/istate/pybroccoli.py @@ -1,4 +1,5 @@ -# @TEST-REQUIRES: grep -vq '#define BROv6' $BUILD/config.h +# @TEST-GROUP: comm +# # @TEST-REQUIRES: test -e $BUILD/aux/broccoli/src/libbroccoli.so || test -e $BUILD/aux/broccoli/src/libbroccoli.dylib # @TEST-REQUIRES: test -e $BUILD/aux/broccoli/bindings/broccoli-python/_broccoli_intern.so # diff --git a/testing/btest/istate/sync.bro b/testing/btest/istate/sync.bro index 1ccdc5538c..db5ea0bbb4 100644 --- a/testing/btest/istate/sync.bro +++ b/testing/btest/istate/sync.bro @@ -1,3 +1,4 @@ +# @TEST-GROUP: comm # # @TEST-EXEC: btest-bg-run sender bro %INPUT ../sender.bro # @TEST-EXEC: btest-bg-run receiver bro %INPUT ../receiver.bro diff --git a/testing/btest/language/expire_func.test b/testing/btest/language/expire_func.test new file mode 100644 index 0000000000..653a4d9a86 --- /dev/null +++ b/testing/btest/language/expire_func.test @@ -0,0 +1,23 @@ +# @TEST-EXEC: bro -C -r $TRACES/var-services-std-ports.trace %INPUT >output +# @TEST-EXEC: btest-diff output + +function inform_me(s: set[string], idx: string): interval + { + print fmt("expired %s", idx); + return 0secs; + } + +global s: set[string] &create_expire=1secs &expire_func=inform_me; + +event bro_init() + { + add s["i"]; + add s["am"]; + add s["here"]; + } + +event new_connection(c: connection) + { + add s[fmt("%s", c$id)]; + print s; + } diff --git a/testing/btest/language/ipv6-literals.bro b/testing/btest/language/ipv6-literals.bro new file mode 100644 index 0000000000..004d104c6e --- /dev/null +++ b/testing/btest/language/ipv6-literals.bro @@ -0,0 +1,32 @@ +# @TEST-EXEC: bro -b %INPUT >output +# @TEST-EXEC: btest-diff output + +local v: vector of addr = vector(); + +v[|v|] = [::1]; +v[|v|] = [::ffff]; +v[|v|] = [::ffff:ffff]; +v[|v|] = [::0a0a:ffff]; +v[|v|] = [1::1]; +v[|v|] = [1::a]; +v[|v|] = [1::1:1]; +v[|v|] = [1::1:a]; +v[|v|] = [a::a]; +v[|v|] = [a::1]; +v[|v|] = [a::a:a]; +v[|v|] = [a::a:1]; +v[|v|] = [a:a::a]; +v[|v|] = [aaaa:0::ffff]; +v[|v|] = [::ffff:192.168.1.100]; +v[|v|] = [ffff::192.168.1.100]; +v[|v|] = [::192.168.1.100]; +v[|v|] = [::ffff:0:192.168.1.100]; +v[|v|] = [805B:2D9D:DC28::FC57:212.200.31.255]; +v[|v|] = [0xaaaa::bbbb]; +v[|v|] = [aaaa:bbbb:cccc:dddd:eeee:ffff:1111:2222]; +v[|v|] = [aaaa:bbbb:cccc:dddd:eeee:ffff:1:2222]; +v[|v|] = [aaaa:bbbb:cccc:dddd:eeee:ffff:0:2222]; +v[|v|] = [aaaa:bbbb:cccc:dddd:eeee:0:0:2222]; + +for (i in v) + print v[i]; diff --git a/testing/btest/language/match-test.bro b/testing/btest/language/match-test.bro deleted file mode 100644 index 9352d0f39f..0000000000 --- a/testing/btest/language/match-test.bro +++ /dev/null @@ -1,20 +0,0 @@ -# @TEST-EXEC: bro %INPUT >output 2>&1 -# @TEST-EXEC: btest-diff output - -global match_stuff = { - [$pred(a: count) = { return a > 5; }, - $result = "it's big", - $priority = 2], - - [$pred(a: count) = { return a > 15; }, - $result = "it's really big", - $priority = 3], - - [$pred(a: count) = { return T; }, - $result = "default", - $priority = 0], -}; - -print match 0 using match_stuff; -print match 10 using match_stuff; -print match 20 using match_stuff; diff --git a/testing/btest/language/match-test2.bro b/testing/btest/language/match-test2.bro deleted file mode 100644 index f1c120adf2..0000000000 --- a/testing/btest/language/match-test2.bro +++ /dev/null @@ -1,51 +0,0 @@ -# @TEST-EXEC: bro %INPUT >output 2>&1 -# @TEST-EXEC: btest-diff output - -type fakealert : record { - alert: string; -}; - - -type match_rec : record { - result : count; - pred : function(rec : fakealert) : bool; - priority: count; -}; - - -#global test_set : set[int] = -#{ -#1, 2, 3 -#}; - -global match_set : set[match_rec] = -{ - [$result = 1, $pred(a: fakealert) = { return T; }, $priority = 8 ], - [$result = 2, $pred(a: fakealert) = { return T; }, $priority = 9 ] -}; - -global al : fakealert; - -#global testset : set[fakealert] = -#{ -# [$alert="hithere"] -#}; - - -type nonalert: record { - alert : string; - pred : function(a : int) : int; -}; - -#global na : nonalert; -#na$alert = "5"; - -#al$alert = "hithere2"; -#if (al in testset) -# print 1; -#else -# print 0; - - -al$alert = "hi"; -print (match al using match_set); diff --git a/testing/btest/language/sizeof.bro b/testing/btest/language/sizeof.bro index 3457f895c9..99d7b51ce8 100644 --- a/testing/btest/language/sizeof.bro +++ b/testing/btest/language/sizeof.bro @@ -20,6 +20,7 @@ type example_record: record { }; global a: addr = 1.2.3.4; +global a6: addr = [::1]; global b: bool = T; global c: count = 10; global d: double = -1.23; @@ -52,8 +53,10 @@ v[4] = "World"; # Print out the sizes of the various vals: #----------------------------------------- -# Size of addr: returns integer representation for IPv4, 0 for IPv6. -print fmt("Address %s: %d", a, |a|); +# Size of addr: returns number of bits required to represent the address +# which is 32 for IPv4 or 128 for IPv6 +print fmt("IPv4 Address %s: %d", a, |a|); +print fmt("IPv6 Address %s: %d", a6, |a6|); # Size of boolean: returns 1 or 0. print fmt("Boolean %s: %d", b, |b|); diff --git a/testing/btest/scripts/base/frameworks/cluster/start-it-up.bro b/testing/btest/scripts/base/frameworks/cluster/start-it-up.bro index b8ee4c33e8..d7b552d962 100644 --- a/testing/btest/scripts/base/frameworks/cluster/start-it-up.bro +++ b/testing/btest/scripts/base/frameworks/cluster/start-it-up.bro @@ -1,3 +1,5 @@ +# @TEST-GROUP: comm +# # @TEST-EXEC: btest-bg-run manager-1 BROPATH=$BROPATH:.. CLUSTER_NODE=manager-1 bro %INPUT # @TEST-EXEC: btest-bg-run proxy-1 BROPATH=$BROPATH:.. CLUSTER_NODE=proxy-1 bro %INPUT # @TEST-EXEC: btest-bg-run proxy-2 BROPATH=$BROPATH:.. CLUSTER_NODE=proxy-2 bro %INPUT diff --git a/testing/btest/scripts/base/frameworks/communication/communication_log_baseline.bro b/testing/btest/scripts/base/frameworks/communication/communication_log_baseline.bro index dc3b43ad67..85b23dbdc0 100644 --- a/testing/btest/scripts/base/frameworks/communication/communication_log_baseline.bro +++ b/testing/btest/scripts/base/frameworks/communication/communication_log_baseline.bro @@ -1,4 +1,5 @@ -# +# @TEST-GROUP: comm +# # @TEST-EXEC: btest-bg-run receiver bro -b ../receiver.bro # @TEST-EXEC: btest-bg-run sender bro -b ../sender.bro # @TEST-EXEC: btest-bg-wait -k 10 diff --git a/testing/btest/scripts/base/frameworks/control/configuration_update.bro b/testing/btest/scripts/base/frameworks/control/configuration_update.bro index 9b16faee69..d09105ca7a 100644 --- a/testing/btest/scripts/base/frameworks/control/configuration_update.bro +++ b/testing/btest/scripts/base/frameworks/control/configuration_update.bro @@ -1,3 +1,5 @@ +# @TEST-GROUP: comm +# # @TEST-EXEC: btest-bg-run controllee BROPATH=$BROPATH:.. bro %INPUT frameworks/control/controllee Communication::listen_port=65531/tcp # @TEST-EXEC: btest-bg-run controller BROPATH=$BROPATH:.. bro %INPUT test-redef frameworks/control/controller Control::host=127.0.0.1 Control::host_port=65531/tcp Control::cmd=configuration_update # @TEST-EXEC: btest-bg-run controller2 BROPATH=$BROPATH:.. bro %INPUT frameworks/control/controller Control::host=127.0.0.1 Control::host_port=65531/tcp Control::cmd=shutdown diff --git a/testing/btest/scripts/base/frameworks/control/id_value.bro b/testing/btest/scripts/base/frameworks/control/id_value.bro index e06fa46e74..7235521034 100644 --- a/testing/btest/scripts/base/frameworks/control/id_value.bro +++ b/testing/btest/scripts/base/frameworks/control/id_value.bro @@ -1,3 +1,5 @@ +# @TEST-GROUP: comm +# # @TEST-EXEC: btest-bg-run controllee BROPATH=$BROPATH:.. bro %INPUT only-for-controllee frameworks/control/controllee Communication::listen_port=65532/tcp # @TEST-EXEC: btest-bg-run controller BROPATH=$BROPATH:.. bro %INPUT frameworks/control/controller Control::host=127.0.0.1 Control::host_port=65532/tcp Control::cmd=id_value Control::arg=test_var # @TEST-EXEC: btest-bg-wait -k 10 diff --git a/testing/btest/scripts/base/frameworks/control/shutdown.bro b/testing/btest/scripts/base/frameworks/control/shutdown.bro index 9953a8382a..ec1ca6da16 100644 --- a/testing/btest/scripts/base/frameworks/control/shutdown.bro +++ b/testing/btest/scripts/base/frameworks/control/shutdown.bro @@ -1,3 +1,5 @@ +# @TEST-GROUP: comm +# # @TEST-EXEC: btest-bg-run controllee BROPATH=$BROPATH:.. bro %INPUT frameworks/control/controllee Communication::listen_port=65530/tcp # @TEST-EXEC: btest-bg-run controller BROPATH=$BROPATH:.. bro %INPUT frameworks/control/controller Control::host=127.0.0.1 Control::host_port=65530/tcp Control::cmd=shutdown # @TEST-EXEC: btest-bg-wait 10 diff --git a/testing/btest/scripts/base/frameworks/logging/remote-types.bro b/testing/btest/scripts/base/frameworks/logging/remote-types.bro index 4e866cc985..ce93495bc8 100644 --- a/testing/btest/scripts/base/frameworks/logging/remote-types.bro +++ b/testing/btest/scripts/base/frameworks/logging/remote-types.bro @@ -1,3 +1,4 @@ +# @TEST-GROUP: comm # # @TEST-EXEC: btest-bg-run sender bro --pseudo-realtime %INPUT ../sender.bro # @TEST-EXEC: btest-bg-run receiver bro --pseudo-realtime %INPUT ../receiver.bro diff --git a/testing/btest/scripts/base/frameworks/logging/remote.bro b/testing/btest/scripts/base/frameworks/logging/remote.bro index 8ed3405aed..bb1e5b8ce3 100644 --- a/testing/btest/scripts/base/frameworks/logging/remote.bro +++ b/testing/btest/scripts/base/frameworks/logging/remote.bro @@ -1,3 +1,4 @@ +# @TEST-GROUP: comm # # @TEST-EXEC: btest-bg-run sender bro --pseudo-realtime %INPUT ../sender.bro # @TEST-EXEC: sleep 1 diff --git a/testing/btest/scripts/base/frameworks/metrics/basic-cluster.bro b/testing/btest/scripts/base/frameworks/metrics/basic-cluster.bro index 23b87053ab..b801074b33 100644 --- a/testing/btest/scripts/base/frameworks/metrics/basic-cluster.bro +++ b/testing/btest/scripts/base/frameworks/metrics/basic-cluster.bro @@ -1,3 +1,5 @@ +# @TEST-GROUP: comm +# # @TEST-EXEC: btest-bg-run manager-1 BROPATH=$BROPATH:.. CLUSTER_NODE=manager-1 bro %INPUT # @TEST-EXEC: btest-bg-run proxy-1 BROPATH=$BROPATH:.. CLUSTER_NODE=proxy-1 bro %INPUT # @TEST-EXEC: sleep 1 diff --git a/testing/btest/scripts/base/frameworks/metrics/cluster-intermediate-update.bro b/testing/btest/scripts/base/frameworks/metrics/cluster-intermediate-update.bro index 45d44898aa..701d2ea378 100644 --- a/testing/btest/scripts/base/frameworks/metrics/cluster-intermediate-update.bro +++ b/testing/btest/scripts/base/frameworks/metrics/cluster-intermediate-update.bro @@ -1,3 +1,5 @@ +# @TEST-GROUP: comm +# # @TEST-EXEC: btest-bg-run manager-1 BROPATH=$BROPATH:.. CLUSTER_NODE=manager-1 bro %INPUT # @TEST-EXEC: btest-bg-run proxy-1 BROPATH=$BROPATH:.. CLUSTER_NODE=proxy-1 bro %INPUT # @TEST-EXEC: sleep 1 diff --git a/testing/btest/scripts/base/frameworks/notice/cluster.bro b/testing/btest/scripts/base/frameworks/notice/cluster.bro index 125d021d82..97470eaa7f 100644 --- a/testing/btest/scripts/base/frameworks/notice/cluster.bro +++ b/testing/btest/scripts/base/frameworks/notice/cluster.bro @@ -1,3 +1,5 @@ +# @TEST-GROUP: comm +# # @TEST-EXEC: btest-bg-run manager-1 BROPATH=$BROPATH:.. CLUSTER_NODE=manager-1 bro %INPUT # @TEST-EXEC: btest-bg-run proxy-1 BROPATH=$BROPATH:.. CLUSTER_NODE=proxy-1 bro %INPUT # @TEST-EXEC: sleep 1 diff --git a/testing/btest/scripts/base/frameworks/notice/suppression-cluster.bro b/testing/btest/scripts/base/frameworks/notice/suppression-cluster.bro index e084fb74e0..d56d940e8e 100644 --- a/testing/btest/scripts/base/frameworks/notice/suppression-cluster.bro +++ b/testing/btest/scripts/base/frameworks/notice/suppression-cluster.bro @@ -1,3 +1,5 @@ +# @TEST-GROUP: comm +# # @TEST-EXEC: btest-bg-run manager-1 BROPATH=$BROPATH:.. CLUSTER_NODE=manager-1 bro %INPUT # @TEST-EXEC: btest-bg-run proxy-1 BROPATH=$BROPATH:.. CLUSTER_NODE=proxy-1 bro %INPUT # @TEST-EXEC: sleep 1 diff --git a/testing/btest/scripts/base/protocols/ftp/ftp-ipv4.bro b/testing/btest/scripts/base/protocols/ftp/ftp-ipv4.bro new file mode 100644 index 0000000000..5cb8b808d5 --- /dev/null +++ b/testing/btest/scripts/base/protocols/ftp/ftp-ipv4.bro @@ -0,0 +1,6 @@ +# This tests both active and passive FTP over IPv4. +# +# @TEST-EXEC: bro -r $TRACES/ftp-ipv4.trace +# @TEST-EXEC: btest-diff conn.log +# @TEST-EXEC: btest-diff ftp.log + diff --git a/testing/btest/scripts/base/protocols/ftp/ftp-ipv6.bro b/testing/btest/scripts/base/protocols/ftp/ftp-ipv6.bro new file mode 100644 index 0000000000..7ce31808c9 --- /dev/null +++ b/testing/btest/scripts/base/protocols/ftp/ftp-ipv6.bro @@ -0,0 +1,6 @@ +# This tests both active and passive FTP over IPv6. +# +# @TEST-EXEC: bro -r $TRACES/ipv6-ftp.trace +# @TEST-EXEC: btest-diff conn.log +# @TEST-EXEC: btest-diff ftp.log + diff --git a/testing/external/Makefile b/testing/external/Makefile index 994d8962c0..b705734003 100644 --- a/testing/external/Makefile +++ b/testing/external/Makefile @@ -24,3 +24,7 @@ push: status: @for repo in $(REPOS); do ( cd $$repo && echo '>>' $$repo && git status -bs && echo ); done +coverage: + @for repo in $(REPOS); do (cd $$repo && echo "Coverage for '$$repo' repo:" && make coverage); done + +.PHONY: all brief init pull push status coverage diff --git a/testing/external/scripts/perftools-adapt-paths b/testing/external/scripts/perftools-adapt-paths index 2eda2477c7..cfecd39993 100755 --- a/testing/external/scripts/perftools-adapt-paths +++ b/testing/external/scripts/perftools-adapt-paths @@ -7,4 +7,4 @@ cat $1 | sed "s#bro *\"\./#../../../build/src/bro \".tmp/$TEST_NAME/#g" | sed 's/ *--gv//g' >$1.tmp && mv $1.tmp $1 -grep -q "No leaks found" $1 +grep -qv "detected leaks of" $1 diff --git a/testing/external/subdir-btest.cfg b/testing/external/subdir-btest.cfg index 7b1d59cb07..c4e74f99fa 100644 --- a/testing/external/subdir-btest.cfg +++ b/testing/external/subdir-btest.cfg @@ -17,4 +17,4 @@ TRACES=%(testbase)s/Traces SCRIPTS=%(testbase)s/../scripts DIST=%(testbase)s/../../.. BUILD=%(testbase)s/../../../build -BRO_PROFILER_FILE=%(testbase)s/.tmp/script-coverage +BRO_PROFILER_FILE=%(testbase)s/.tmp/script-coverage.XXXXXX diff --git a/testing/scripts/btest-bg-run b/testing/scripts/btest-bg-run deleted file mode 100755 index 64a38b9759..0000000000 --- a/testing/scripts/btest-bg-run +++ /dev/null @@ -1,7 +0,0 @@ -#! /usr/bin/env bash - -# This is a wrapper script to btest's real btest-bg-run. It's used -# when collecting Bro script coverage statistics so that two independent -# Bro processing don't try to write those usage statistics to the same file. - -BRO_PROFILER_FILE=`mktemp $TMPDIR/script-coverage.XXXX` $BTEST_PATH/btest-bg-run $@ diff --git a/testing/scripts/diff-canon-notice-policy b/testing/scripts/diff-canon-notice-policy new file mode 100755 index 0000000000..f05abaa103 --- /dev/null +++ b/testing/scripts/diff-canon-notice-policy @@ -0,0 +1,10 @@ +#! /usr/bin/awk -f +# +# A diff canonifier that removes the priorities in notice_policy.log. + +/^#/ && $2 == "notice_policy" { filter = 1; } + +filter == 1 && /^[^#]/ { sub("^[0-9]*", "X"); } + +{ print; } + diff --git a/testing/scripts/diff-canonifier-external b/testing/scripts/diff-canonifier-external index 1f953183d3..e788a4a1bb 100755 --- a/testing/scripts/diff-canonifier-external +++ b/testing/scripts/diff-canonifier-external @@ -6,3 +6,6 @@ | `dirname $0`/diff-remove-uids \ | `dirname $0`/diff-remove-mime-types \ | `dirname $0`/diff-remove-x509-names \ + | `dirname $0`/diff-canon-notice-policy \ + | `dirname $0`/diff-sort + diff --git a/testing/scripts/diff-sort b/testing/scripts/diff-sort new file mode 100755 index 0000000000..08b36c79bf --- /dev/null +++ b/testing/scripts/diff-sort @@ -0,0 +1,19 @@ +#! /usr/bin/env bash +# +# A diff canonifier that sorts all lines but keeps all comments +# at the top. It also adds a note at the beginning as a reminder +# that the output has been sorted. + +if [ "$TMP" == "" ]; then + TMP=/tmp +fi + +tmp=$TMP/`basename $0`.$$.tmp + +cat >$tmp + +echo "### NOTE: This file has been sorted with `basename $0`." +cat $tmp | grep ^# +cat $tmp | grep -v ^# | sort -s + +rm -f $tmp