diff --git a/CHANGES b/CHANGES index e40230bf25..9286a5409e 100644 --- a/CHANGES +++ b/CHANGES @@ -1,4 +1,28 @@ +2.0-57 | 2012-02-10 00:02:35 -0800 + + * Fix typos in the documentation. (Daniel Thayer) + + * Fix compiler warning about Brofiler ctor init list order. (Jon Siwek) + + * Fix missing optional field access in webapp signature_match handler. (Jon Siwek) + +2.0-41 | 2012-02-03 04:10:53 -0500 + + * Updates to the Software framework to simplify the API. (Bernhard + Amann) + +2.0-40 | 2012-02-03 01:55:27 -0800 + + * Fix typos in documentation. (Daniel Thayer) + + * Fix sorting of lines in Brofiler coverage.log. (Daniel Thayer) + +2.0-38 | 2012-01-31 11:50:53 -0800 + + * Canonify sorting of lines in Brofiler coverage.log. (Daniel + Thayer) + 2.0-36 | 2012-01-27 10:38:14 -0800 * New "Brofiler" mode that tracks and records script statements diff --git a/VERSION b/VERSION index f5351d6ef2..8dd930b077 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -2.0-36 +2.0-57 diff --git a/aux/bro-aux b/aux/bro-aux index df700afd77..139cc2e1e0 160000 --- a/aux/bro-aux +++ b/aux/bro-aux @@ -1 +1 @@ -Subproject commit df700afd778d7712c057637b365ba7d6219144df +Subproject commit 139cc2e1e049c4e1cc7e95f20866102be1d3d599 diff --git a/doc/cluster.rst b/doc/cluster.rst index 9bffc35fec..ba26471edc 100644 --- a/doc/cluster.rst +++ b/doc/cluster.rst @@ -45,7 +45,7 @@ This is the Bro process that sniffs network traffic and does protocol analysis o The rule of thumb we have followed recently is to allocate approximately 1 core for every 80Mbps of traffic that is being analyzed, however this estimate could be extremely traffic mix specific. It has generally worked for mixed traffic with many users and servers. For example, if your traffic peaks around 2Gbps (combined) and you want to handle traffic at peak load, you may want to have 26 cores available (2048 / 80 == 25.6). If the 80Mbps estimate works for your traffic, this could be handled by 3 physical hosts dedicated to being workers with each one containing dual 6-core processors. -Once a flow based load balancer is put into place this model is extremely easy to scale as well so it’s recommended that you guess at the amount of hardware you will need to fully analyze your traffic. If it turns out that you need more, it’s relatively easy to easy increase the size of the cluster in most cases. +Once a flow based load balancer is put into place this model is extremely easy to scale as well so it’s recommended that you guess at the amount of hardware you will need to fully analyze your traffic. If it turns out that you need more, it’s relatively easy to increase the size of the cluster in most cases. Frontend Options ---------------- @@ -58,7 +58,7 @@ Discrete hardware flow balancers cPacket ^^^^^^^ -If you are monitoring one or more 10G physical interfaces, the recommended solution is to use either a cFlow or cVu device from cPacket because they are currently being used very successfully at a number of sites. These devices will perform layer-2 load balancing by rewriting the destination ethernet MAC address to cause each packet associated with a particular flow to have the same destination MAC. The packets can then be passed directly to a monitoring host where each worker has a BPF filter to limit it's visibility to only that stream of flows or onward to a commodity switch to split the traffic out to multiple 1G interfaces for the workers. This can ultimately greatly reduce costs since workers can use relatively inexpensive 1G interfaces. +If you are monitoring one or more 10G physical interfaces, the recommended solution is to use either a cFlow or cVu device from cPacket because they are currently being used very successfully at a number of sites. These devices will perform layer-2 load balancing by rewriting the destination ethernet MAC address to cause each packet associated with a particular flow to have the same destination MAC. The packets can then be passed directly to a monitoring host where each worker has a BPF filter to limit its visibility to only that stream of flows or onward to a commodity switch to split the traffic out to multiple 1G interfaces for the workers. This can ultimately greatly reduce costs since workers can use relatively inexpensive 1G interfaces. OpenFlow Switches ^^^^^^^^^^^^^^^^^ @@ -76,7 +76,7 @@ The PF_RING software for Linux has a “clustering” feature which will do flow Netmap ^^^^^^ -FreeBSD has an in-progress project named Netmap which will enabled flow based load balancing as well. When it becomes viable for real world use, this document will be updated. +FreeBSD has an in-progress project named Netmap which will enable flow based load balancing as well. When it becomes viable for real world use, this document will be updated. Click! Software Router ^^^^^^^^^^^^^^^^^^^^^^ diff --git a/doc/faq.rst b/doc/faq.rst index b72b933136..8545cc57ee 100644 --- a/doc/faq.rst +++ b/doc/faq.rst @@ -156,8 +156,8 @@ alteration tools. Bro has two options to workaround such situations and ignore bad checksums: 1) The ``-C`` command line option to ``bro``. -2) An option called ``ignore_checksums`` that can be redefined at the policy - policy script layer (e.g. in your ``$PREFIX/share/bro/site/local/bro``): +2) An option called ``ignore_checksums`` that can be redefined at the + policy script layer (e.g. in your ``$PREFIX/share/bro/site/local.bro``): .. code:: bro diff --git a/doc/logging.rst b/doc/logging.rst index 2b5a7167b3..30a793df7d 100644 --- a/doc/logging.rst +++ b/doc/logging.rst @@ -63,7 +63,7 @@ to work with. Filtering --------- -To create new a new output file for an existing stream, you can add a +To create a new output file for an existing stream, you can add a new filter. A filter can, e.g., restrict the set of fields being logged: @@ -195,7 +195,7 @@ predicate that will be called for each log record: Log::add_filter(Conn::LOG, filter); } -This will results in a log file ``conn-http.log`` that contains only +This will result in a log file ``conn-http.log`` that contains only traffic detected and analyzed as HTTP traffic. Extending @@ -254,7 +254,7 @@ being logged. For these cases, a stream can specify an event that will be generated every time a log record is written to it. All of Bro's default log streams define such an event. For example, the connection log stream raises the event :bro:id:`Conn::log_conn`. You -could use that for example for flagging when a connection to +could use that for example for flagging when a connection to a specific destination exceeds a certain duration: .. code:: bro @@ -269,7 +269,7 @@ specific destination exceeds a certain duration: { if ( rec$duration > 5mins ) NOTICE([$note=Long_Conn_Found, - $msg=fmt("unsually long conn to %s", rec$id$resp_h), + $msg=fmt("unusually long conn to %s", rec$id$resp_h), $id=rec$id]); } @@ -319,7 +319,7 @@ example for the ``Foo`` module: module Foo; export { - # Create an ID for the our new stream. By convention, this is + # Create an ID for our new stream. By convention, this is # called "LOG". redef enum Log::ID += { LOG }; diff --git a/doc/notice.rst b/doc/notice.rst index bab44ab9e4..5849e605a9 100644 --- a/doc/notice.rst +++ b/doc/notice.rst @@ -21,7 +21,7 @@ Let's start with a little bit of background on Bro's philosophy on reporting things. Bro ships with a large number of policy scripts which perform a wide variety of analyses. Most of these scripts monitor for activity which might be of interest for the user. However, none of these scripts determines the -importance of what it finds itself. Instead, the scripts only flags situations +importance of what it finds itself. Instead, the scripts only flag situations as *potentially* interesting, leaving it to the local configuration to define which of them are in fact actionable. This decoupling of detection and reporting allows Bro to address the different needs that sites have: @@ -221,7 +221,7 @@ framework. * - :bro:see:`Notice::not_suppressed_types` - Adding a :bro:see:`Notice::Type` to this set results in that notice - no longer undergoes the normal notice suppression that would + no longer undergoing the normal notice suppression that would take place. Be careful when using this in production it could result in a dramatic increase in the number of notices being processed. @@ -270,21 +270,21 @@ fields used when raising notices are described in the following table: information about this particular instance of the notice type. * - ``$sub`` - - This is a sub-message which meant for human readability but will + - This is a sub-message meant for human readability but will frequently also be used to contain data meant to be matched with the ``Notice::policy``. * - ``$conn`` - If a connection record is available when the notice is being raised - and the notice represents some attribute of the connection the + and the notice represents some attribute of the connection, then the connection record can be given here. Other fields such as ``$id`` and ``$src`` will automatically be populated from this value. * - ``$id`` - If a conn_id record is available when the notice is being raised and - the notice represents some attribute of the connection, the connection - be given here. Other fields such as ``$src`` will automatically be - populated from this value. + the notice represents some attribute of the connection, then the + connection can be given here. Other fields such as ``$src`` will + automatically be populated from this value. * - ``$src`` - If the notice represents an attribute of a single host then it's @@ -313,7 +313,7 @@ of the notice the best information about the notice. If the notice is representative of many connections and is an attribute of a host (e.g. a scanning host) it probably makes most sense to fill out the ``$src`` field and not give a connection or conn_id. If a notice is representative of a -connection attribute (e.g. an apparent SSH login) the it makes sense to fill +connection attribute (e.g. an apparent SSH login) then it makes sense to fill out either ``$conn`` or ``$id`` based on the data that is available when the notice is raised. Using care when inserting data into a notice will make later analysis easier when only the data to fully represent the occurrence that diff --git a/doc/quickstart.rst b/doc/quickstart.rst index df6cb1d063..5201420856 100644 --- a/doc/quickstart.rst +++ b/doc/quickstart.rst @@ -100,7 +100,7 @@ The following dependencies are required to build Bro: Optional Dependencies ~~~~~~~~~~~~~~~~~~~~~ -Bro can use libGeoIP for geo-locating IP addresses and sendmail for +Bro can use libGeoIP for geo-locating IP addresses, and sendmail for sending emails. * RPM/RedHat-based Linux: @@ -127,7 +127,7 @@ sending emails. Vanilla OS X installations don't ship with libmagic or libGeoIP, but if installed from your preferred package management system (e.g. MacPorts, - Fink Homebrew), they should be automatically detected and Bro will compile + Fink, or Homebrew), they should be automatically detected and Bro will compile against them. Additional steps may be needed to :doc:`get the right GeoIP database ` @@ -151,8 +151,8 @@ for downloading the full source code experience for Bro via git is: .. note:: If you choose to clone the ``bro`` repository non-recursively for a "minimal Bro experience", be aware that compiling it depends on - BinPAC, which has it's own ``binpac`` repository. Either install it - first or initizalize/update the cloned ``bro`` repository's + BinPAC, which has its own ``binpac`` repository. Either install it + first or initialize/update the cloned ``bro`` repository's ``aux/binpac`` submodule. See the ``INSTALL`` file included with the source code for more information @@ -196,7 +196,7 @@ BroControl is an interactive shell for easily operating/managing Bro installations on a single system or even across multiple systems in a traffic-monitoring cluster. -.. note:: Below, ``$PREFIX``, is used to reference the Bro installation +.. note:: Below, ``$PREFIX`` is used to reference the Bro installation root directory. A Minimal Starting Configuration @@ -449,7 +449,7 @@ that only takes the email action for SSH logins to a defined set of servers: ] }; -You'll just have to trust the syntax for now, but what we've done is first +You'll just have to trust the syntax for now, but what we've done is first declare our own variable to hold a set of watched addresses, ``watched_servers``; then added a record to the policy that will generate an email on the condition that the predicate function evaluates to true, which diff --git a/doc/reporting-problems.rst b/doc/reporting-problems.rst index fa3f32a620..5e55b2ac90 100644 --- a/doc/reporting-problems.rst +++ b/doc/reporting-problems.rst @@ -4,7 +4,7 @@ Reporting Problems .. rst-class:: opening - Here we summarizes some steps to follow when you see Bro doing + Here we summarize some steps to follow when you see Bro doing something it shouldn't. To provide help, it is often crucial for us to have a way of reliably reproducing the effect you're seeing. Unfortunately, reproducing problems can be rather tricky with Bro @@ -20,13 +20,13 @@ Reporting Problems Generally, when you encounter a problem with Bro, the best thing to do is opening a new ticket in `Bro's issue tracker `__ and include information on how to -reproduce the issue. Ideallt, your ticket should come with the +reproduce the issue. Ideally, your ticket should come with the following: * The Bro version you're using (if working directly from the git repository, the branch and revision number.) -* The output you're seeing along with a description what you'd expect +* The output you're seeing along with a description of what you'd expect Bro to do instead. * A *small* trace in `libpcap format `__ @@ -39,7 +39,7 @@ following: * Any non-standard scripts you're using (but please only those really necessary; just a small code snippet triggering the problem would - perfect). + be perfect). * If you encounter a crash, information from the core dump, such as the stack backtrace, can be very helpful. See below for more on @@ -51,10 +51,10 @@ How Do I Get a Trace File? As Bro is usually running live, coming up with a small trace file that reproduces a problem can turn out to be quite a challenge. Often it -works to best to start with a large trace that triggers the problem, -and then successively thin it out as much a possible. +works best to start with a large trace that triggers the problem, +and then successively thin it out as much as possible. -To get to the initial large trace, here are few things you can try: +To get to the initial large trace, here are a few things you can try: * Capture a trace with `tcpdump `__, either on the same interface Bro is running on, or on another host where @@ -66,14 +66,14 @@ To get to the initial large trace, here are few things you can try: (e.g., for HTTP only, try ``port 80``). * Bro's command-line option ``-w `` records all packets it - processes into the given the file. You can then later run Bro + processes into the given file. You can then later run Bro offline on this trace and it will process the packets in the same way as it did live. This is particularly helpful with problems that only occur after Bro has already been running for some time. For example, sometimes a crash may be triggered by a particular kind of traffic only occurring rarely. Running Bro live with ``-w`` and then, after the crash, offline on the recorded trace might, with a - little bit of luck, reproduce the the problem reliably. However, be + little bit of luck, reproduce the problem reliably. However, be careful with ``-w``: it can result in huge trace files, quickly filling up your disk. (One way to mitigate the space issues is to periodically delete the trace file by configuring @@ -96,7 +96,7 @@ much as possible. Here are a few things you can try to this end: * Very often, a single connection is able to demonstrate the problem. If you can identify which one it is (e.g., from one of Bro's ``*.log`` files) you can extract the connection's packets from the - trace usong tcpdump by filtering for the corresponding 4-tuple of + trace using tcpdump by filtering for the corresponding 4-tuple of addresses and ports: .. console:: @@ -131,8 +131,8 @@ First, you should configure Bro with the option ``--enable-debug`` and recompile; this will disable all compiler optimizations and thus make the core dump more useful (don't expect great performance with this version though; compiling Bro without optimization has a noticeable -impact on its CPU usage.). Then enable core dumps if you don't have -already (e.g., ``ulimit -c unlimited`` if you're using a bash). +impact on its CPU usage.). Then enable core dumps if you haven't +already (e.g., ``ulimit -c unlimited`` if you're using bash). Once Bro has crashed, start gdb with the Bro binary and the file containing the core dump. (Alternatively, you can also run Bro @@ -188,7 +188,7 @@ belonging to the ``Connection`` class. That class has members Note that these values are stored in `network byte order `__ -so you will need flip the bytes around if you are on a low-endian +so you will need to flip the bytes around if you are on a low-endian machine (which is why the above example prints them in hex). For example, if an IP address prints as ``0100007f`` , that's 127.0.0.1 . diff --git a/doc/scripts/builtins.rst b/doc/scripts/builtins.rst index ef6738a1a6..3a299bbf69 100644 --- a/doc/scripts/builtins.rst +++ b/doc/scripts/builtins.rst @@ -22,7 +22,7 @@ The Bro scripting language supports the following built-in types. is a string of digits preceded by a ``+`` or ``-`` sign, e.g. ``-42`` or ``+5``. When using type inferencing use care so that the intended type is inferred, e.g. ``local size_difference = 0`` will - infer the :bro:type:`count` while ``local size_difference = +0`` + infer :bro:type:`count`, while ``local size_difference = +0`` will infer :bro:type:`int`. .. bro:type:: count @@ -32,7 +32,7 @@ The Bro scripting language supports the following built-in types. .. bro:type:: counter - An alias to :bro:type:`count` + An alias to :bro:type:`count`. .. TODO: is there anything special about this type? @@ -70,7 +70,7 @@ The Bro scripting language supports the following built-in types. A type used to hold character-string values which represent text. String constants are created by enclosing text in double quotes (") - and the backslash character (\) introduces escape sequences. + and the backslash character (\\) introduces escape sequences. Note that Bro represents strings internally as a count and vector of bytes rather than a NUL-terminated byte string (although string @@ -135,7 +135,7 @@ The Bro scripting language supports the following built-in types. type color: enum { Red, White, Blue, }; - The last comma is after ``Blue`` is optional. + The last comma after ``Blue`` is optional. .. bro:type:: timer @@ -150,8 +150,8 @@ The Bro scripting language supports the following built-in types. followed by one of ``/tcp``, ``/udp``, ``/icmp``, or ``/unknown``. Ports can be compared for equality and also for ordering. When - comparing order across transport-level protocols, ``/unknown`` < - ``/tcp`` < ``/udp`` < ``icmp``, for example ``65535/tcp`` is smaller + comparing order across transport-level protocols, ``unknown`` < + ``tcp`` < ``udp`` < ``icmp``, for example ``65535/tcp`` is smaller than ``0/udp``. .. bro:type:: addr @@ -230,7 +230,7 @@ The Bro scripting language supports the following built-in types. global a: table[count] of table[addr, port] of string; - which declared a table indexed by :bro:type:`count` and yielding + which declares a table indexed by :bro:type:`count` and yielding another :bro:type:`table` which is indexed by an :bro:type:`addr` and :bro:type:`port` to yield a :bro:type:`string`. @@ -392,7 +392,7 @@ The Bro scripting language supports the following built-in types. :bro:attr:`&optional` or have a :bro:attr:`&default` attribute must be specified. - To test for existence of field that is :bro:attr:`&optional`, use the + To test for existence of a field that is :bro:attr:`&optional`, use the ``?$`` operator: .. code:: bro @@ -412,7 +412,7 @@ The Bro scripting language supports the following built-in types. print f, "hello, world"; close(f); - Writing to files like this for logging usually isn't recommend, for better + Writing to files like this for logging usually isn't recommended, for better logging support see :doc:`/logging`. .. bro:type:: func @@ -512,22 +512,22 @@ scripting language supports the following built-in attributes. .. bro:attr:: &optional - Allows record field to be missing. For example the type ``record { + Allows a record field to be missing. For example the type ``record { a: int, b: port &optional }`` could be instantiated both as singleton ``[$a=127.0.0.1]`` or pair ``[$a=127.0.0.1, $b=80/tcp]``. .. bro:attr:: &default Uses a default value for a record field or container elements. For - example, ``table[int] of string &default="foo" }`` would create - table that returns The :bro:type:`string` ``"foo"`` for any + example, ``table[int] of string &default="foo" }`` would create a + table that returns the :bro:type:`string` ``"foo"`` for any non-existing index. .. bro:attr:: &redef Allows for redefinition of initial object values. This is typically used with constants, for example, ``const clever = T &redef;`` would - allow the constant to be redifined at some later point during script + allow the constant to be redefined at some later point during script execution. .. bro:attr:: &rotate_interval @@ -536,7 +536,7 @@ scripting language supports the following built-in attributes. .. bro:attr:: &rotate_size - Rotates af file after it has reached a given size in bytes. + Rotates a file after it has reached a given size in bytes. .. bro:attr:: &add_func diff --git a/doc/signatures.rst b/doc/signatures.rst index c44e1b571a..7a1b164dbb 100644 --- a/doc/signatures.rst +++ b/doc/signatures.rst @@ -57,7 +57,7 @@ contain signatures: By using the ``-s`` flag when you invoke Bro, or by extending the Bro variable :bro:id:`signature_files` using the ``+=`` operator. If a signature file is given without a path, it is searched along the normal ``BROPATH``. The default extension of the file name -is ``.sig``, and Bro appends that automatically when neccesary. +is ``.sig``, and Bro appends that automatically when necessary. Signature language ================== @@ -94,7 +94,7 @@ against. The following keywords are defined: given as IP addresses or CIDR masks. ``src-port``/``dst-port`` ```` - Source and destination port, repectively. + Source and destination port, respectively. ``ip-proto tcp|udp|icmp`` IP protocol. @@ -126,8 +126,8 @@ CIDR notation for netmasks and is translated into a corresponding bitmask applied to the packet's value prior to the comparison (similar to the optional ``& integer``). -Putting all together, this is an example condition that is -equivalent to ``dst- ip == 1.2.3.4/16, 5.6.7.8/24``: +Putting it all together, this is an example condition that is +equivalent to ``dst-ip == 1.2.3.4/16, 5.6.7.8/24``: .. code:: bro-sig @@ -143,7 +143,7 @@ Content conditions are defined by regular expressions. We differentiate two kinds of content conditions: first, the expression may be declared with the ``payload`` statement, in which case it is matched against the raw payload of a connection (for reassembled TCP -streams) or of a each packet (for ICMP, UDP, and non-reassembled TCP). +streams) or of each packet (for ICMP, UDP, and non-reassembled TCP). Second, it may be prefixed with an analyzer-specific label, in which case the expression is matched against the data as extracted by the corresponding analyzer. @@ -208,7 +208,7 @@ To define dependencies between signatures, there are two conditions: ``requires-reverse-signature [!] `` Similar to ``requires-signature``, but ``id`` has to match for the - opposite direction of the same connection, compared the current + opposite direction of the same connection, compared to the current signature. This allows to model the notion of requests and replies. diff --git a/doc/upgrade.rst b/doc/upgrade.rst index 885f8f8b8c..9c1537754a 100644 --- a/doc/upgrade.rst +++ b/doc/upgrade.rst @@ -55,13 +55,13 @@ renamed to ``scripts/`` and contains major subdirectories ``base/``, further. The contents of the new ``scripts/`` directory, like the old/flat -``policy/`` still gets installed under under the ``share/bro`` +``policy/`` still gets installed under the ``share/bro`` subdirectory of the installation prefix path just like previous versions. For example, if Bro was compiled like ``./configure --prefix=/usr/local/bro && make && make install``, then the script hierarchy can be found in ``/usr/local/bro/share/bro``. -THe main +The main subdirectories of that hierarchy are as follows: - ``base/`` contains all scripts that are loaded by Bro by default @@ -132,7 +132,7 @@ Logging Framework - The new logging framework makes it possible to extend, customize, and filter logs very easily. See the :doc:`logging framework ` - more information on usage. + for more information on usage. - A common pattern found in the new scripts is to store logging stream records for protocols inside the ``connection`` records so that @@ -209,8 +209,8 @@ live analysis. BroControl now has an extensive plugin interface for adding new commands and options. Note that this is still considered experimental. -We have remove the ``analysis`` command, and BroControl does currently -not not send daily alarm summaries anymore (this may be restored +We have removed the ``analysis`` command, and BroControl currently +does not send daily alarm summaries anymore (this may be restored later). Removed Functionality @@ -233,11 +233,11 @@ Development Infrastructure ========================== Bro development has moved from using SVN to Git for revision control. -Users that like to use the latest Bro developments by checking it out +Users that want to use the latest Bro development snapshot by checking it out from the source repositories should see the `development process `_. Note that all the various -sub-components now reside on their own repositories. However, the -top-level Bro repository includes them as git submodules so it's easu +sub-components now reside in their own repositories. However, the +top-level Bro repository includes them as git submodules so it's easy to check them all out simultaneously. Bro now uses `CMake `_ for its build system so diff --git a/scripts/base/frameworks/software/main.bro b/scripts/base/frameworks/software/main.bro index f5bbf00a2f..7471076335 100644 --- a/scripts/base/frameworks/software/main.bro +++ b/scripts/base/frameworks/software/main.bro @@ -36,15 +36,17 @@ export { ## The record type that is used for representing and logging software. type Info: record { ## The time at which the software was detected. - ts: time &log; + ts: time &log &optional; ## The IP address detected running the software. host: addr &log; + ## The Port on which the software is running. Only sensible for server software. + host_p: port &log &optional; ## The type of software detected (e.g. :bro:enum:`HTTP::SERVER`). software_type: Type &log &default=UNKNOWN; ## Name of the software (e.g. Apache). - name: string &log; + name: string &log &optional; ## Version of the software. - version: Version &log; + version: Version &log &optional; ## The full unparsed version string found because the version parsing ## doesn't always work reliably in all cases and this acts as a ## fallback in the logs. @@ -65,31 +67,12 @@ export { const asset_tracking = LOCAL_HOSTS &redef; ## Other scripts should call this function when they detect software. - ## unparsed_version: This is the full string from which the - ## :bro:type:`Software::Info` was extracted. - ## ## id: The connection id where the software was discovered. ## ## info: A record representing the software discovered. ## ## Returns: T if the software was logged, F otherwise. - global found: function(id: conn_id, info: Software::Info): bool; - - ## Take many common software version strings and parse them - ## into a sensible :bro:type:`Software::Version` record. There are - ## still many cases where scripts may have to have their own specific - ## version parsing though. - ## - ## unparsed_version: The raw version string. - ## - ## host: The host where the software was discovered. - ## - ## software_type: The type of software. - ## - ## Returns: A complete record ready for the :bro:id:`Software::found` function. - global parse: function(unparsed_version: string, - host: addr, - software_type: Type): Info; + global found: function(id: conn_id, info: Info): bool; ## Compare two version records. ## @@ -117,112 +100,23 @@ export { global log_software: event(rec: Info); } -event bro_init() +event bro_init() &priority=5 { Log::create_stream(Software::LOG, [$columns=Info, $ev=log_software]); } - -function parse_mozilla(unparsed_version: string, - host: addr, - software_type: Type): Info - { - local software_name = ""; - local v: Version; - local parts: table[count] of string; - if ( /Opera [0-9\.]*$/ in unparsed_version ) - { - software_name = "Opera"; - parts = split_all(unparsed_version, /Opera [0-9\.]*$/); - if ( 2 in parts ) - v = parse(parts[2], host, software_type)$version; - } - else if ( / MSIE / in unparsed_version ) - { - software_name = "MSIE"; - if ( /Trident\/4\.0/ in unparsed_version ) - v = [$major=8,$minor=0]; - else if ( /Trident\/5\.0/ in unparsed_version ) - v = [$major=9,$minor=0]; - else if ( /Trident\/6\.0/ in unparsed_version ) - v = [$major=10,$minor=0]; - else - { - parts = split_all(unparsed_version, /MSIE [0-9]{1,2}\.*[0-9]*b?[0-9]*/); - if ( 2 in parts ) - v = parse(parts[2], host, software_type)$version; - } - } - else if ( /Version\/.*Safari\// in unparsed_version ) - { - software_name = "Safari"; - parts = split_all(unparsed_version, /Version\/[0-9\.]*/); - if ( 2 in parts ) - { - v = parse(parts[2], host, software_type)$version; - if ( / Mobile\/?.* Safari/ in unparsed_version ) - v$addl = "Mobile"; - } - } - else if ( /(Firefox|Netscape|Thunderbird)\/[0-9\.]*/ in unparsed_version ) - { - parts = split_all(unparsed_version, /(Firefox|Netscape|Thunderbird)\/[0-9\.]*/); - if ( 2 in parts ) - { - local tmp_s = parse(parts[2], host, software_type); - software_name = tmp_s$name; - v = tmp_s$version; - } - } - else if ( /Chrome\/.*Safari\// in unparsed_version ) - { - software_name = "Chrome"; - parts = split_all(unparsed_version, /Chrome\/[0-9\.]*/); - if ( 2 in parts ) - v = parse(parts[2], host, software_type)$version; - } - else if ( /^Opera\// in unparsed_version ) - { - if ( /Opera M(ini|obi)\// in unparsed_version ) - { - parts = split_all(unparsed_version, /Opera M(ini|obi)/); - if ( 2 in parts ) - software_name = parts[2]; - parts = split_all(unparsed_version, /Version\/[0-9\.]*/); - if ( 2 in parts ) - v = parse(parts[2], host, software_type)$version; - else - { - parts = split_all(unparsed_version, /Opera Mini\/[0-9\.]*/); - if ( 2 in parts ) - v = parse(parts[2], host, software_type)$version; - } - } - else - { - software_name = "Opera"; - parts = split_all(unparsed_version, /Version\/[0-9\.]*/); - if ( 2 in parts ) - v = parse(parts[2], host, software_type)$version; - } - } - else if ( /AppleWebKit\/[0-9\.]*/ in unparsed_version ) - { - software_name = "Unspecified WebKit"; - parts = split_all(unparsed_version, /AppleWebKit\/[0-9\.]*/); - if ( 2 in parts ) - v = parse(parts[2], host, software_type)$version; - } +type Description: record { + name: string; + version: Version; + unparsed_version: string; +}; - return [$ts=network_time(), $host=host, $name=software_name, $version=v, - $software_type=software_type, $unparsed_version=unparsed_version]; - } +# Defining this here because of a circular dependency between two functions. +global parse_mozilla: function(unparsed_version: string): Description; # Don't even try to understand this now, just make sure the tests are # working. -function parse(unparsed_version: string, - host: addr, - software_type: Type): Info +function parse(unparsed_version: string): Description { local software_name = ""; local v: Version; @@ -230,7 +124,7 @@ function parse(unparsed_version: string, # Parse browser-alike versions separately if ( /^(Mozilla|Opera)\/[0-9]\./ in unparsed_version ) { - return parse_mozilla(unparsed_version, host, software_type); + return parse_mozilla(unparsed_version); } else { @@ -255,7 +149,7 @@ function parse(unparsed_version: string, if ( 4 in version_numbers && version_numbers[4] != "" ) v$addl = strip(version_numbers[4]); else if ( 3 in version_parts && version_parts[3] != "" && - version_parts[3] != ")" ) + version_parts[3] != ")" ) { if ( /^[[:blank:]]*\([a-zA-Z0-9\-\._[:blank:]]*\)/ in version_parts[3] ) { @@ -292,9 +186,102 @@ function parse(unparsed_version: string, v$major = extract_count(version_numbers[1]); } } - return [$ts=network_time(), $host=host, $name=software_name, - $version=v, $unparsed_version=unparsed_version, - $software_type=software_type]; + + return [$version=v, $unparsed_version=unparsed_version, $name=software_name]; + } + + +function parse_mozilla(unparsed_version: string): Description + { + local software_name = ""; + local v: Version; + local parts: table[count] of string; + + if ( /Opera [0-9\.]*$/ in unparsed_version ) + { + software_name = "Opera"; + parts = split_all(unparsed_version, /Opera [0-9\.]*$/); + if ( 2 in parts ) + v = parse(parts[2])$version; + } + else if ( / MSIE / in unparsed_version ) + { + software_name = "MSIE"; + if ( /Trident\/4\.0/ in unparsed_version ) + v = [$major=8,$minor=0]; + else if ( /Trident\/5\.0/ in unparsed_version ) + v = [$major=9,$minor=0]; + else if ( /Trident\/6\.0/ in unparsed_version ) + v = [$major=10,$minor=0]; + else + { + parts = split_all(unparsed_version, /MSIE [0-9]{1,2}\.*[0-9]*b?[0-9]*/); + if ( 2 in parts ) + v = parse(parts[2])$version; + } + } + else if ( /Version\/.*Safari\// in unparsed_version ) + { + software_name = "Safari"; + parts = split_all(unparsed_version, /Version\/[0-9\.]*/); + if ( 2 in parts ) + { + v = parse(parts[2])$version; + if ( / Mobile\/?.* Safari/ in unparsed_version ) + v$addl = "Mobile"; + } + } + else if ( /(Firefox|Netscape|Thunderbird)\/[0-9\.]*/ in unparsed_version ) + { + parts = split_all(unparsed_version, /(Firefox|Netscape|Thunderbird)\/[0-9\.]*/); + if ( 2 in parts ) + { + local tmp_s = parse(parts[2]); + software_name = tmp_s$name; + v = tmp_s$version; + } + } + else if ( /Chrome\/.*Safari\// in unparsed_version ) + { + software_name = "Chrome"; + parts = split_all(unparsed_version, /Chrome\/[0-9\.]*/); + if ( 2 in parts ) + v = parse(parts[2])$version; + } + else if ( /^Opera\// in unparsed_version ) + { + if ( /Opera M(ini|obi)\// in unparsed_version ) + { + parts = split_all(unparsed_version, /Opera M(ini|obi)/); + if ( 2 in parts ) + software_name = parts[2]; + parts = split_all(unparsed_version, /Version\/[0-9\.]*/); + if ( 2 in parts ) + v = parse(parts[2])$version; + else + { + parts = split_all(unparsed_version, /Opera Mini\/[0-9\.]*/); + if ( 2 in parts ) + v = parse(parts[2])$version; + } + } + else + { + software_name = "Opera"; + parts = split_all(unparsed_version, /Version\/[0-9\.]*/); + if ( 2 in parts ) + v = parse(parts[2])$version; + } + } + else if ( /AppleWebKit\/[0-9\.]*/ in unparsed_version ) + { + software_name = "Unspecified WebKit"; + parts = split_all(unparsed_version, /AppleWebKit\/[0-9\.]*/); + if ( 2 in parts ) + v = parse(parts[2])$version; + } + + return [$version=v, $unparsed_version=unparsed_version, $name=software_name]; } @@ -379,7 +366,7 @@ function software_fmt(i: Info): string # Insert a mapping into the table # Overides old entries for the same software and generates events if needed. -event software_register(id: conn_id, info: Info) +event register(id: conn_id, info: Info) { # Host already known? if ( info$host !in tracked ) @@ -407,7 +394,31 @@ function found(id: conn_id, info: Info): bool { if ( info$force_log || addr_matches_host(info$host, asset_tracking) ) { - event software_register(id, info); + if ( !info?$ts ) + info$ts=network_time(); + + if ( info?$version ) # we have a version number and don't have to parse. check if the name is also set... + { + if ( ! info?$name ) + { + Reporter::error("Required field name not present in Software::found"); + return F; + } + } + else # no version present, we have to parse... + { + if ( !info?$unparsed_version ) + { + Reporter::error("No unparsed version string present in Info record with version in Software::found"); + return F; + } + local sw = parse(info$unparsed_version); + info$unparsed_version = sw$unparsed_version; + info$name = sw$name; + info$version = sw$version; + } + + event register(id, info); return T; } else diff --git a/scripts/base/init-bare.bro b/scripts/base/init-bare.bro index 200947938d..9f4e0355f0 100644 --- a/scripts/base/init-bare.bro +++ b/scripts/base/init-bare.bro @@ -601,10 +601,10 @@ function add_signature_file(sold: string, snew: string): string } ## Signature files to read. Use ``redef signature_files += "foo.sig"`` to -## extend. Signature files will be searched relative to ``BRO_PATH``. +## extend. Signature files will be searched relative to ``BROPATH``. global signature_files = "" &add_func = add_signature_file; -## ``p0f`` fingerprint file to use. Will be searched relative to ``BRO_PATH``. +## ``p0f`` fingerprint file to use. Will be searched relative to ``BROPATH``. const passive_fingerprint_file = "base/misc/p0f.fp" &redef; # todo::testing to see if I can remove these without causing problems. diff --git a/scripts/policy/protocols/ftp/software.bro b/scripts/policy/protocols/ftp/software.bro index 8834e5cd2e..0ae963d552 100644 --- a/scripts/policy/protocols/ftp/software.bro +++ b/scripts/policy/protocols/ftp/software.bro @@ -23,7 +23,6 @@ event ftp_request(c: connection, command: string, arg: string) &priority=4 { if ( command == "CLNT" ) { - local si = Software::parse(arg, c$id$orig_h, CLIENT); - Software::found(c$id, si); + Software::found(c$id, [$unparsed_version=arg, $host=c$id$orig_h, $software_type=CLIENT]); } } diff --git a/scripts/policy/protocols/http/detect-webapps.bro b/scripts/policy/protocols/http/detect-webapps.bro index af9bc8a134..796da5c29a 100644 --- a/scripts/policy/protocols/http/detect-webapps.bro +++ b/scripts/policy/protocols/http/detect-webapps.bro @@ -27,7 +27,8 @@ event signature_match(state: signature_state, msg: string, data: string) &priori if ( /^webapp-/ !in state$sig_id ) return; local c = state$conn; - local si = Software::parse(msg, c$id$resp_h, WEB_APPLICATION); + local si = Software::Info; + si = [$name=msg, $unparsed_version=msg, $host=c$id$resp_h, $host_p=c$id$resp_p, $software_type=WEB_APPLICATION]; si$url = build_url_http(c$http); if ( c$id$resp_h in Software::tracked && si$name in Software::tracked[c$id$resp_h] ) diff --git a/scripts/policy/protocols/http/software-browser-plugins.bro b/scripts/policy/protocols/http/software-browser-plugins.bro index a035a2dcfc..b466a9da40 100644 --- a/scripts/policy/protocols/http/software-browser-plugins.bro +++ b/scripts/policy/protocols/http/software-browser-plugins.bro @@ -27,8 +27,7 @@ event http_header(c: connection, is_orig: bool, name: string, value: string) &pr # Flash doesn't include it's name so we'll add it here since it # simplifies the version parsing. value = cat("Flash/", value); - local flash_version = Software::parse(value, c$id$orig_h, BROWSER_PLUGIN); - Software::found(c$id, flash_version); + Software::found(c$id, [$unparsed_version=value, $host=c$id$orig_h, $software_type=BROWSER_PLUGIN]); } } else @@ -55,7 +54,7 @@ event log_http(rec: Info) local plugins = split(sw, /[[:blank:]]*;[[:blank:]]*/); for ( i in plugins ) - Software::found(rec$id, Software::parse(plugins[i], rec$id$orig_h, BROWSER_PLUGIN)); + Software::found(rec$id, [$unparsed_version=plugins[i], $host=rec$id$orig_h, $software_type=BROWSER_PLUGIN]); } } - } \ No newline at end of file + } diff --git a/scripts/policy/protocols/http/software.bro b/scripts/policy/protocols/http/software.bro index 8907d0853e..2ad4246a56 100644 --- a/scripts/policy/protocols/http/software.bro +++ b/scripts/policy/protocols/http/software.bro @@ -23,18 +23,18 @@ event http_header(c: connection, is_orig: bool, name: string, value: string) &pr if ( is_orig ) { if ( name == "USER-AGENT" && ignored_user_agents !in value ) - Software::found(c$id, Software::parse(value, c$id$orig_h, BROWSER)); + Software::found(c$id, [$unparsed_version=value, $host=c$id$orig_h, $software_type=BROWSER]); } else { if ( name == "SERVER" ) - Software::found(c$id, Software::parse(value, c$id$resp_h, SERVER)); + Software::found(c$id, [$unparsed_version=value, $host=c$id$resp_h, $host_p=c$id$resp_p, $software_type=SERVER]); else if ( name == "X-POWERED-BY" ) - Software::found(c$id, Software::parse(value, c$id$resp_h, APPSERVER)); + Software::found(c$id, [$unparsed_version=value, $host=c$id$resp_h, $host_p=c$id$resp_p, $software_type=APPSERVER]); else if ( name == "MICROSOFTSHAREPOINTTEAMSERVICES" ) { value = cat("SharePoint/", value); - Software::found(c$id, Software::parse(value, c$id$resp_h, APPSERVER)); + Software::found(c$id, [$unparsed_version=value, $host=c$id$resp_h, $host_p=c$id$resp_p, $software_type=APPSERVER]); } } } diff --git a/scripts/policy/protocols/smtp/software.bro b/scripts/policy/protocols/smtp/software.bro index 3c4c870885..f520485338 100644 --- a/scripts/policy/protocols/smtp/software.bro +++ b/scripts/policy/protocols/smtp/software.bro @@ -75,8 +75,7 @@ event log_smtp(rec: Info) if ( addr_matches_host(rec$id$orig_h, detect_clients_in_messages_from) ) { - local s = Software::parse(rec$user_agent, client_ip, s_type); - Software::found(rec$id, s); + Software::found(rec$id, [$unparsed_version=rec$user_agent, $host=client_ip, $software_type=s_type]); } } } diff --git a/scripts/policy/protocols/ssh/software.bro b/scripts/policy/protocols/ssh/software.bro index 44704ec845..ba03bed284 100644 --- a/scripts/policy/protocols/ssh/software.bro +++ b/scripts/policy/protocols/ssh/software.bro @@ -18,14 +18,12 @@ event ssh_client_version(c: connection, version: string) &priority=4 { # Get rid of the protocol information when passing to the software framework. local cleaned_version = sub(version, /^SSH[0-9\.\-]+/, ""); - local si = Software::parse(cleaned_version, c$id$orig_h, CLIENT); - Software::found(c$id, si); + Software::found(c$id, [$unparsed_version=cleaned_version, $host=c$id$orig_h, $software_type=CLIENT]); } event ssh_server_version(c: connection, version: string) &priority=4 { # Get rid of the protocol information when passing to the software framework. local cleaned_version = sub(version, /SSH[0-9\.\-]{2,}/, ""); - local si = Software::parse(cleaned_version, c$id$resp_h, SERVER); - Software::found(c$id, si); + Software::found(c$id, [$unparsed_version=cleaned_version, $host=c$id$resp_h, $host_p=c$id$resp_p, $software_type=SERVER]); } diff --git a/src/Brofiler.cc b/src/Brofiler.cc index 60e57f0964..783d027761 100644 --- a/src/Brofiler.cc +++ b/src/Brofiler.cc @@ -5,7 +5,7 @@ #include "util.h" Brofiler::Brofiler() - : delim('\t'), ignoring(0) + : ignoring(0), delim('\t') { } diff --git a/testing/btest/scripts/base/frameworks/software/version-parsing.bro b/testing/btest/scripts/base/frameworks/software/version-parsing.bro index dda3edea4b..03327a25cd 100644 --- a/testing/btest/scripts/base/frameworks/software/version-parsing.bro +++ b/testing/btest/scripts/base/frameworks/software/version-parsing.bro @@ -1,116 +1,116 @@ # @TEST-EXEC: bro %INPUT > output # @TEST-EXEC: btest-diff output -global ts = network_time(); -global host = 0.0.0.0; +module Software; -global matched_software: table[string] of Software::Info = { +global matched_software: table[string] of Software::Description = { ["OpenSSH_4.4"] = - [$name="OpenSSH", $version=[$major=4,$minor=4], $host=host, $ts=ts], + [$name="OpenSSH", $version=[$major=4,$minor=4], $unparsed_version=""], ["OpenSSH_5.2"] = - [$name="OpenSSH", $version=[$major=5,$minor=2], $host=host, $ts=ts], + [$name="OpenSSH", $version=[$major=5,$minor=2], $unparsed_version=""], ["Apache/2.0.63 (Unix) mod_auth_kerb/5.3 mod_ssl/2.0.63 OpenSSL/0.9.7a mod_fastcgi/2.4.2"] = - [$name="Apache", $version=[$major=2,$minor=0,$minor2=63,$addl="Unix"], $host=host, $ts=ts], + [$name="Apache", $version=[$major=2,$minor=0,$minor2=63,$addl="Unix"], $unparsed_version=""], ["Apache/1.3.19 (Unix)"] = - [$name="Apache", $version=[$major=1,$minor=3,$minor2=19,$addl="Unix"], $host=host, $ts=ts], + [$name="Apache", $version=[$major=1,$minor=3,$minor2=19,$addl="Unix"], $unparsed_version=""], ["ProFTPD 1.2.5rc1 Server (Debian)"] = - [$name="ProFTPD", $version=[$major=1,$minor=2,$minor2=5,$addl="rc1"], $host=host, $ts=ts], + [$name="ProFTPD", $version=[$major=1,$minor=2,$minor2=5,$addl="rc1"], $unparsed_version=""], ["wu-2.4.2-academ[BETA-18-VR14](1)"] = - [$name="wu", $version=[$major=2,$minor=4,$minor2=2,$addl="academ"], $host=host, $ts=ts], + [$name="wu", $version=[$major=2,$minor=4,$minor2=2,$addl="academ"], $unparsed_version=""], ["wu-2.6.2(1)"] = - [$name="wu", $version=[$major=2,$minor=6,$minor2=2,$addl="1"], $host=host, $ts=ts], + [$name="wu", $version=[$major=2,$minor=6,$minor2=2,$addl="1"], $unparsed_version=""], ["Java1.2.2-JDeveloper"] = - [$name="Java", $version=[$major=1,$minor=2,$minor2=2,$addl="JDeveloper"], $host=host, $ts=ts], + [$name="Java", $version=[$major=1,$minor=2,$minor2=2,$addl="JDeveloper"], $unparsed_version=""], ["Java/1.6.0_13"] = - [$name="Java", $version=[$major=1,$minor=6,$minor2=0,$addl="13"], $host=host, $ts=ts], + [$name="Java", $version=[$major=1,$minor=6,$minor2=0,$addl="13"], $unparsed_version=""], ["Python-urllib/3.1"] = - [$name="Python-urllib", $version=[$major=3,$minor=1], $host=host, $ts=ts], + [$name="Python-urllib", $version=[$major=3,$minor=1], $unparsed_version=""], ["libwww-perl/5.820"] = - [$name="libwww-perl", $version=[$major=5,$minor=820], $host=host, $ts=ts], + [$name="libwww-perl", $version=[$major=5,$minor=820], $unparsed_version=""], ["Wget/1.9+cvs-stable (Red Hat modified)"] = - [$name="Wget", $version=[$major=1,$minor=9,$addl="+cvs"], $host=host, $ts=ts], + [$name="Wget", $version=[$major=1,$minor=9,$addl="+cvs"], $unparsed_version=""], ["Wget/1.11.4 (Red Hat modified)"] = - [$name="Wget", $version=[$major=1,$minor=11,$minor2=4,$addl="Red Hat modified"], $host=host, $ts=ts], + [$name="Wget", $version=[$major=1,$minor=11,$minor2=4,$addl="Red Hat modified"], $unparsed_version=""], ["curl/7.15.1 (i486-pc-linux-gnu) libcurl/7.15.1 OpenSSL/0.9.8a zlib/1.2.3 libidn/0.5.18"] = - [$name="curl", $version=[$major=7,$minor=15,$minor2=1,$addl="i486-pc-linux-gnu"], $host=host, $ts=ts], + [$name="curl", $version=[$major=7,$minor=15,$minor2=1,$addl="i486-pc-linux-gnu"], $unparsed_version=""], ["Apache"] = - [$name="Apache", $host=host, $ts=ts], + [$name="Apache", $unparsed_version=""], ["Zope/(Zope 2.7.8-final, python 2.3.5, darwin) ZServer/1.1 Plone/Unknown"] = - [$name="Zope/(Zope", $version=[$major=2,$minor=7,$minor2=8,$addl="final"], $host=host, $ts=ts], + [$name="Zope/(Zope", $version=[$major=2,$minor=7,$minor2=8,$addl="final"], $unparsed_version=""], ["The Bat! (v2.00.9) Personal"] = - [$name="The Bat!", $version=[$major=2,$minor=0,$minor2=9,$addl="Personal"], $host=host, $ts=ts], + [$name="The Bat!", $version=[$major=2,$minor=0,$minor2=9,$addl="Personal"], $unparsed_version=""], ["Flash/10,2,153,1"] = - [$name="Flash", $version=[$major=10,$minor=2,$minor2=153,$addl="1"], $host=host, $ts=ts], + [$name="Flash", $version=[$major=10,$minor=2,$minor2=153,$addl="1"], $unparsed_version=""], ["mt2/1.2.3.967 Oct 13 2010-13:40:24 ord-pixel-x2 pid 0x35a3 13731"] = - [$name="mt2", $version=[$major=1,$minor=2,$minor2=3,$addl="967"], $host=host, $ts=ts], + [$name="mt2", $version=[$major=1,$minor=2,$minor2=3,$addl="967"], $unparsed_version=""], ["CacheFlyServe v26b"] = - [$name="CacheFlyServe", $version=[$major=26,$addl="b"], $host=host, $ts=ts], + [$name="CacheFlyServe", $version=[$major=26,$addl="b"], $unparsed_version=""], ["Apache/2.0.46 (Win32) mod_ssl/2.0.46 OpenSSL/0.9.7b mod_jk2/2.0.4"] = - [$name="Apache", $version=[$major=2,$minor=0,$minor2=46,$addl="Win32"], $host=host, $ts=ts], + [$name="Apache", $version=[$major=2,$minor=0,$minor2=46,$addl="Win32"], $unparsed_version=""], # I have no clue how I'd support this without a special case. #["Apache mod_fcgid/2.3.6 mod_auth_passthrough/2.1 mod_bwlimited/1.4 FrontPage/5.0.2.2635"] = - # [$name="Apache", $version=[], $host=host, $ts=ts], + # [$name="Apache", $version=[], $unparsed_version=""], ["Apple iPhone v4.3.1 Weather v1.0.0.8G4"] = - [$name="Apple iPhone", $version=[$major=4,$minor=3,$minor2=1,$addl="Weather"], $host=host, $ts=ts], + [$name="Apple iPhone", $version=[$major=4,$minor=3,$minor2=1,$addl="Weather"], $unparsed_version=""], ["Mozilla/5.0 (iPhone; U; CPU iPhone OS 4_3_2 like Mac OS X; en-us) AppleWebKit/533.17.9 (KHTML, like Gecko) Version/5.0.2 Mobile/8H7 Safari/6533.18.5"] = - [$name="Safari", $version=[$major=5,$minor=0,$minor2=2,$addl="Mobile"], $host=host, $ts=ts], + [$name="Safari", $version=[$major=5,$minor=0,$minor2=2,$addl="Mobile"], $unparsed_version=""], ["Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_7; en-US) AppleWebKit/534.16 (KHTML, like Gecko) Chrome/10.0.648.205 Safari/534.16"] = - [$name="Chrome", $version=[$major=10,$minor=0,$minor2=648,$addl="205"], $host=host, $ts=ts], + [$name="Chrome", $version=[$major=10,$minor=0,$minor2=648,$addl="205"], $unparsed_version=""], ["Opera/9.80 (Windows NT 6.1; U; sv) Presto/2.7.62 Version/11.01"] = - [$name="Opera", $version=[$major=11,$minor=1], $host=host, $ts=ts], + [$name="Opera", $version=[$major=11,$minor=1], $unparsed_version=""], ["Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.9.2.11) Gecko/20101013 Lightning/1.0b2 Thunderbird/3.1.5"] = - [$name="Thunderbird", $version=[$major=3,$minor=1,$minor2=5], $host=host, $ts=ts], + [$name="Thunderbird", $version=[$major=3,$minor=1,$minor2=5], $unparsed_version=""], ["iTunes/9.0 (Macintosh; Intel Mac OS X 10.5.8) AppleWebKit/531.9"] = - [$name="iTunes", $version=[$major=9,$minor=0,$addl="Macintosh"], $host=host, $ts=ts], + [$name="iTunes", $version=[$major=9,$minor=0,$addl="Macintosh"], $unparsed_version=""], ["Java1.3.1_04"] = - [$name="Java", $version=[$major=1,$minor=3,$minor2=1,$addl="04"], $host=host, $ts=ts], + [$name="Java", $version=[$major=1,$minor=3,$minor2=1,$addl="04"], $unparsed_version=""], ["Mozilla/5.0 (Linux; U; Android 2.3.3; zh-tw; HTC Pyramid Build/GRI40) AppleWebKit/533.1 (KHTML, like Gecko) Version/4.0 Mobile Safari/533.1"] = - [$name="Safari", $version=[$major=4,$minor=0,$addl="Mobile"], $host=host, $ts=ts], + [$name="Safari", $version=[$major=4,$minor=0,$addl="Mobile"], $unparsed_version=""], ["Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_6; en-us) AppleWebKit/533.20.25 (KHTML, like Gecko) Version/5.0.4 Safari/533.20.27"] = - [$name="Safari", $version=[$major=5,$minor=0,$minor2=4], $host=host, $ts=ts], + [$name="Safari", $version=[$major=5,$minor=0,$minor2=4], $unparsed_version=""], ["Mozilla/5.0 (iPod; U; CPU iPhone OS 4_0 like Mac OS X; en-us) AppleWebKit/532.9 (KHTML, like Gecko) Version/4.0.5 Mobile/8A293 Safari/6531.22.7"] = - [$name="Safari", $version=[$major=4,$minor=0,$minor2=5,$addl="Mobile"], $host=host, $ts=ts], + [$name="Safari", $version=[$major=4,$minor=0,$minor2=5,$addl="Mobile"], $unparsed_version=""], ["Opera/9.80 (J2ME/MIDP; Opera Mini/9.80 (S60; SymbOS; Opera Mobi/23.348; U; en) Presto/2.5.25 Version/10.54"] = - [$name="Opera Mini", $version=[$major=10,$minor=54], $host=host, $ts=ts], + [$name="Opera Mini", $version=[$major=10,$minor=54], $unparsed_version=""], ["Opera/9.80 (J2ME/MIDP; Opera Mini/5.0.18741/18.794; U; en) Presto/2.4.15"] = - [$name="Opera Mini", $version=[$major=5,$minor=0,$minor2=18741], $host=host, $ts=ts], + [$name="Opera Mini", $version=[$major=5,$minor=0,$minor2=18741], $unparsed_version=""], ["Opera/9.80 (Windows NT 5.1; Opera Mobi/49; U; en) Presto/2.4.18 Version/10.00"] = - [$name="Opera Mobi", $version=[$major=10,$minor=0], $host=host, $ts=ts], + [$name="Opera Mobi", $version=[$major=10,$minor=0], $unparsed_version=""], ["Mozilla/4.0 (compatible; MSIE 8.0; Android 2.2.2; Linux; Opera Mobi/ADR-1103311355; en) Opera 11.00"] = - [$name="Opera", $version=[$major=11,$minor=0], $host=host, $ts=ts], + [$name="Opera", $version=[$major=11,$minor=0], $unparsed_version=""], ["Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.2) Gecko/20040804 Netscape/7.2 (ax)"] = - [$name="Netscape", $version=[$major=7,$minor=2], $host=host, $ts=ts], + [$name="Netscape", $version=[$major=7,$minor=2], $unparsed_version=""], ["Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.0; GTB5; SLCC1; .NET CLR 2.0.50727; Media Center PC 5.0; .NET CLR 3.0.04506; InfoPath.2)"] = - [$name="MSIE", $version=[$major=7,$minor=0], $host=host, $ts=ts], + [$name="MSIE", $version=[$major=7,$minor=0], $unparsed_version=""], ["Mozilla/4.0 (compatible; MSIE 7.0b; Windows NT 5.1; Media Center PC 3.0; .NET CLR 1.0.3705; .NET CLR 1.1.4322; .NET CLR 2.0.50727; InfoPath.1)"] = - [$name="MSIE", $version=[$major=7,$minor=0,$addl="b"], $host=host, $ts=ts], + [$name="MSIE", $version=[$major=7,$minor=0,$addl="b"], $unparsed_version=""], ["Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; Trident/4.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; Tablet PC 2.0; InfoPath.2; InfoPath.3)"] = - [$name="MSIE", $version=[$major=8,$minor=0], $host=host, $ts=ts], + [$name="MSIE", $version=[$major=8,$minor=0], $unparsed_version=""], ["Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0)"] = - [$name="MSIE", $version=[$major=9,$minor=0], $host=host, $ts=ts], + [$name="MSIE", $version=[$major=9,$minor=0], $unparsed_version=""], ["Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.1; WOW64; Trident/5.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; .NET4.0C; .NET4.0E; InfoPath.3; Creative AutoUpdate v1.40.02)"] = - [$name="MSIE", $version=[$major=9,$minor=0], $host=host, $ts=ts], + [$name="MSIE", $version=[$major=9,$minor=0], $unparsed_version=""], ["Mozilla/5.0 (compatible; MSIE 10.0; Windows NT 6.1; WOW64; Trident/6.0)"] = - [$name="MSIE", $version=[$major=10,$minor=0], $host=host, $ts=ts], + [$name="MSIE", $version=[$major=10,$minor=0], $unparsed_version=""], ["The Bat! (3.0.1 RC3) Professional"] = - [$name="The Bat!", $version=[$major=3,$minor=0,$minor2=1,$addl="RC3"], $host=host, $ts=ts], + [$name="The Bat!", $version=[$major=3,$minor=0,$minor2=1,$addl="RC3"], $unparsed_version=""], # This is an FTP client (found with CLNT command) ["Total Commander"] = - [$name="Total Commander", $version=[], $host=host, $ts=ts], + [$name="Total Commander", $version=[], $unparsed_version=""], ["(vsFTPd 2.0.5)"] = - [$name="vsFTPd", $version=[$major=2,$minor=0,$minor2=5], $host=host, $ts=ts], + [$name="vsFTPd", $version=[$major=2,$minor=0,$minor2=5], $unparsed_version=""], ["Apple Mail (2.1084)"] = - [$name="Apple Mail", $version=[$major=2,$minor=1084], $host=host, $ts=ts], + [$name="Apple Mail", $version=[$major=2,$minor=1084], $unparsed_version=""], }; event bro_init() { for ( sw in matched_software ) { - local output = Software::parse(sw, host, Software::UNKNOWN); - local baseline: Software::Info; - baseline = matched_software[sw]; + local output = Software::parse(sw); + local baseline = matched_software[sw]; + if ( baseline$name == output$name && + sw == output$unparsed_version && Software::cmp_versions(baseline$version,output$version) == 0 ) print fmt("success on: %s", sw); else diff --git a/testing/scripts/coverage-calc b/testing/scripts/coverage-calc index 53e818fc32..cc5253c75c 100755 --- a/testing/scripts/coverage-calc +++ b/testing/scripts/coverage-calc @@ -7,7 +7,7 @@ # # The last argument is used to point to a root directory containing all # the Bro distribution's scripts. It's used to cull out test scripts -# that are not part of the distribution and which should not count towrads +# that are not part of the distribution and which should not count towards # the coverage calculation. import os @@ -24,22 +24,30 @@ for filename in glob.glob(inputglob): for line in f.read().splitlines(): parts = line.split("\t") exec_count = int(parts[0]) - location = os.path.normpath(parts[1]) + # grab file path and line numbers separately + filepath, srclines = parts[1].rsplit(",", 1) + filepath = os.path.normpath(filepath) # ignore scripts that don't appear to be part of Bro distribution - if not location.startswith(scriptdir): + if not filepath.startswith(scriptdir): continue + # keep only the line number (or line number range) + srclines = srclines.split()[1] + # For sorting purposes (so that line numbers get sorted correctly), + # construct a specially-formatted key string. + sortkey = filepath + ", line " + ("%6s" % srclines.split("-")[0]) + location = filepath + ", line " + srclines desc = parts[2] - # keying by location + desc may result in duplicate data + # Keying by location + desc may result in duplicate data # as some descs change as a result of differing configurations # producing record (re)definitions key = location if key in stats: stats[key][0] += exec_count else: - stats[key] = [exec_count, location, desc] + stats[key] = [exec_count, location, desc, sortkey] with open(outputfile, 'w') as f: - for k in sorted(stats, key=lambda i: stats[i][1]): + for k in sorted(stats, key=lambda i: stats[i][3]): f.write("%s\t%s\t%s\n" % (stats[k][0], stats[k][1], stats[k][2])) num_covered = 0