diff --git a/CHANGES b/CHANGES index 1fd1222e7c..e149b3f4d0 100644 --- a/CHANGES +++ b/CHANGES @@ -1,4 +1,49 @@ +2.2-beta-73 | 2013-10-14 14:28:25 -0700 + + * Fix misc. Coverity-reported issues (leaks, potential null pointer + deref, dead code, uninitialized values, + time-of-check-time-of-use). (Jon Siwek) + + * Add check for sqlite3 command to tests that require it. (Daniel + Thayer) + +2.2-beta-68 | 2013-10-14 09:26:09 -0700 + + * Add check for curl command to active-http.test. (Daniel Thayer) + +2.2-beta-64 | 2013-10-14 09:20:04 -0700 + + * Review usage of Reporter::InternalError, addresses BIT-1045. + + Replaced some with InternalWarning or AnalyzerError, the later + being a new method which signals the analyzer to not process + further input. (Jon Siwek) + + * Add new event for TCP content file write failures: + "contents_file_write_failure". (Jon Siwek) + +2.2-beta-57 | 2013-10-11 17:23:25 -0700 + + * Improve Broxygen end-of-sentence detection. (Jon Siwek) + +2.2-beta-55 | 2013-10-10 13:36:38 -0700 + + * A couple of new TLS extension numbers. (Bernhard Amann) + + * Suport for three more new TLS ciphers. (Bernhard Amann) + + * Removing ICSI notary from default site config. (Robin Sommer) + +2.2-beta-51 | 2013-10-07 17:33:56 -0700 + + * Polishing the reference and scripting sections of the manual. + (Robin Sommer) + + * Fixing the historical CHANGES record. (Robin Sommer) + + * Updating copyright notice. (Robin Sommer) + 2.2-beta-38 | 2013-10-02 11:03:29 -0700 * Fix uninitialized (or unused) fields. (Jon Siwek) @@ -4496,30 +4541,67 @@ - ClamAV support has been removed, which has been non-functional for a while already. -1.5.2.7 Sun Sep 12 19:39:49 PDT 2010 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ -- Addressed a number of lint nits (Vern Paxson). +1.5.3 Thu Mar 3 08:55:11 PST 2011 +- Removing aux/broctl/policy/cluster-addrs.hot.bro from the + distribution. The script is no longer needed and could in fact break + an installation because it redefines an old variable that has went + away. (Robin Sommer) -1.5.2.6 Sun Sep 12 17:00:13 PDT 2010 +- Smarter way to increase the communication module's pipe's socket + buffer size, resulting in a value closer to the allowed maximum. + (Craig Leres) + +- BroControl now also maintains links from the log archive to the + current set of logs when running in standalone mode. (Robin Sommer) + +- Bug fix for a file descriptor leak in the remote communication + module. (Scott Campbell) + +- Bug fix for BroControl to now activate trace-summary's sampling in + cluster mode, but not anymore in standalone mode. (Robin Sommer) + +- Broccoli updates: + + * Accept empty strings ("") as values in the configuration file. + (Craig Leres) + + * Support for specifying a separate host key for SSL-enabled + operation, with documentation update. (Craig Leres) + +1.5.2 Wed Jan 12 17:34:55 PST 2011 + +- Portability fixes for --enable-int64 (Vern Paxson). + +- Bug fix for Active Mapping support (Kevin Lo). + +- Broccoli compiler warning fixes (Kevin Lo). + +- Bug fixes for --enable-int64 and for avoiding bogus statistics / + bad memory references when generating profiling information upon + exit (Vern Paxson). + +- Bug fixes for terminating connections (Tyler Schoenke and Vern Paxson). + +- Removed now-quite-stale SSHv1 overflow detection, as it's more prone + to false positives than useful detection (Vern Paxson). - The SWIG file now explicitly lists those pieces from broccoli.h which it wants to wrap, rather than just including all of broccoli.h (Robin Sommer). - This fixes the problem that the SWIG bindings depend on what configure - finds out about the availability of libpcap even though the corresponding - functions don't need to be wrapped anyway. -- http-header.bro now includes a global include_header: set[string] - (Robin Sommer). If it contains any strings, then only those headers - will be processed. If left empty, then you continue to get the current - behavior of processing all headers. +- http-header.bro now includes a global "include_header: set[string]" If it + contains any strings, then only those headers will be processed. If left + empty, then you continue to get the current behavior of processing all + headers. (Robin Sommer). - Several changes to drop.bro (Robin Sommer): - * If true, the new flag Drop::dont_drop_locals indicates that + * If True, the new flag Drop::dont_drop_locals indicates that local hosts should never be dropped. On by default. - * If true, the new flag Drop::debugging activates extensive debugging + * If True, the new flag Drop::debugging activates extensive debugging output for the catch-and-release logic. Off by default. * The timeout for tracking dropping information is now 1 day @@ -4533,14 +4615,14 @@ Sommer). - The HTTP analyzer no longer attempts to track Server/User-Agent - versions, as these are hugely voluminous (Seth Hall). Ideally this - would still be available as an option for someone who truly wants - the full set. + versions, as these are hugely voluminous (Seth Hall). - HTTP and SMTP no longer have extra-short inactivity timeouts, as these were too often leading to premature expiration of a connection (Robin Sommer). +- Tracking of HTTP refer[r]er's by setting log_referrer. (Vern Paxson). + - The "rst" tool (aux/rst/) now takes an optional "-I " argument that instructs it to inject as payload rather than sending a RST packet (Vern Paxson). must be NUL-terminated, and the NUL is not @@ -4549,39 +4631,113 @@ - Bug fix for crashes in the DNS analyzer when processing replies for which no request was seen (Robin Sommer). +- Addressed a number of lint nits (Vern Paxson). -1.5.2.5 Mon Jul 19 16:20:58 PDT 2010 +- Rotation post-processors are now passed an additional argument + indicating whether Bro is terminating (Robin Sommer). -- Removed now-quite-stale SSHv1 overflow detection, as it's more prone - to false positives than useful detection (Vern Paxson). +- Bro now consistently generates a file_opened event for all fopen() calls. + (Robin Sommer). +- The "cf" utility now ignores a leading "t=" prefix, for compatibility + with Bro's "tagged" logging format (Robin Sommer). -1.5.2.4 Fri Jun 4 16:02:11 PDT 2010 +- You can now redefine the email_notice_to function (Robin Sommer). -- Bug fixes for terminating connections (Tyler Schoenke and Vern Paxson). +- Fix for packet processing resumption when a remote Bro dies during + state synchronization (Robin Sommer). +- OpenSSL/X509 portability fix, at long last (Gregor Maier & Christian + Kreibich). -1.5.2.3 Wed Mar 24 18:23:57 PDT 2010 +- Fix for compatibility with newer versions of autoconf (Gregor Maier). -- Bug fixes for --enable-int64 and for avoiding bogus statistics / - bad memory references when generating profiling information upon - exit (Vern Paxson). +- A larger BroControl update (Robin Sommer, if not marked otherwise): + o Increasing default timeouts for scan detector significantly. -1.5.2.2 Tue Jan 12 12:33:42 PST 2010 + o Increasing the manager's max_remote_events_processed to + something large, as it would slow down the process too much + otherwise and there's no other work to be interleaved with it + anyway. -- Broccoli compiler warning fixes (Kevin Lo). + o Adding debug output to cluster's part of catch-and-release + (extends the debugging already present in policy/debug.bro) + o Fixing typo in util.py. Closes #223. -1.5.2.1 Sun Jan 10 16:59:01 PST 2010 + o Added note to README pointing to HTML version. -- Bug fix for Active Mapping support (Kevin Lo). + o Disabling print_hook for proxies' remote.log. + o broctl's capstats now reports a total as well, and stats.log + tracks these totals. Closes #160. -1.5.2 Sat Dec 26 18:38:37 PST 2009 + o Avoiding spurious "waiting for lock" messages in cron mode. + Closes #206. -- Portability fixes for --enable-int64 (Vern Paxson). + o Bug fixes for installation on NFS. + o Bug fix for top command on FreeBSD 8. + + o crash-diag now checks whether gdb is available. + + o trace-summary reports the sample factor in use in its output, + and now also applies it to the top-local-networks output (not + doing the latter was a bug). + + o Removed the default twice-a-day rotation for conn.log. The + default rotation for conn.log now is now once every 24h, just + like for all other logs with the exception of mail.log (which is + still rotated twice a day, and thus the alarms are still mailed + out twice a day). + + o Fixed the problem of logs sometimes being filed into the wrong + directory (see the (now gone) FAQ entry in the README). + + o One can now customize the archive naming scheme. See the + corresponding FAQ entry in the README. + + o Cleaned up, and extended, collection of cluster statistics. + + ${logdir}/stats now looks like this: + + drwxr-xr-x 4 bro wheel 59392 Apr 5 17:55 . + drwxr-xr-x 96 bro wheel 2560 Apr 6 12:00 .. + -rw-r--r-- 1 bro wheel 576 Apr 6 16:40 meta.dat + drwxr-xr-x 2 bro wheel 2048 Apr 6 16:40 profiling + -rw-r--r-- 1 bro wheel 771834825 Apr 6 16:40 stats.log + drwxr-xr-x 2 bro wheel 2048 Apr 6 16:25 www + + stats.log accumulates cluster statistics collected every time + "cron" is called. + + - profiling/ keeps the nodes' prof.logs. + + - www/ keeps a subset of stats.log in CSV format for easy plotting. + + - meta.dat contains meta information about the current cluster + state (in particular which nodes we have, and when the last + stats update was done). + + Note that there is no Web setup yet to actually visualize the data in + www/. + + o BroControl now automatically maintains links inside today's log + archive directory pointing to the current live version of the + corresponding log file (if Bro is running). For example: + + smtp.log.11:52:18-current -> /usr/local/cluster/spool/manager/smtp.log + + o Alarms mailed out by BroControl now (1) have the notice msg in the + subject; and (2) come with the full mail.log entry in the body. + + o Fixing broctl's top output. (Seth Hall). + + o Fixing broctl's df output in certain situations. + + o BroControl fix for dealing with large vsize values reported by + "top" (Craig Leres). 1.5.1 Fri Dec 18 15:17:12 PST 2009 @@ -10513,8314 +10669,1450 @@ a sorted version (Ruoming Pang). It currently uses an N^2 algorithm so shouldn't be used on large arrays. +- The new built-in subst() substitutes all instances of a given pattern + in a given string with another string (Ruoming Pang). For example, + subst("fooboo", /oo/, "xxx") returns "fxxxbxxx". +- The new built-in cut_tail() trims characters from the end of a string + (Ruoming Pang). For example, cut_tail("fooboo", 2) returns "foob". +- sub_bytes() can now take a negative argument, which is with respect to + the end of the string rather than the beginning (Ruoming Pang). -2.1-826 | 2013-07-25 10:12:26 -0700 +- The new built-in md5_hmac() returns an HMAC-MD5 hash of the given string + (Ruoming Pang). The HMAC secret key is generated from available entropy + when Bro starts up, or it can be specified for repeatability using + the new -K flag. - * bif files declared with bif_target() are now automatically - compiled in. No more manual includes to pull them in. (Robin - Sommer) +- The new built-in split_all() is like split() except that the returned + array also includes the parts of the string that match the pattern + used for the splitting (Ruoming Pang). - * Covenience make target in testing/btest to update the three - coverage tests that usually need tweaking when scripts get - added/removed. (Robin Sommer) +- The new built-in split_n() splits up to a given number of instances, + optionally returning the parts matching the split pattern (Ruoming Pang). -2.1-824 | 2013-07-22 14:25:14 -0400 +- The new built-in split_complete() is the most general of the split functions + (the others are written in terms of it). It splits a given string + with separators that appear in either a given pattern or a given set + of strings, optionally including the text matching the separators in + its return value, and optionally limiting the number of matches to + a specified maximum. (Ruoming Pang) - * Fixed a scriptland state issue that manifested especially badly on proxies. (Seth Hall) +- The new built-in to_string_literal() returns an escaped version of a string + suitable to feeding into Bro's parser. For example, calling it on + "foo\nbar" (where '\n' is an embedded newline) returns "foo\x0abar". + (Ruoming Pang) - * Another test fix. (Robin Sommer) +- subst_substring() has been renamed subst_string (Ruoming Pang). - * Canonyfying the output of core.print-bpf-filters. (Robin Sommer) +- unescape_URI() no longer takes the connection as an argument (Ruoming Pang). -2.1-820 | 2013-07-18 12:30:04 -0700 +- config.guess and config.sub updated - * Extending external canonifier to remove fractional values from - capture_loss.log. (Robin Sommer) +- String escape expansion has been refined (Ruoming Pang) and some bugs + fixed. It now supports a format that's consistent with Bro string literals. - * Canonifying internal order for plugins and their components to - make it deterministic. (Robin Sommer) +- Scanning of octal and hex escape sequences now stops after 3 or 2 + characters, respectively. For example, "\0007" now expands to a + NUL followed by the character '7'. (Ruoming Pang) - * Small raw reader tweaks that got left our earlier. (Robin Sommer) - -2.1-814 | 2013-07-15 18:18:20 -0700 +- Bug fixes for handling of bare and 's when expected + (Ruoming Pang), and associated "weird" events. - * Fixing raw reader crash when accessing nonexistant file, and - memory leak when reading from file. Addresses #1038. (Bernhard - Amann) +- A bug in signature matching reassembly has been fixed (Robin Sommer). -2.1-811 | 2013-07-14 08:01:54 -0700 - - * Bump sqlite to 3.7.17. (Bernhard Amann) - - * Small test fixes. (Seth Hall) - - * Fix a bug where the same analyzer tag was reused for two different - analyzers. (Seth Hall) - - * Moved DPD signatures into script specific directories. Left out - the BitTorrent signatures pending further updates to that - analyzer. (Seth Hall) - -2.1-802 | 2013-07-10 10:55:14 -0700 - - * Const adjustment for methods. (Jon Siwek) - -2.1-798 | 2013-07-08 13:05:37 -0700 - - * Rewrite of the packet filter framework. (Seth Hall) - - This includes: - - - Plugin interface for adding filtering mechanisms. - - - Integrated the packet filter framework with the analyzer - framework to retrieve well-known ports from there. - - - Support for BPF-based load balancing (IPv4 and IPv6). This will - tie in with upcoming BroControl support for configuring this. - - - Support for BPF-based connection sampling. - - - Support for "shunting" traffic with BPF filters. - - - Replaced PacketFilter::all_packets with - PacketFilter::enable_auto_protocol_capture_filters. - -2.1-784 | 2013-07-04 22:28:48 -0400 - - * Add a call to lookup_connection in SSH scripts to update connval. (Seth Hall) - - * Updating submodule(s). (Robin Sommer) - -2.1-782 | 2013-07-03 17:00:39 -0700 - - * Remove the SSL log queueing mechanism that was included with the - log delay mechanism. (Seth Hall) - -2.1-780 | 2013-07-03 16:46:26 -0700 - - * Rewrite of the RAW input reader for improved robustness and new - features. (Bernhard Amann) This includes: - - - Send "end_of_data" event for all kind of streams. - - Send "process_finished" event with exit code of child - process at process termination. - - Expose name of input stream to readers. - - Better error handling. - - New "force_kill" option which SIGKILLs processes on reader termination. - - Supports reading from stdout and stderr simultaneously. - - Support sending data to stdin of child process. - - Streaming reads from external commands work without blocking. - -2.1-762 | 2013-07-03 16:33:22 -0700 - - * Fix to correct support for TLS 1.2. Addresses #1020. (Seth Hall, - with help from Rafal Lesniak). - -2.1-760 | 2013-07-03 16:31:36 -0700 - - * Teach broxygen to generate protocol analyzer plugin reference. - (Jon Siwek) - - * Adding 'const' to a number of C++ methods. (Jon Siwek) - -2.1-757 | 2013-07-03 16:28:10 -0700 - - * Fix redef of table index from clearing table. - - `redef foo["x"] = 1` now acts like `redef foo += { ["x"] = 1 }` - instead of `redef foo = { ["x"] = 1 }`. - - Addresses #1013. (Jon Siwek) - - -2.1-755 | 2013-07-03 16:22:43 -0700 - - * Add a general file analysis overview/how-to document. (Jon Siwek) - - * Improve file analysis doxygen comments. (Jon Siwek) - - * Improve tracking of HTTP file extraction. http.log now has files - taken from request and response bodies in different fields for - each, and can now track multiple files per body. That is, the - "extraction_file" field is now "extracted_request_files" and - "extracted_response_files". Addresses #988. (Jon Siwek) - - * Fix HTTP multipart body file analysis. Each part now gets assigned - a different file handle/id. (Jon Siwek) - - * Remove logging of analyzers field of FileAnalysis::Info. (Jon - Siwek) - - * Remove extraction counter in default file extraction scripts. (Jon - Siwek) - - * Remove FileAnalysis::postpone_timeout. - FileAnalysis::set_timeout_interval can now perform same function. - (Jon Siwek) - - * Make default get_file_handle handlers &priority=5 so they're - easier to override. (Jon Siwek) - - * Add input interface to forward data for file analysis. The new - Input::add_analysis function is used to automatically forward - input data on to the file analysis framework. (Jon Siwek) - - * File analysis framework interface simplifications. (Jon Siwek) - - - Remove script-layer data input interface (will be managed directly - by input framework later). - - - Only track files internally by file id hash. Chance of collision - too small to justify also tracking unique file string. - - -2.1-741 | 2013-06-07 17:28:50 -0700 - - * Fixing typo that could cause an assertion to falsely trigger. - (Robin Sommer) - -2.1-740 | 2013-06-07 16:37:32 -0700 - - * Fix for CMake 2.6.x. (Robin Sommer) - -2.1-738 | 2013-06-07 08:38:13 -0700 - - * Remove invalid free on non-allocated pointer in hash function - object. Addresses #1018. (Matthias Vallentin) - -2.1-736 | 2013-06-06 10:05:20 -0700 - - * New "magic constants" @DIR and @FILENAME that expand to the - directory path of the current script and just the script file name - without path, respectively. (Jon Siwek) - -2.1-731 | 2013-06-04 21:19:08 -0700 - - * Reorginization of internal protocol analyzer code. We're moving - them to a modularized structure, based on a plugin model. Along - with this change comes generic plugin infrastructure that we'll - later extend to other Bro component as well. For now all plugins - are compiled in statically, but in the future we plan to also - enable dynamic loading at run time. (Robin Sommer) - - * Ignoring file ids in external tests. (Robin Sommer) - -2.1-675 | 2013-06-02 20:03:19 -0700 - - * Fix a compiler warning. (Robin Sommer) - - * Allow named vector/set/table/record constructors. Addresses #983. - (Jon Siwek) - - * Adding Makefile target test-all that also runs the BroControl test - suite. (Robin Sommer) - -2.1-664 | 2013-05-28 21:37:46 -0700 - - * Dangling pointer fix. Addresses #1004. (Jon Siwek) - -2.1-659 | 2013-05-24 17:24:18 -0700 - - * Fix broken/missing documentation. (Jon Siwek) - - * Fixing test that would fail without ES/curl support. (Robin - Sommer) - -2.1-656 | 2013-05-17 15:58:07 -0700 - - * Fix mutex lock problem for writers. (Bernhard Amann) - -2.1-654 | 2013-05-17 13:49:52 -0700 - - * Tweaks to sqlite3 configuration to address threading issues. - (Bernhard Amann) - -2.1-651 | 2013-05-17 13:37:16 -0700 - - * Fix uninitialized DPM member. (Jon Siwek) - - * Fix issue with transaction ID reuse in a single DNS connection. (Seth Hall) - - * New function added to the queue.bro script to support peeking at - the new gettable item in the queue without removing it. (Seth Hall) - -2.1-647 | 2013-05-17 07:47:14 -0700 - - * Fixing Broxygen generation to have BROMAGIC set. (Robin Sommer) - - * Fix for 'fchmod undeclared here' on FreeBSD. (Robin Sommer) - - * CMake policy fix to avoid errors with older versions. (Robin - Sommer) - -2.1-641 | 2013-05-15 18:15:09 -0700 - - * Test update. (Robin Sommer) - -2.1-640 | 2013-05-15 17:24:09 -0700 - - * Support for cleaning up threads that have terminated. (Bernhard - Amann and Robin Sommer). Includes: - - - Both logging and input frameworks now clean up threads once - they aren't further needed anymnore. - - - New function Log::remove_stream() that removes a logging - stream, stopping all writer threads that are associated with - it. Note, however, that removing a *filter* from a stream - still doesn't clean up any threads. The problem is that - because of the output paths potentially being created - dynamically it's unclear if the writer thread will still be - needed in the future. - -2.1-626 | 2013-05-15 16:09:31 -0700 - - * Add "reservoir" sampler for SumStats framework. This maintains - a set of N uniquely distributed random samples. (Bernhard Amann) - -2.1-619 | 2013-05-15 16:01:42 -0700 - - * SQLite reader and writer combo. This allows to read/write - persistent data from on disk SQLite databases. The current - interface is quite low-level, we'll add higher-level abstractions - in the future. (Bernhard Amann) - -2.1-576 | 2013-05-15 14:29:09 -0700 - - * Initial version of new file analysis framework. This moves most of - the processing of file content from script-land into the core, - where it belongs. Much of this is an internal change, and at this - point the new code has essentially feature-equality with the old - one. More script-level changes to come. (Jon Siwek) - -2.1-502 | 2013-05-10 19:29:37 -0700 - - * Allow default function/hook/event parameters. Addresses #972. (Jon - Siwek) - - * Change the endianness parameter of bytestring_to_count() BIF to - default to false (big endian). (Jon Siwek) - -2.1-500 | 2013-05-10 19:22:24 -0700 - - * Fix to prevent merge-hook of SumStat's unique plugin from damaging - source data. (Bernhard Amann) - -2.1-498 | 2013-05-03 17:44:08 -0700 - - * Table lookups return copy of non-const &default vals. This - prevents unintentional modifications to the &default value itself. - Addresses #981. (Jon Siwek) - -2.1-496 | 2013-05-03 15:54:47 -0700 - - * Fix memory leak and unnecessary allocations in OpaqueVal. - Addresses #986. (Matthias Vallentin) - -2.1-492 | 2013-05-02 12:46:26 -0700 - - * Work-around for sumstats framework not propagating updates after - intermediate check in cluster environments. (Bernhard Amann) - - * Always apply tcp_connection_attempt. Before this change it was - only applied when a connection_attempt() event handler was - defined. (Robin Sommer) - - * Fixing coverage.bare-mode-errors test. (Robin Sommer) - -2.1-487 | 2013-05-01 18:03:22 -0700 - - * Always apply tcp_connection_attempt timer, even if no - connection_attempt() event handler is defined. (Robin Sommer) - -2.1-486 | 2013-05-01 15:28:45 -0700 - - * New framework for computing summary statistics in - base/framework/sumstats. This replaces the metrics frameworks, and - comes with a number of applications build on top, see NEWS. More - documentation to follow. (Seth Hall) - -2.1-397 | 2013-04-29 21:19:00 -0700 - - * Fixing memory leaks in CompHash implementation. Addresses #987. - (Robin Sommer) - -2.1-394 | 2013-04-27 15:02:31 -0700 - - * Fixed a bug in the vulnerable software script and added a test. - (Seth Hall) - - * Fix schedule statements used outside event handlers. Addresses - #974. (Jon Siwek) - - * Fix record coercion for default inner record fields. Addresses - #973. (Jon Siwek) - - * Add bytestring_to_count function to bro.bif. Addresses #968. (Yun - Zheng Hu) - -2.1-386 | 2013-03-22 12:41:50 -0700 - - * Added reverse() function to strings.bif. (Yun Zheng Hu) - -2.1-384 | 2013-03-22 12:10:14 -0700 - - * Fix record constructors in table initializer indices. Addresses - #660. (Jon Siwek) - -2.1-382 | 2013-03-22 12:01:34 -0700 - - * Add support for 802.1ah (Q-in-Q). Addresses #641. (Seth Hall) - -2.1-380 | 2013-03-18 12:18:10 -0700 - - * Fix gcc compile warnings in base64 encoder and benchmark reader. - (Bernhard Amann) - -2.1-377 | 2013-03-17 17:36:09 -0700 - - * Fixing potential leak in DNS error case. (Vlad Grigorescu) - -2.1-375 | 2013-03-17 13:14:26 -0700 - - * Add base64 encoding functionality, including new BiFs - encode_base64() and encode_base64_custom(). (Bernhard Amann) - - * Replace call to external "openssl" in extract-certs-pem.bro with - that encode_base64(). (Bernhard Amann) - - * Adding a test for extract-certs-pem.pem. (Robin Sommer) - - * Renaming Base64Decoder to Base64Converter. (Robin Sommer) - -2.1-366 | 2013-03-17 12:35:59 -0700 - - * Correctly handle DNS lookups for software version ranges. (Seth - Hall) - - * Improvements to vulnerable software detection. (Seth Hall) - - - Add a DNS based updating method. This needs to be tested - still. - - - Vulnerable version ranges are used now instead of only single - versions. This can deal with software with multiple stable - major versions. - - * Update software version parsing and comparison to account for a - third numeric subversion. Also, $addl is now compared numerically - if the value is actually numeric. (Seth Hall) - -2.1-361 | 2013-03-13 07:18:22 -0700 - - * Add check for truncated link frames. Addresses #962. (Jacob - Baines) - - * Fix large memory allocation in IP fragment reassembly. Addresses - #961. (Jacob Baines) - -2.1-357 | 2013-03-08 09:18:35 -0800 - - * Fix race-condition in table-event test. (Bernhard Amann) - - * s/bro-ids.org/bro.org/g. (Robin Sommer) - -2.1-353 | 2013-03-07 13:31:37 -0800 - - * Fix function type-equivalence requiring same parameter names. - Addresses #957. (Jon Siwek) - -2.1-351 | 2013-03-07 13:27:29 -0800 - - * Fix new/delete mismatch. Addresses #958. (Jacob Baines) - - * Fix compiler warnings. (Jon Siwek) - -2.1-347 | 2013-03-06 16:48:44 -0800 - - * Remove unused parameter from vector assignment method. (Bernhard Amann) - - * Remove the byte_len() and length() bifs. (Bernhard Amann) - -2.1-342 | 2013-03-06 15:42:52 -0800 - - * Moved the Notice::notice event and Notice::policy table to both be - hooks. See documentation and NEWS for information. (Seth Hall). - -2.1-338 | 2013-03-06 15:10:43 -0800 - - * Fix init of local sets/vectors via curly brace initializer lists. - (Jon Siwek) - -2.1-336 | 2013-03-06 15:08:06 -0800 - - * Fix memory leaks resulting from 'when' and 'return when' - statements. Addresses #946. (Jon Siwek) - - * Fix three bugs with 'when' and 'return when' statements. Addresses - #946. (Jon Siwek) - -2.1-333 | 2013-03-06 14:59:47 -0800 - - * Add parsing for GTPv1 extension headers and control messages. (Jon Siwek) - - This includes: - - - A new generic gtpv1_message() event generated for any GTP - message type. - - - Specific events for the create/update/delete PDP context - request/response messages. - - Addresses #934. - -2.1-331 | 2013-03-06 14:54:33 -0800 - - * Fix possible null pointer dereference in identify_data BIF. Also - centralized libmagic calls for consistent error handling/output. - (Jon Siwek) - - * Fix build on OpenBSD 5.2. (Jon Siwek) - -2.1-328 | 2013-02-05 01:34:29 -0500 - - * New script to query the ICSI Certificate Notary - (http://notary.icsi.berkeley.edu/) over DNS and add information - to the SSL log at runtime. (Matthias Vallentin) - - * Add delayed logging to SSL base scripts. (Matthias Vallentin) - -2.1-319 | 2013-02-04 09:45:34 -0800 - - * Update input tests to use exit_only_after_terminate. (Bernhard - Amann) - - * New option exit_only_after_terminate to prevent Bro from exiting. - If set, the main loop won't terminate before somebody calls - terminate(). (Robin Sommer) - -2.1-311 | 2013-02-01 08:03:01 -0800 - - * Updating submodule(s). - -2.1-310 | 2013-01-30 20:09:27 -0800 - - * Add an error for record coercions that would orphan a field. (Jon - Siwek) - - * Fixing several scripts where a field in an inlined record was - never removed after a code refactor. (Jon Siwek) - -2.1-307 | 2013-01-25 13:50:57 -0800 - - * Fix runaway reference counting bug in record coercion. (Jon Siwek) - - * Fix memory leak in some reporter messaging cases. (Jon Siwek) - -2.1-304 | 2013-01-23 19:43:27 -0800 - - * Making a test portable. (Robin Sommer) - -2.1-302 | 2013-01-23 16:17:29 -0800 - - * Refactoring ASCII formatting/parsing from loggers/readers into a - separate AsciiFormatter class. (Bernhard Amann) - - * Fix uninitialized locals in event/hook handlers from having a - value. Addresses #932. (Jon Siwek) - - * Add a null value check in CompositeHash::ComputeHash. Addresses - #930. (Jon Siwek) - - * Change reporter messages to more reliably print to stderr. - Addressed #930 (and revisits #836). (Jon Siwek) - - * Changing test=suite's btest call to use "-j" instead of "-j 5". - (Robin Sommer) - - * Require "case" blocks to end with either "break", "return", or a - new "fallthrough" statement that passes control on to the - subsequent case. This gives us the best mix of safety, - readability, and flexibility. Addresses #754. (Jon Siwek) - -2.1-279 | 2013-01-18 17:18:22 -0800 - - * Revert "Trick for parallelizing input framework unit tests." The - old way of doing the tests seems more reliable for now. (Jon - Siwek) - - * Fixing variable size issues with http response code in - ElasticSearch writer. (Gilbert Clark) - - * Removing unused class member. (Robin Sommer) - - * Add opaque type-ignoring for the accept_unsupported_types input - framework option. (Bernhard Amann) - -2.1-271 | 2013-01-08 10:18:57 -0800 - - * Change substring index notation to use a colon. String slice - notation is now written as `s[1:2]`. Addresses #422. (Jon Siwek) - -2.1-268 | 2013-01-07 09:43:44 -0800 - - * Fix memory leak in OpaqueType::DoUnserialize. (Jon Siwek) - -2.1-265 | 2012-12-20 17:38:42 -0800 - - * Add array-style index accessor for strings. Addresses #422. (Jon - Siwek) - - The index expression can take up to two indices for the start and - end index of the substring to return (e.g. "mystring[1,3]"). - Negative indices are allowed, with -1 representing the last - character in the string. The indexing is not cyclic -- if the - starting index is >= the length of the string an empty string is - returned, and if the ending index is >= the length of the string - then it's interpreted as the last index of the string. Assigning - to substrings accessed like this isn't allowed. - -2.1-263 | 2012-12-20 16:22:09 -0800 - - * Bro's language now has a new set of types "opaque of X". (Matthias - Vallentin) - - Opaque values can be passed around like other values but they can - only be manipulated with BiF functions, not with other operators. - Currently, the following opaque types are supported: - - - opaque of md5 - - opaque of sha1 - - opaque of sha256 - - opaquey of entropy. - - They go along with the corrsponding BiF functions md5_*, sha1_*, - sha256_*, and entropy_*, respectively. Note that these functions - have changed their signatures to work with opaques types rather - than global state as it was before. - -2.1-240 | 2012-12-20 15:21:07 -0800 - - * Improve error for invalid use of types as values. Addresses #923. - (Jon Siwek) - -2.1-238 | 2012-12-20 15:11:25 -0800 - - * Finish implementation of script-layer switch statement. Addresses - #754. (Jon Siwek) - - They behave like C-style switches except case labels can be - comprised of multiple literal constants delimited by commas. Only - atomic types are allowed for now. Case label bodies that don't - execute a "return" or "break" statement will fall through to - subsequent cases. A default case label is allowed. - - * Fix a case where c$resp$size is misrepresented. Addresses #730. - (Jon Siwek) - -2.1-234 | 2012-12-20 12:12:19 -0800 - - * Fix return value of hook calls that have no handlers. For this - case, the return value is always true. (Jon Siwek) - - * Fix to_port() BIF for port strings with a port number of zero. - (Jon Siwek) - -2.1-231 | 2012-12-14 14:51:35 -0800 - - * Make const variables actually constant. Both local and global - variables declared with "const" could be modified, but now - expressions that would modify them generate an error message at - parse-time. Addresses #922. (Jon Siwek) - -2.1-229 | 2012-12-14 14:46:12 -0800 - - * Fix memory leak in ASCII reader when encoutering errors in input. - (Bernhard Amann) - - * Improvements for the "bad checksums" detector to make it detect - bad TCP checksums. (Seth Hall) - -2.1-223 | 2012-12-12 14:25:15 -0800 - - * Trick for parallelizing input framework unit tests. Instead of - loading listen.bro to block until files are read, just read a pcap - file in pseudo-realtime. (Jon Siwek) - - * Fix reliability of a unit test that relies on when statements. - (Jon Siwek) - - * Remove unused attributes. (Daniel Thayer) - - Removed attributes &postprocessor and &match from documentation and source code. - - Removed undocumented attribute &attr from source code. - - Removed internal attribute "(&tracked)" from documentation. - -2.1-218 | 2012-12-10 14:45:04 -0800 - - * Add GPRS Tunnelling Protocol (GTPv1) decapsulation. This currently - supports automatic decapsulation of GTP-U packets on UDP port 2152. - The GTPv1 headers for such tunnels can be inspected by handling - the "gtpv1_g_pdu_packet" event, which has a parameter of type - "gtpv1_hdr". Addresses #690. (Jon Siwek; derived from patch by - Carsten Langer) - - * Change BinPAC exceptions in AYIYA/GTP analyzers to do - "protocol_violation". (Jon Siwek) - -2.1-212 | 2012-12-07 19:42:03 -0800 - - * Changing the HTTP parser to accept request methods in alignment - with the RFC. (Robin Sommer) - -2.1-209 | 2012-12-05 16:44:04 -0800 - - * Adapting the HTTP request line parsing to only accept methods - consisting of letters [A-Za-z]. (Robin Sommer) - -2.1-207 | 2012-12-05 15:47:32 -0800 - - * Reporting warnings if kill/waitpid fail in communication system. - (Bill Parker) - - * Replace() bzero with memset(). (Bill Parker) - - * Merge remote-tracking branch 'vlad/topic/vladg/http-verbs' - - * vlad/topic/vladg/http-verbs: - A test for HTTP methods, including some horribly illegal requests. - Remove hardcoded HTTP verbs from the analyzer (#741) - - I added a "bad_HTTP_request" weird for HTTP request lines that don't - have more than a single word. - - Closes #741. (Robin Sommer) - - * A test for HTTP methods, including some horribly illegal requests. (Vlad Grigorescu) - - * Remove hardcoded HTTP verbs from the analyzer (#741) (Vlad Grigorescu) - - -2.1-203 | 2012-12-05 14:36:56 -0800 - - * Fix segfault: Synchronization of state between connecting peers - now skips over identifiers that aren't initialized with a value - yet. Addresses #66. (Jon Siwek) - - * Fix segfault: Delete correct entry in error case in input - framework. (Bernhard Amann) - - * Bad record constructor initializers now give an error. Addresses - #34. (Jon Siwek) - - * Invalid vector indices now generate error message. Addresses #24. - (Jon Siwek) - - * Bump CPack RPM package requirement to Python >= 2.6.0. (Jon Siwek) - - * Interpreter exceptions occurring in "when" blocks are now handled. - Addresses #779 (Jon Siwek) - -2.1-195 | 2012-12-03 14:50:33 -0800 - - * Catching out-of-memory in patricia tree code. (Bill Parker) - -2.1-194 | 2012-12-03 14:36:26 -0800 - - * Renaming ASCII writer filter option 'only_single_header_row' to - 'tsv'. Also clarifying usage. Closes #912. (Robin Sommer) - -2.1-193 | 2012-12-03 14:11:14 -0800 - - * Fix a set of bugs with table/set attributes. (Jon Siwek) - - - Identifiers that are initialized with set()/table() constructor - expressions now inherit attributes from the expression. Before, - statements like - - const i: set[string] = set() &redef; - - associated the attribute with the set() constructor, but not the - "i" identifier, preventing redefinition. Addresses #866. - - - Allow &default attribute to apply to tables initialized as empty - (via either "{ }" or "table()") or if the expression supplied to it - can evaluate to a type that's promotable to the same yield type as - the table. - -2.1-191 | 2012-12-03 14:08:56 -0800 - - * Add test of record() constructor to table initializer unit test. - (Jon Siwek) - - * Fix table(), set(), vector() constructors in table initializer - lists. Also adds type checking of yield values to table() - constructor and fixes the type checking of yield values in - vector() constructor. Addresses #5. (Jon Siwek) - -2.1-188 | 2012-12-03 14:04:29 -0800 - - * Hook functions now callable with "hook" expression (i.e., hook is - no longer a statement). The return value of the call is an - implicit boolean value of T if all hook handlers ran, or F if one - hook handler exited as a result of a break statement and - potentially prevented other handlers from running. - - Scripts don't need to declare hooks with an explicit return type of bool - (internally, that's assumed), and any values given to (optional) return - statements in handler definitions are just ignored. - - Addresses #918. (Jon Siwek) - - * Clarification in hook documentation. (Jon Siwek) - -2.1-184 | 2012-12-03 13:59:50 -0800 - - * Slightly fix up file name extraction from Content-Disposition - headers. (Seth Hall) - - * Adding -b flag to bro in unit tests so they run faster. - - * Fixed a DNS attribute issue. Reported by Matt Thompson. (Seth - Hall) - - * Adding NEWS placeholder for hooks and CSV mode. (Robin Sommer) - -2.1-178 | 2012-11-23 19:35:32 -0800 - - * The ASCII writer now supports a new filter config option - "only_single_header_row" that turns the output into CSV format - when set to "T". (Carsten Langer) - - * Add new function flavor called a "hook". This new flavor of - function behaves like a "synchronous event". See - doc/scripts/builtins.rst more details on usage. (Jon Siwek) - - * Improve auto-generated enum documentation. The names of enum types - are tracked so that variables holding a value of a given enum type - can generate a reference to it instead of just listing the type as - a generic "enum". (Jon Siwek) - -2.1-171 | 2012-11-23 18:24:15 -0800 - - * Fix ambiguity between composite table index and record ctor - expressions. If a table type is "global t = table[conn_id, bool] - of count", then checking membership like "[c$id, is_orig] in t" - now works. Addresses #80. (Jon Siwek) - -2.1-169 | 2012-11-23 18:21:32 -0800 - - * Fix some warnings from sphinx when building docs. (Jon Siwek) - -2.1-167 | 2012-11-14 13:19:17 -0800 - - * Add a new BIF "bytestring_to_double" for converting from a binary - representation of a double. Addresses #908. (Carsten Langer/Daniel - Thayer) - -2.1-162 | 2012-11-13 17:29:00 -0800 - - * Fix modbus register array parsing. (Jon Siwek) - - * Adjustments to modbus test cases. (Jon Siwek) - -2.1-157 | 2012-11-08 16:22:00 -0800 - - * Fix for lookup_hostname BIF. (Jon Siwek) - - * Fix for modbus test portability. (Robin Sommer) - -2.1-152 | 2012-11-05 16:52:34 -0800 - - * Initial version of a completely reworked intelligence framework. - See doc/intel.rst for more information. (Seth Hall) - - * Experimental Modbus analyzer. See policy/protocols/modbus/* for - example policies. (Dina Hadziosmanovic, Seth Hall) - -2.1-112 | 2012-11-05 13:58:20 -0800 - - * New base script for detecting cases of checksum offloading. - Reporter messages will now tell if one has bad checksums. (Seth - Hall) - - * Clarifying ownership rules for BroString constructors. (Robin - Sommer) - -2.1-109 | 2012-11-05 13:39:34 -0800 - - * Add detection rate threshold for MHR. (Vlad Grigorescu) - - * lookup_hostname_txt fixes. (Vlad Grigorescu) - -2.1-104 | 2012-11-01 10:37:50 -0700 - - * A new built-in function lookup_hostname_txt() provides support for - DNS TXT queries. (Vlad Grigorescu) - -2.1-101 | 2012-10-31 14:30:26 -0700 - - * Documentation reorg: The install info has been consolidated into a - single document (INSTALL), the upgrade info has been moved from - the FAQ to a section in the install doc, and the "upgrading from - 1.5 to 2.0" document has been updated (and renamed) to also - include 2.0 to 2.1 upgrade info. (Daniel Thayer) - -2.1-96 | 2012-10-31 14:23:50 -0700 - - * Renaming option defining the frequency of alarm summary mails to - 'Logging::default_alarm_mail_interval'. (Daniel Thayer) - -2.1-91 | 2012-10-24 16:04:47 -0700 - - * Adding PPPoE support to Bro. (Seth Hall) - -2.1-87 | 2012-10-24 15:40:06 -0700 - - * Adding missing &redef for some TCP options. Addresses #905, #906, - #907. (Carsten Langer) - -2.1-86 | 2012-10-24 15:37:11 -0700 - - * Add parsing rules for IPv4/IPv6 subnet literal constants. - Addresses #888. (Jon Siwek) - -2.1-84 | 2012-10-19 15:12:56 -0700 - - * Added a BiF strptime() to wrap the corresponding C function. (Seth - Hall) - -2.1-82 | 2012-10-19 15:05:40 -0700 - - * Add IPv6 support to signature header conditions. (Jon Siwek) - - - "src-ip" and "dst-ip" conditions can now use IPv6 addresses/subnets. - They must be written in colon-hexadecimal representation and enclosed - in square brackets (e.g. [fe80::1]). Addresses #774. - - - "icmp6" is now a valid protocol for use with "ip-proto" and "header" - conditions. This allows signatures to be written that can match - against ICMPv6 payloads. Addresses #880. - - - "ip6" is now a valid protocol for use with the "header" condition. - (also the "ip-proto" condition, but it results in a no-op in that - case since signatures apply only to the inner-most IP packet when - packets are tunneled). This allows signatures to match specifically - against IPv6 packets (whereas "ip" only matches against IPv4 packets). - - - "ip-proto" conditions can now match against IPv6 packets. Before, - IPv6 packets were just silently ignored which meant DPD based on - signatures did not function for IPv6 -- protocol analyzers would only - get attached to a connection over IPv6 based on the well-known ports - set in the "dpd_config" table. - -2.1-80 | 2012-10-19 14:48:42 -0700 - - * Change how "gridftp" gets added to service field of connection - records. In addition to checking for a finished SSL handshake over - an FTP connection, it now also requires that the SSL handshake - occurs after the FTP client requested AUTH GSSAPI, more - specifically identifying the characteristics of GridFTP control - channels. Addresses #891. (Jon Siwek) - - * Allow faster rebuilds in certain cases. Previously, when - rebuilding with a different "--prefix" or "--scriptdir", all Bro - source files were recompiled. With this change, only util.cc is - recompiled. (Daniel Thayer) - -2.1-76 | 2012-10-12 10:32:39 -0700 - - * Add support for recognizing GridFTP connections as an extension to - the standard FTP analyzer. (Jon Siwek) - - This is enabled by default and includes: - - - An analyzer for GSI mechanism of GSSAPI FTP AUTH method. GSI - authentication involves an encoded TLS/SSL handshake over the - FTP control session. For FTP sessions that attempt GSI - authentication, the *service* field of the connection log will - include "gridftp" (as well as also "ftp" and "ssl"). - - - Add an example of a GridFTP data channel detection script. It - relies on the heuristics of GridFTP data channels commonly - default to SSL mutual authentication with a NULL bulk cipher - and that they usually transfer large datasets (default - threshold of script is 1 GB). The script also defaults to - skip_further_processing() after detection to try to save - cycles analyzing the large, benign connection. - - For identified GridFTP data channels, the *services* fields of - the connection log will include "gridftp-data". - - * Add *client_subject* and *client_issuer_subject* as &log'd fields - to SSL::Info record. Also add *client_cert* and - *client_cert_chain* fields to track client cert chain. (Jon Siwek) - - * Add a script in base/protocols/conn/polling that generalizes the - process of polling a connection for interesting features. The - GridFTP data channel detection script depends on it to monitor - bytes transferred. (Jon Siwek) - -2.1-68 | 2012-10-12 09:46:41 -0700 - - * Rename the Input Framework's update_finished event to end_of_data. - It will now not only fire after table-reads have been completed, - but also after the last event of a whole-file-read (or - whole-db-read, etc.). (Bernhard Amann) - - * Fix for DNS log problem when a DNS response is seen with 0 RRs. - (Seth Hall) - -2.1-64 | 2012-10-12 09:36:41 -0700 - - * Teach --disable-dataseries/--disable-elasticsearch to ./configure. - Addresses #877. (Jon Siwek) - - * Add --with-curl option to ./configure. Addresses #877. (Jon Siwek) - -2.1-61 | 2012-10-12 09:32:48 -0700 - - * Fix bug in the input framework: the config table did not work. - (Bernhard Amann) - -2.1-58 | 2012-10-08 10:10:09 -0700 - - * Fix a problem with non-manager cluster nodes applying - Notice::policy. This could, for example, result in duplicate - emails being sent if Notice::emailed_types is redef'd in local.bro - (or any script that gets loaded on all cluster nodes). (Jon Siwek) - -2.1-56 | 2012-10-03 16:04:52 -0700 - - * Add general FAQ entry about upgrading Bro. (Jon Siwek) - -2.1-53 | 2012-10-03 16:00:40 -0700 - - * Add new Tunnel::delay_teredo_confirmation option that indicates - that the Teredo analyzer should wait until it sees both sides of a - connection using a valid Teredo encapsulation before issuing a - protocol_confirmation. Default is on. Addresses #890. (Jon Siwek) - -2.1-50 | 2012-10-02 12:06:08 -0700 - - * Fix a typing issue that prevented the ElasticSearch timeout to - work. (Matthias Vallentin) - - * Use second granularity for ElasticSearch timeouts. (Matthias - Vallentin) - - * Fix compile issues with older versions of libcurl, which don't - offer *_MS timeout constants. (Matthias Vallentin) - -2.1-47 | 2012-10-02 11:59:29 -0700 - - * Fix for the input framework: BroStrings were constructed without a - final \0, which makes them unusable by basically all internal - functions (like to_count). (Bernhard Amann) - - * Remove deprecated script functionality (see NEWS for details). - (Daniel Thayer) - -2.1-39 | 2012-09-29 14:09:16 -0700 - - * Reliability adjustments to istate tests with network - communication. (Jon Siwek) - -2.1-37 | 2012-09-25 14:21:37 -0700 - - * Reenable some tests that previously would cause Bro to exit with - an error. (Daniel Thayer) - - * Fix parsing of large integers on 32-bit systems. (Daniel Thayer) - - * Serialize language.when unit test with the "comm" group. (Jon - Siwek) - -2.1-32 | 2012-09-24 16:24:34 -0700 - - * Fix race condition in language/when.bro test. (Daniel Thayer) - -2.1-26 | 2012-09-23 08:46:03 -0700 - - * Add an item to FAQ page about broctl options. (Daniel Thayer) - - * Add more language tests. We now have tests of all built-in Bro - data types (including different representations of constant - values, and max./min. values), keywords, and operators (including - special properties of certain operators, such as short-circuit - evaluation and associativity). (Daniel Thayer) - - * Fix construction of ip6_ah (Authentication Header) record values. - - Authentication Headers with a Payload Len field set to zero would - cause a crash due to invalid memory allocation because the - previous code assumed Payload Len would always be great enough to - contain all mandatory fields of the header. (Jon Siwek) - - * Update compile/dependency docs for OS X. (Jon Siwek) - - * Adjusting Mac binary packaging script. Setting CMAKE_PREFIX_PATH - helps link against standard system libs instead of ones that come - from other package manager (e.g. MacPorts). (Jon Siwek) - - * Adjusting some unit tests that do cluster communication. (Jon Siwek) - - * Small change to non-blocking DNS initialization. (Jon Siwek) - - * Reorder a few statements in scan.l to make 1.5msecs etc work. - Adresses #872. (Bernhard Amann) - -2.1-6 | 2012-09-06 23:23:14 -0700 - - * Fixed a bug where "a -= b" (both operands are intervals) was not - allowed in Bro scripts (although "a = a - b" is allowed). (Daniel - Thayer) - - * Fixed a bug where the "!=" operator with subnet operands was - treated the same as the "==" operator. (Daniel Thayer) - - * Add sleeps to configuration_update test for better reliability. - (Jon Siwek) - - * Fix a segfault when iterating over a set when using malformed - index. (Daniel Thayer) - -2.1 | 2012-08-28 16:46:42 -0700 - - * Make bif.identify_magic robust against FreeBSD's libmagic config. - (Robin Sommer) - - * Remove automatic use of gperftools on non-Linux systems. - --enable-perftools must now explicity be supplied to ./configure - on non-Linux systems to link against the tcmalloc library. - - * Fix uninitialized value for 'is_partial' in TCP analyzer. (Jon - Siwek) - - * Parse 64-bit consts in Bro scripts correctly. (Bernhard Amann) - - * Output 64-bit counts correctly on 32-bit machines (Bernhard Amann) - - * Input framework fixes, including: (Bernhard Amann) - - - One of the change events got the wrong parameters. - - - Escape commas in sets and vectors that were unescaped before - tokenization. - - - Handling of zero-length-strings as last element in a set was - broken (sets ending with a ,). - - - Hashing of lines just containing zero-length-strings was broken. - - - Make set_separators different from , work for input framework. - - - Input framework was not handling counts and ints out of - 32-bit-range correctly. - - - Errors in single lines do not kill processing, but simply ignore - the line, log it, and continue. - - * Update documentation for builtin types. (Daniel Thayer) - - - Add missing description of interval "msec" unit. - - - Improved description of pattern by clarifying the issue of - operand order and difference between exact and embedded - matching. - - * Documentation fixes for signature 'eval' conditions. (Jon Siwek) - - * Remove orphaned 1.5 unit tests. (Jon Siwek) - - * Add type checking for signature 'eval' condition functions. (Jon - Siwek) - - * Adding an identifier to the SMTP blocklist notices for duplicate - suppression. (Seth Hall) - -2.1-beta-45 | 2012-08-22 16:11:10 -0700 - - * Add an option to the input framework that allows the user to chose - to not die upon encountering files/functions. (Bernhard Amann) - -2.1-beta-41 | 2012-08-22 16:05:21 -0700 - - * Add test serialization to "leak" unit tests that use - communication. (Jon Siwek) - - * Change to metrics/basic-cluster unit test for reliability. (Jon - Siwek) - - * Fixed ack tracking which could overflow quickly in some - situations. (Seth Hall) - - * Minor tweak to coverage.bare-mode-errors unit test to work with a - symlinked 'scripts' dir. (Jon Siwek) - -2.1-beta-35 | 2012-08-22 08:44:52 -0700 - - * Add testcase for input framework reading sets (rather than - tables). (Bernhard Amann) - -2.1-beta-31 | 2012-08-21 15:46:05 -0700 - - * Tweak to rotate-custom.bro unit test. (Jon Siwek) - - * Ignore small mem leak every rotation interval for dataseries logs. - (Jon Siwek) - -2.1-beta-28 | 2012-08-21 08:32:42 -0700 - - * Linking ES docs into logging document. (Robin Sommer) - -2.1-beta-27 | 2012-08-20 20:06:20 -0700 - - * Add the Stream record to Log:active_streams to make more dynamic - logging possible. (Seth Hall) - - * Fix portability of printing to files returned by - open("/dev/stderr"). (Jon Siwek) - - * Fix mime type diff canonifier to also skip mime_desc columns. (Jon - Siwek) - - * Unit test tweaks/fixes. (Jon Siwek) - - - Some baselines for tests in "leaks" group were outdated. - - - Changed a few of the cluster/communication tests to terminate - more explicitly instead of relying on btest-bg-wait to kill - processes. This makes the tests finish faster in the success case - and makes the reason for failing clearer in the that case. - - * Fix memory leak of serialized IDs when compiled with - --enable-debug. (Jon Siwek) - -2.1-beta-21 | 2012-08-16 11:48:56 -0700 - - * Installing a handler for running out of memory in "new". Bro will - now print an error message in that case rather than abort with an - uncaught exception. (Robin Sommer) - -2.1-beta-20 | 2012-08-16 11:43:31 -0700 - - * Fixed potential problems with ElasticSearch output plugin. (Seth - Hall) - -2.1-beta-13 | 2012-08-10 12:28:04 -0700 - - * Reporter warnings and error now print to stderr by default. New - options Reporter::warnings_to_stderr and - Reporter::errors_to_stderr to disable. (Seth Hall) - -2.1-beta-9 | 2012-08-10 12:24:29 -0700 - - * Add more BIF tests. (Daniel Thayer) - -2.1-beta-6 | 2012-08-10 12:22:52 -0700 - - * Fix bug in input framework with an edge case. (Bernhard Amann) - - * Fix small bug in input framework test script. (Bernhard Amann) - -2.1-beta-3 | 2012-08-03 10:46:49 -0700 - - * Merge branch 'master' of ssh://git.bro-ids.org/bro (Robin Sommer) - - * Fix configure script to exit with non-zero status on error (Jon - Siwek) - - * Improve ASCII output performance. (Robin Sommer) - -2.1-beta | 2012-07-30 11:59:53 -0700 - - * Improve log filter compatibility with remote logging. Addresses - #842. (Jon Siwek) - -2.0-907 | 2012-07-30 09:13:36 -0700 - - * Add missing breaks to switch cases in - ElasticSearch::HTTPReceive(). (Jon Siwek) - -2.0-905 | 2012-07-28 16:24:34 -0700 - - * Fix log manager hanging on waiting for pending file rotations, - plus writer API tweak for failed rotations. Addresses #860. (Jon - Siwek and Robin Sommer) - - * Tweaking logs-to-elasticsearch.bro so that it doesn't do anything - if ES server is unset. (Robin Sommer) - -2.0-902 | 2012-07-27 12:42:13 -0700 - - * New variable in logging framework Log::active_streams to indicate - Log:ID enums which are currently active. (Seth Hall) - - * Reworked how the logs-to-elasticsearch scripts works to stop - abusing the logging framework. (Seth Hall) - - * Fix input test for recent default change on fastpath. (Robin - Sommer) - -2.0-898 | 2012-07-27 12:22:03 -0700 - - * Small (potential performance) improvement for logging framework. (Seth Hall) - - * Script-level rotation postprocessor fix. This fixes a problem with - writers that don't have a postprocessor. (Seth Hall) - - * Update input framework documentation to reflect want_record - change. (Bernhard Amann) - - * Fix crash when encountering an InterpreterException in a predicate - in logging or input Framework. (Bernhard Amann) - - * Input framework: Make want_record=T the default for events - (Bernhard Amann) - - * Changing the start/end markers in logs to open/close now - reflecting wall clock. (Robin Sommer) - -2.0-891 | 2012-07-26 17:15:10 -0700 - - * Reader/writer API: preventing plugins from receiving further - messages after a failure. (Robin Sommer) - - * New test for input framework that fails to find a file. (Robin - Sommer) - - * Improving error handling for threads. (Robin Sommer) - - * Tweaking the custom-rotate test to produce stable output. (Robin - Sommer) - -2.0-884 | 2012-07-26 14:33:21 -0700 - - * Add comprehensive error handling for close() calls. (Jon Siwek) - - * Add more test cases for input framework. (Bernhard Amann) - - * Input framework: make error output for non-matching event types - much more verbose. (Bernhard Amann) - -2.0-877 | 2012-07-25 17:20:34 -0700 - - * Fix double close() in FilerSerializer class. (Jon Siwek) - - * Fix build warnings. (Daniel Thayer) - - * Fixes to ElasticSearch plugin to make libcurl handle http - responses correctly. (Seth Hall) - - * Fixing FreeBSD compiler error. (Robin Sommer) - - * Silencing compiler warnings. (Robin Sommer) - -2.0-871 | 2012-07-25 13:08:00 -0700 - - * Fix complaint from valgrind about uninitialized memory usage. (Jon - Siwek) - - * Fix differing log filters of streams from writing to same - writer/path (which now produces a warning, but is otherwise - skipped for the second). Addresses #842. (Jon Siwek) - - * Fix tests and error message for to_double BIF. (Daniel Thayer) - - * Compile fix. (Robin Sommer) - -2.0-866 | 2012-07-24 16:02:07 -0700 - - * Correct a typo in usage message. (Daniel Thayer) - - * Fix file permissions of log files (which were created with execute - permissions after a recent change). (Daniel Thayer) - -2.0-862 | 2012-07-24 15:22:52 -0700 - - * Fix initialization problem in logging class. (Jon Siwek) - - * Input framework now accepts escaped ASCII values as input (\x##), - and unescapes appropiately. (Bernhard Amann) - - * Make reading ASCII logfiles work when the input separator is - different from \t. (Bernhard Amann) - - * A number of smaller fixes for input framework. (Bernhard Amann) - -2.0-851 | 2012-07-24 15:04:14 -0700 - - * New built-in function to_double(s: string). (Scott Campbell) - -2.0-849 | 2012-07-24 11:06:16 -0700 - - * Adding missing include needed on some systems. (Robin Sommer) - -2.0-846 | 2012-07-23 16:36:37 -0700 - - * Fix WriterBackend::WriterInfo serialization, reenable ascii - start/end tags. (Jon Siwek) - -2.0-844 | 2012-07-23 16:20:59 -0700 - - * Reworking parts of the internal threading/logging/input APIs for - thread-safety. (Robin Sommer) - - * Bugfix for SSL version check. (Bernhard Amann) - - * Changing a HTTP DPD from port 3138 to 3128. Addresses #857. (Robin - Sommer) - - * ElasticSearch logging writer. See logging-elasticsearch.rst for - more information. (Vlad Grigorescu and Seth Hall). - - * Give configure a --disable-perftools option to disable Perftools - support even if found. (Robin Sommer) - - * The ASCII log writer now includes "#start " and "#end - lines in the each file. (Robin Sommer) - - * Renamed ASCII logger "header" options to "meta". (Robin Sommer) - - * ASCII logs now escape '#' at the beginning of log lines. Addresses - #763. (Robin Sommer) - - * Fix bug, where in dns.log rcode always was set to 0/NOERROR when - no reply package was seen. (Bernhard Amann) - - * Updating to Mozilla's current certificate bundle. (Seth Hall) - -2.0-769 | 2012-07-13 16:17:33 -0700 - - * Fix some Info:Record field documentation. (Vlad Grigorescu) - - * Fix overrides of TCP_ApplicationAnalyzer::EndpointEOF. (Jon Siwek) - - * Fix segfault when incrementing whole vector values. Also removed - RefExpr::Eval(Val*) method since it was never called. (Jon Siwek) - - * Remove baselines for some leak-detecting unit tests. (Jon Siwek) - - * Unblock SIGFPE, SIGILL, SIGSEGV and SIGBUS for threads, so that - they now propagate to the main thread. Adresses #848. (Bernhard - Amann) - -2.0-761 | 2012-07-12 08:14:38 -0700 - - * Some small fixes to further reduce SOCKS false positive logs. (Seth Hall) - - * Calls to pthread_mutex_unlock now log the reason for failures. - (Bernhard Amann) - -2.0-757 | 2012-07-11 08:30:19 -0700 - - * Fixing memory leak. (Seth Hall) - -2.0-755 | 2012-07-10 16:25:16 -0700 - - * Add sorting canonifier to rotate-custom unit test. Addresses #846. - (Jon Siwek) - - * Fix many compiler warnings. (Daniel Thayer) - - * Fix segfault when there's an error/timeout resolving DNS requests. - Addresses #846. (Jon Siwek) - - * Remove a non-portable test case. (Daniel Thayer) - - * Fix typos in input framework doc. (Daniel Thayer) - - * Fix typos in DataSeries documentation. (Daniel Thayer) - - * Bugfix making custom rotate functions work again. (Robin Sommer) - - * Tiny bugfix for returning writer name. (Robin Sommer) - - * Moving make target update-doc-sources from top-level Makefile to - btest Makefile. (Robin Sommer) - -2.0-733 | 2012-07-02 15:31:24 -0700 - - * Extending the input reader DoInit() API. (Bernhard Amann). It now - provides a Info struct similar to what we introduced for log - writers, including a corresponding "config" key/value table. - - * Fix to make writer-info work when debugging is enabled. (Bernhard - Amann) - -2.0-726 | 2012-07-02 15:19:15 -0700 - - * Extending the log writer DoInit() API. (Robin Sommer) - - We now pass in a Info struct that contains: - - - the path name (as before) - - the rotation interval - - the log_rotate_base_time in seconds - - a table of key/value pairs with further configuration options. - - To fill the table, log filters have a new field "config: table[string] - of strings". This gives a way to pass arbitrary values from - script-land to writers. Interpretation is left up to the writer. - - * Split calc_next_rotate() into two functions, one of which is - thread-safe and can be used with the log_rotate_base_time value - from DoInit(). - - * Updates to the None writer. (Robin Sommer) - - - It gets its own script writers/none.bro. - - - New bool option LogNone::debug to enable debug output. It then - prints out all the values passed to DoInit(). - - - Fixed a bug that prevented Bro from terminating. - -2.0-723 | 2012-07-02 15:02:56 -0700 - - * Extract ICMPv6 NDP options and include in ICMP events. This adds - a new parameter of type "icmp6_nd_options" to the ICMPv6 neighbor - discovery events. Addresses #833. (Jon Siwek) - - * Set input frontend type before starting the thread. This means - that the thread type will be output correctly in the error - message. (Bernhard Amann) - -2.0-719 | 2012-07-02 14:49:03 -0700 - - * Fix inconsistencies in random number generation. The - srand()/rand() interface was being intermixed with the - srandom()/random() one. The later is now used throughout. (Jon - Siwek) - - * Changed the srand() and rand() BIFs to work deterministically if - Bro was given a seed file. Addresses #825. (Jon Siwek) - - * Updating input framework unit tests to make them more reliable and - execute quicker. (Jon Siwek) - - * Fixed race condition in writer and reader initializations. (Jon - Siwek) - - * Small tweak to make test complete quicker. (Jon Siwek) - - * Drain events before terminating log/thread managers. (Jon Siwek) - - * Fix strict-aliasing warning in RemoteSerializer.cc. Addresses - #834. (Jon Siwek) - - * Fix typos in event documentation. (Daniel Thayer) - - * Fix typos in NEWS for Bro 2.1 beta. (Daniel Thayer) - -2.0-709 | 2012-06-21 10:14:24 -0700 - - * Fix exceptions thrown in event handlers preventing others from running. (Jon Siwek) - - * Add another SOCKS command. (Seth Hall) - - * Fixed some problems with the SOCKS analyzer and tests. (Seth Hall) - - * Updating NEWS in preparation for beta. (Robin Sommer) - - * Accepting different AF_INET6 values for loopback link headers. - (Robin Sommer) - -2.0-698 | 2012-06-20 14:30:40 -0700 - - * Updates for the SOCKS analyzer (Seth Hall). - - - A SOCKS log! - - - Now supports SOCKSv5 in the analyzer and the DPD sigs. - - - Added protocol violations. - - * Updates to the tunnels framework. (Seth Hall) - - - Make the uid field optional since it's conceptually incorrect - for proxies being treated as tunnels to have it. - - - Reordered two fields in the log. - - - Reduced the default tunnel expiration interface to something - more reasonable (1 hour). - - * Make Teredo bubble packet parsing more lenient. (Jon Siwek) - - * Fix a crash in NetSessions::ParseIPPacket(). (Jon Siwek) - -2.0-690 | 2012-06-18 16:01:33 -0700 - - * Support for decapsulating tunnels via the new tunnel framework in - base/frameworks/tunnels. - - Bro currently supports Teredo, AYIYA, IP-in-IP (both IPv4 and - IPv6), and SOCKS. For all these, it logs the outher tunnel - connections in both conn.log and tunnel.log, and proceeds to - analyze the inner payload as if it were not tunneled, including - also logging it in conn.log (with a new tunnel_parents column - pointing back to the outer connection(s)). (Jon Siwek, Seth Hall, - Gregor Maier) - - * The options "tunnel_port" and "parse_udp_tunnels" have been - removed. (Jon Siwek) - -2.0-623 | 2012-06-15 16:24:52 -0700 - - * Changing an error in the input framework to a warning. (Robin - Sommer) - -2.0-622 | 2012-06-15 15:38:43 -0700 - - * Input framework updates. (Bernhard Amann) - - - Disable streaming reads from executed commands. This lead to - hanging Bros because pclose apparently can wait for eternity if - things go wrong. - - - Automatically delete disabled input streams. - - - Documentation. - -2.0-614 | 2012-06-15 15:19:49 -0700 - - * Remove an old, unused diff canonifier. (Jon Siwek) - - * Improve an error message in ICMP analyzer. (Jon Siwek) - - * Fix a warning message when building docs. (Daniel Thayer) - - * Fix many errors in the event documentation. (Daniel Thayer) - -2.0-608 | 2012-06-11 15:59:00 -0700 - - * Add more error handling code to logging of enum vals. Addresses - #829. (Jon Siwek) - -2.0-606 | 2012-06-11 15:55:56 -0700 - - * Fix summary lines for BIF documentation and corrected the - description of "fmt" and "floor" BIFs. (Daniel Thayer) - - * Fix val_size BIF tests and improve docs. (Daniel Thayer) - -2.0-602 | 2012-06-07 15:06:19 -0700 - - * Include header for usleep(), caused compile failure on Archlinux. (Jon Siwek) - - * Revert "Fixed a bug with the MIME analyzer not removing whitespace - on wrapped headers." Needs discussion. (Robin Sommer) - -2.0-598 | 2012-06-06 11:47:00 -0700 - - * Add @load-sigs directive for loading signature files (addresses - #551). This can be used to load signatures relative to the current - scripts (e.g., "@load-sigs ./foo.sig"). (Jon Siwek) - - -2.0-596 | 2012-06-06 11:41:00 -0700 - - * Fixes for some BiFs and their documentation. (Daniel Thayer) - - * Many new unit tests for BiFs. (Daniel Thayer) - -2.0-579 | 2012-06-06 11:04:46 -0700 - - * Memory leak fixes for bad usages of VectorVal ctor. (Jon Siwek) - - * Fixed a bug with the MIME analyzer not removing whitespace on - wrapped headers. (Seth Hall) - - * Change Input::update_finished lookup to happen at init time. (Jon Siwek) - - * Fix going through the internal_handler() function which will now - set the event as "used" (i.e. it's marked as being raised - somewhere). Addresses #823. (Jon Siwek) - - * Fix format specifier on RemoteSerializer::Connect. This caused - 32-bit systems to show a warning at compile-time, and fail when - connecting to peers. (Jon Siwek) - - * Fixes for running tests in parallel. (Robin Sommer) - -2.0-571 | 2012-05-30 19:12:43 -0700 - - * Updating submodule(s). - -2.0-570 | 2012-05-30 19:08:18 -0700 - - * A new input framework enables scripts to read in external data - dynamically on the fly as Bro is processing network traffic. - (Bernhard Amann) - - Currently, the framework supports reading ASCII input that's - structured similar as Bro's log files as well as raw blobs of - data. Other formats will come in the future. - - See doc/input.rst for more information (this will be extended - further soon). - -2.0-395 | 2012-05-30 17:03:31 -0700 - - * Remove unnecessary assert in ICMP analyzer which could lead to - aborts. Addresses #822. - - * Improve script debugger backtrace and print commands. (Jon Siwek) - - * Switching default DS compression to gzip. (Robin Sommer) - - * Improve availability of IPv6 flow label in connection records. - This adds a "flow_label" field to the "endpoint" record type, - which is used for both the "orig" and "resp" fields of - "connection" records. The new "connection_flow_label_changed" - event also allows tracking of changes in flow labels: it's raised - each time one direction of the connection starts using a different - label. (Jon Siwek) - - * Add unit tests for Broccoli SSL and Broccoli IPv6 connectivity. - (Jon Siwek) - - * Remove AI_ADDRCONFIG getaddrinfo hints flag for listening sockets. - (Jon Siwek) - - * Undo unnecessary communication protocol version bump. (Jon Siwek) - - * Add support to Bro for connecting with peers over IPv6. (Jon Siwek) - - - Communication::listen_ipv6 needs to be redef'd to true in order - for IPv6 listening sockets to be opened. - - - Added Communication::listen_retry option as an interval at which - to retry binding to socket addresses that were already in use. - - - Added some explicit baselines to check in the istate.events and - istate.events-ssl tests -- the SSL test was incorrectly passing - because it compared two empty files. (The files being empty - because "http/base" was given as an argument to Bro which it - couldn't handle because that script doesn't exist anymore). - - - Support for communication over non-global IPv6 addresses. This - usually requires specifying an additional zone identifier (see - RFC 4007). The connect() and listen() BIFs have been changed to - accept this zone identifier as an argument. - - -2.0-377 | 2012-05-24 16:46:06 -0700 - - * Documentation fixes. (Jon Siwek and Daniel Thayer) - -2.0-372 | 2012-05-17 13:59:45 -0700 - - * Fix compile errors. (Jon Siwek) - - * Linking in the DS docs. (Robin Sommer) - - * Fix mobility checksums unit test. (Jon Siwek) - -2.0-367 | 2012-05-17 12:42:30 -0700 - - * Adding support for binary output via DataSeries. See - logging-dataseries.rst for more information. (Gilbert Clark and - Robin Sommer) - - * Adding target update-doc-sources to top-level Makefile that runs - genDocSourcesList.sh. (Robin Sommer) - - * Moving trace for rotation test into traces directory. (Robin Sommer) - - * Fixing a rotation race condition at termination. (Robin Sommer) - - * Extending log post-processor call to include the name of the - writer. (Robin Sommer) - - * In threads, an internal error now immediately aborts. Otherwise, - the error won't make it back to the main thread for a while and - subsequent code in the thread would still execute. (Robin Sommer) - - * DataSeries cleanup. (Robin Sommer) - - * Fixing threads' DoFinish() method. It wasn't called reliably. Now, - it's always called before the thread is destroyed (assuming - processing has went normally so far). (Robin Sommer) - -2.0-341 | 2012-05-17 09:54:30 -0700 - - * Add a comment to explain the ICMPv6 error message types. (Daniel Thayer) - - * Quieting external test output somehwat. (Robin Sommer) - -2.0-336 | 2012-05-14 17:15:44 -0700 - - * Don't print the various "weird" events to stderr. Address #805. - (Daniel Thayer) - - * Generate icmp_error_message event for ICMPv6 error msgs. - Previously, icmp_sent was being generated, but icmp_error_message - contains more info. - - * Improved documentation comments for icmp-related events. (Daniel - Thayer) - -2.0-330 | 2012-05-14 17:05:56 -0700 - - * Add `addr_to_uri` script-level function that adds brackets to an - address if it's IPv6 and will be included in a URI or when a - ":" needs to be appended to it. (Jon Siwek) - - * Also add a test case for content extraction. (Jon Siwek) - - * Fix typos and improve INSTALL document. (Daniel Thayer) - - * Switching to new btest command TEST-SERIALIZE for communication - tests. (Robin Sommer) - -2.0-323 | 2012-05-04 21:04:34 -0700 - - * Add SHA1 and SHA256 hashing BIFs. Addresses #542. - - * Refactor all internal MD5 stuff to use OpenSSL's. (Jon Siwek) - - * Changes to open-file caching limits and uncached file unserialization. (Jon Siwek) - - - Unserializing files that were previously kicked out of the open-file - cache would cause them to be fopen'd with the original access - permissions which is usually 'w' and causes truncation. They - are now opened in 'a' mode. (addresses #780) - - - Add 'max_files_in_cache' script option to manually set the maximum - amount of opened files to keep cached. Mainly this just helped - to create a simple test case for the above change. - - - Remove unused NO_HAVE_SETRLIMIT preprocessor switch. - - - On systems that don't enforce a limit on number of files opened for - the process, raise default max size of open-file cache from - 32 to 512. - -2.0-319 | 2012-05-03 13:24:44 -0700 - - * SSL bugfixes and cleanup. (Seth Hall) - - - SSL related files and classes renamed to remove the "binpac" term. - - - A small fix for DPD scripts to make the DPD log more helpful if - there are multiple continued failures. - - - Fixed the SSL analyzer to make it stop doing repeated violation - messages for some handshake failures. - - - Added a $issuer_subject to the SSL log. - - - Created a basic test for SSL. - - - Fixed parsing of TLS server extensions. (Seth Hall) - -2.0-315 | 2012-05-03 11:44:17 -0700 - - * Add two more TLS extension values that we see in live traffic. - (Bernhard Amann) - - * Fixed IPv6 link local unicast CIDR and added IPv6 loopback to - private address space. (Seth Hall) - - * Fixed a problem where cluster workers were still processing - notices in some cases. (Seth Hall) - - * Added a configure option to specify the 'etc' directory. Addresses - #801. (Daniel Thayer) - - -2.0-306 | 2012-04-24 14:37:00 -0700 - - * Add further TLS extension values "extended_random" and - "heartbeat". (Seth Hall) - - * Fix problem with extracting FTP passwords and add "ftpuser" as - another anonymous username. (Seth Hall, discovered by Patrik - Lundin). - -2.0-303 | 2012-04-19 10:01:06 -0700 - - * Changes related to ICMPv6 Neighbor Discovery messages. (Jon Siwek) - - - The 'icmp_conn' record now contains an 'hlim' field since hop limit - in the IP header is an interesting field for at least these ND - messages. - - - Fixed and extended 'icmp_router_advertisement' event parameters. - - - Changed 'icmp_neighbor_advertisement' event parameters to add - more of the known boolean flags. - -2.0-301 | 2012-04-17 17:58:55 -0700 - - * Bro now support ICMPv6. (Matti Mantere, Jon Siwek, Robin Sommer, - Daniel Thayer). - - Overall, Bro now raises the following ICMP events for v4 and v6 as - appropiate: - - event icmp_sent(c: connection, icmp: icmp_conn); - event icmp_echo_request(c: connection, icmp: icmp_conn, id: count, seq: count, payload: string); - event icmp_echo_reply(c: connection, icmp: icmp_conn, id: count, seq: count, payload: string); - event icmp_error_message(c: connection, icmp: icmp_conn, code: count, context: icmp_context); - event icmp_unreachable(c: connection, icmp: icmp_conn, code: count, context: icmp_context); - event icmp_packet_too_big(c: connection, icmp: icmp_conn, code: count, context: icmp_context); - event icmp_time_exceeded(c: connection, icmp: icmp_conn, code: count, context: icmp_context); - event icmp_parameter_problem(c: connection, icmp: icmp_conn, code: count, context: icmp_context); - event icmp_router_solicitation(c: connection, icmp: icmp_conn); - event icmp_router_advertisement(c: connection, icmp: icmp_conn, hop_limit: count, managed: bool, router_lifetime: count, reachable_time: interval, retrans_timer: interval); - event icmp_neighbor_solicitation(c: connection, icmp: icmp_conn, tgt:addr); - event icmp_neighbor_advertisement(c: connection, icmp: icmp_conn, tgt:addr); - event icmp_redirect(c: connection, icmp: icmp_conn, tgt: addr, dest: addr); - - The `icmp_conn` record got a new boolean field 'v6' that indicates - whether the ICMP message is v4 or v6. - - This change also includes further low-level work on existing IP - and ICMP code, including a reorganization of how ICMPv4 is - handled. - -2.0-281 | 2012-04-17 17:40:39 -0700 - - * Small updates for the bittorrent analyzer to support 64bit types - in binpac. (Seth Hall) - - * Removed the attempt at bittorrent resynchronization. (Seth Hall) - -2.0-276 | 2012-04-17 17:35:56 -0700 - - * Add more support for 's that lack some structure - definitions. (Jon Siwek) - -2.0-273 | 2012-04-16 18:08:56 -0700 - - * Removing QR flag from DNS log in response, which should not have - been there in the first place. (Seth Hall) - - * Sync up patricia.c/h with pysubnettree repo. (Daniel Thayer) - - * Adding missing leak groups to a couple tests. Also activating leak - checking for proxy in basic-cluster test. (Robin Sommer) - -2.0-267 | 2012-04-09 17:47:28 -0700 - - * Add support for mobile IPv6 Mobility Header (RFC 6275). (Jon - Siwek) - - - Enabled through a new --enable-mobile-ipv6 configure-time - option. If not enabled, the mobility header (routing type 2) and - Home Address Destination option are ignored. - - - Accessible at script-layer through 'mobile_ipv6_message' event. - - * Refactor IP_Hdr routing header handling, add MobileIPv6 Home - Address handling. Packets that use the Home Address Destination - option use that option's address as the connection's originator. - (Jon Siwek) - - * Revert TCP checksumming to cache common data, like it did before. - (Jon Siwek) - - * Improve handling of IPv6 routing type 0 extension headers. (Jon - Siwek) - - - flow_weird event with name argument value of "routing0_hdr" is raised - for packets containing an IPv6 routing type 0 header because this - type of header is now deprecated according to RFC 5095. - - - Packets with a routing type 0 header and non-zero segments left - now use the last address in that header in order to associate - with a connection/flow and for calculating TCP/UDP checksums. - - - Added a set of IPv4/IPv6 TCP/UDP checksum unit tests (Jon Siwek) - - * Fix table expiry for values assigned in bro_init() when reading - live. (Jon Siwek) - -2.0-257 | 2012-04-05 15:32:43 -0700 - - * Fix CMake from warning about unused ENABLE_PERFTOOLS_DEBUG - variable. (Jon Siwek) - - * Fix handling of IPv6 atomic fragments. (Jon Siwek) - - * Fix that prevents Bro processes that do neither local logging nor - request remote logs from spawning threads. (Robin Sommer) - - * Fixing perftools-debug support. (Robin Sommer) - - * Reverting SocketComm change tuning I/O behaviour. (Robin Sommer) - - * Adding notice_policy.log canonification for external tests. (Robin Sommer) - - -2.0-245 | 2012-04-04 17:25:20 -0700 - - * Internal restructuring of the logging framework: we now spawn - threads doing the I/O. From a user's perspective not much should - change, except that the OS may now show a bunch of Bro threads. - (Gilbert Clark and Robin Sommer). - - * When building Bro, we now always link in tcmalloc if it's found at - configure time. If it's installed but not picked up, - --with-perftools may help. (Robin Sommer) - - * Renaming the configure option --enable-perftools to - --enable-perftool-debug to indicate that the switch is only - relevant for debugging the heap. It's not needed to pick up - tcmalloc for better performance. (Robin Sommer) - -2.0-184 | 2012-03-28 15:11:11 -0700 - - * Improve handling of IPv6 Routing Type 0 headers. (Jon Siwek) - - - For RH0 headers with non-zero segments left, a - "routing0_segleft" flow_weird event is raised (with a - destination indicating the last address in the routing header), - and an "rh0_segleft" event can also be handled if the other - contents of the packet header are of interest. No further - analysis is done as the complexity required to correctly - identify destination endpoints of connections doesn't seem worth - it as RH0 has been deprecated by RFC 5095. - - - For RH0 headers without any segments left, a "routing0_header" - flow_weird event is raised, but further analysis still occurs as - normal. - -2.0-182 | 2012-03-28 15:01:57 -0700 - - * Remove dead tcp_checksum function from net_util. (Jon Siwek) - - * Change routing0_data_to_addrs BIF to return vector of addresses. - The order of addresses in type 0 routing headers is - interesting/important. (Jon Siwek) - - -2.0-179 | 2012-03-23 17:43:31 -0700 - - * Remove the default "tcp or udp or icmp" filter. In default mode, - Bro would load the packet filter script framework which installs a - filter that allows all packets, but in bare mode (the -b option), - this old filter would not follow IPv6 protocol chains and thus - filter out packets with extension headers. (Jon Siwek) - - * Update PacketFilter/Discarder code for IP version independence. - (Jon Siwek) - - * Fix some IPv6 header related bugs. (Jon Siwek) - - * Add IPv6 fragment reassembly. (Jon Siwek) - - * Add handling for IPv6 extension header chains. Addresses #531. - (Jon Siwek) - - - The script-layer 'pkt_hdr' type is extended with a new 'ip6' field - representing the full IPv6 header chain. - - - The 'new_packet' event is now raised for IPv6 packets. Addresses - #523. - - - A new event called 'ipv6_ext_header' is raised for any IPv6 - packet containing extension headers. - - - A new event called 'esp_packet' is raised for any packets using - ESP ('new_packet' and 'ipv6_ext_header' events provide - connection info, but that info can't be provided here since the - upper-layer payload is encrypted). - - - The 'unknown_protocol' weird is now raised more reliably when - Bro sees a transport protocol or IPv6 extension header it can't - handle. Addresses #522. - - * Add unit tests for IPv6 fragment reassembly, ipv6_ext_headers and - esp_packet events. (Jon Siwek) - - * Adapt FreeBSD's inet_ntop implementation for internal use. Now we - get consistent text representations of IPv6 addresses across - platforms. (Jon Siwek) - - * Update documentation for new syntax of IPv6 literals. (Jon Siwek) - - -2.0-150 | 2012-03-13 16:16:22 -0700 - - * Changing the regular expression to allow Site::local_nets in - signatures. (Julien Sentier) - - * Removing a line of dead code. Found by . Closes #786. (Julien - Sentier) - -2.0-146 | 2012-03-13 15:39:38 -0700 - - * Change IPv6 literal constant syntax to require encasing square - brackets. (Jon Siwek) - -2.0-145 | 2012-03-09 15:10:35 -0800 - - * Remove the match expression. 'match' and 'using' are no longer - keywords. Addressed #753. (Jon Siwek) - -2.0-143 | 2012-03-09 15:07:42 -0800 - - * Fix a BRO_PROFILER_FILE/mkstemp portability issue. Addresses #794. - (Jon Siwek) - -2.0-139 | 2012-03-02 09:33:04 -0800 - - * Changes to how script coverage integrates with test suites. (Jon Siwek) - - - BRO_PROFILER_FILE now passes .X* templated filenames to mkstemp - for generating unique coverage state files. - - - Rearranging Makefile targets. The general rule is that if the - all/brief target fails out due to a test failure, then the dependent - coverage target won't run, but can still be invoked directly later. - (e.g. make brief || make coverage) - - * Standardized on the &default function for SSL constants. (Seth - Hall) - - * Adding btest group "leaks" to leak tests. (Robin Sommer) - - * Adding btest group "comm" to communication tests for parallelizing - execution with new btest version. (Robin Sommer) - - * Sorting all output for diffing in the external tests. (Robin - Sommer) - - * Cleaned up dead code from the old SSL analyzers. Reported by - Julien Sentier. (Seth Hall) - - * Update/add tests for broccoli IPv6 addr/subnet support. Addresses - #448. (Jon Siwek) - - * Remove connection compressor. Addresses #559. (Jon Siwek) - - * Refactor IP_Hdr class ctors. Addresses #532. (Jon Siwek) - - -2.0-121 | 2012-02-24 16:34:17 -0800 - - * A number of smaller memory fixes and code cleanups. (Julien - Sentier) - - * Add to_subnet bif. Fixes #782). (Jon Siwek) - - * Fix IPAddr::Mask/ReverseMask not allowing argument of 0. (Jon - Siwek) - - * Refactor IPAddr v4 initialization from string. Fixes #775. (Jon Siwek) - - * Parse the dotted address string directly instead of canonicalizing - and passing to inet_pton. (Jon Siwek) - - -2.0-108 | 2012-02-24 15:21:07 -0800 - - * Refactoring a number of usages of new IPAddr class. (Jon Siwek) - - * Fixed a bug in remask_addr bif. (Jon Siwek) - -2.0-106 | 2012-02-24 15:02:20 -0800 - - * Raise minimum required CMake version to 2.6.3. (Jon Siwek) - -2.0-104 | 2012-02-24 14:59:12 -0800 - - * Add test case for FTP over IPv4. (Daniel Thayer) - - * Fix IPv6 URLs in ftp.log. (Daniel Thayer) - - * Add a test for FTP over IPv6 (Daniel Thayer) - - * Fix parsing of FTP EPRT command and EPSV response. (Daniel Thayer) - -2.0-95 | 2012-02-22 05:27:34 -0800 - - * GeoIP installation documentation update. (Seth Hall) - - * Decrease strictness of parsing IPv4 strings into addrs. Fixes #775. (Jon Siwek) - - * Fix memory leak in DNS manager. Fixes #777. (Jon Siwek) - - * Fix IPAddr/IPPrefix serialization bugs. (Jon Siwek) - - * Fix compile error. (Jon Siwek) - -2.0-86 | 2012-02-17 15:41:06 -0800 - - * Changing ARP detection to always kick in even if no analyzer is - activated. (Robin Sommer) - - * DNS name lookups performed by Bro now also query AAAA records. - DNS_Mgr handles combining the results of the A and AAAA queries - for a given hostname such that at the scripting layer, the name - resolution can yield a set with both IPv4 and IPv6 addresses. (Jon - Siwek) - - * Add counts_to_addr and addr_to_counts conversion BIFs. (Jon Siwek) - - * Change HashKey threshold for using H3 to 36 bytes. (Jon Siwek) - - * Remove mention of --enable-brov6 in docs. (Daniel Thayer) - - * Remove --enable-brov6 from configure usage text (Daniel Thayer) - - * Add a test and baseline for addr_to_ptr_name BiF. (Daniel Thayer) - - * Adding a test and baseline for ptr_name_to_addr BiF. (Seth Hall) - - * Fix the ptr_name_to_addr BiF to work with IPv6 (Daniel Thayer) - - * Fix a memory leak that perftools now complains about. (Jon Siwek) - - * Remove --enable-brov6 flag, IPv6 now supported by default. (Jon Siwek) - - Some script-layer changes of note: - - - dns_AAAA_reply event signature changed: the string representation - of an IPv6 addr is easily derived from the addr value, it doesn't - need to be another parameter. This event also now generated directly - by the DNS analyzer instead of being "faked" into a dns_A_reply event. - - - Removed addr_to_count BIF. It used to return the host-order - count representation of IPv4 addresses only. To make it more - generic, we might later add a BIF to return a vector of counts - in order to support IPv6. - - - Changed the result of enclosing addr variables in vertical pipes - (e.g. |my_addr|) to return the bit-width of the address type which - is 128 for IPv6 and 32 for IPv4. It used to function the same - way as addr_to_count mentioned above. - - - Remove bro_has_ipv6 BIF - -2.0-57 | 2012-02-10 00:02:35 -0800 - - * Fix typos in the documentation. (Daniel Thayer) - - * Fix compiler warning about Brofiler ctor init list order. (Jon Siwek) - - * Fix missing optional field access in webapp signature_match handler. (Jon Siwek) - -2.0-41 | 2012-02-03 04:10:53 -0500 - - * Updates to the Software framework to simplify the API. (Bernhard - Amann) - -2.0-40 | 2012-02-03 01:55:27 -0800 - - * Fix typos in documentation. (Daniel Thayer) - - * Fix sorting of lines in Brofiler coverage.log. (Daniel Thayer) - -2.0-38 | 2012-01-31 11:50:53 -0800 - - * Canonify sorting of lines in Brofiler coverage.log. (Daniel - Thayer) - -2.0-36 | 2012-01-27 10:38:14 -0800 - - * New "Brofiler" mode that tracks and records script statements - executed during runtime. (Jon Siwek) - - Use the BROFILER_FILE environment variable to point to a file in - which statement usage statistics from Bro script-layer can be - output. - - Script statements that should be ignored can be marked with a "# - @no-test" comment. For example: - - print "don't cover"; # @no-test - - if ( F ) - { # @no-test - ... - } - - * Integrated coverage measurement into test-suite. (Jon Siwek) - -2.0-20 | 2012-01-25 16:34:51 -0800 - - * BiF cleanup (Matthias Vallentin) - - - Rename NFS3::mode2string to a more generic file_mode(). - - - Unify do_profiling()/make_connection_persistent()/expect_connection() - to return any (i.e., nothing) instead of bools. - - - Perform type checking on count-to-port conversion. Related to #684. - - - Remove redundant connection_record() BiF. The same - functionality is provided by lookup_connection(). - - - Remove redundant active_connection() BiF. The same - functionality is provided by connection_exists(). - - - exit() now takes the exit code as argument. - - - to_port() now received a string instead of a count. - -2.0-9 | 2012-01-25 13:47:13 -0800 - - * Allow local table variables to be initialized with {} list - expressions. (Jon Siwek) - -2.0-7 | 2012-01-25 13:38:09 -0800 - - * Teach CompHash to allow indexing by records with vector/table/set - fields. Addresses #464. (Jon Siwek) - -2.0-5 | 2012-01-25 13:25:19 -0800 - - * Fixed a bug resulting in over-logging of detected webapps. (Seth Hall) - - * Make communication log baseline test more reliable. (Jon Siwek) - - * Fixed some broken links in documentation. (Daniel Thayer) - -2.0 | 2012-01-11 13:52:22 -0800 - - * Adding script reference documentation. (The Team). - -2.0-beta-194 | 2012-01-10 10:44:32 -0800 - - * Added an option for filtering out URLs before they are turned into - HTTP::Incorrect_File_Type notices. (Seth Hall) - - * Fix ref counting bug in BIFs that call internal_type. Addresses - #740. (Jon Siwek) - - * Adding back the stats.bro file. (Seth Hall) - - -2.0-beta-188 | 2012-01-10 09:49:29 -0800 - - * Change SFTP/SCP log rotators to use 4-digit year in filenames - Fixes #745. (Jon Siwek) - - * Adding back the stats.bro file. Addresses #656. (Seth Hall) - -2.0-beta-185 | 2012-01-09 18:00:50 -0800 - - * Tweaks for OpenBSD support. (Jon Siwek) - -2.0-beta-181 | 2012-01-08 20:49:04 -0800 - - * Add SFTP log postprocessor that transfers logs to remote hosts. - Addresses #737. (Jon Siwek) - - * Add FAQ entry about disabling NIC offloading features. (Jon Siwek) - - * Add a file NEWS with release notes. (Robin Sommer) - -2.0-beta-177 | 2012-01-05 15:01:07 -0800 - - * Replace the --snaplen/-l command line option with a - scripting-layer option called "snaplen" (which can also be - redefined on the command line, e.g. `bro -i eth0 snaplen=65535`). - - * Reduce snaplen default from 65535 to old default of 8192. Fixes - #720. (Jon Siwek) - -2.0-beta-174 | 2012-01-04 12:47:10 -0800 - - * SSL improvements. (Seth Hall) - - - Added the ssl_session_ticket_handshake event back. - - - Fixed a few bugs. - - - Removed the SSLv2.cc file since it's not used. - -2.0-beta-169 | 2012-01-04 12:44:39 -0800 - - * Tuning the pretty-printed alarm mails, which now include the - covered time range into the subject. (Robin Sommer) - - * Adding top-level "test" target to Makefile. (Robin Sommer) - - * Adding SWIG as dependency to INSTALL. (Robin Sommer) - -2.0-beta-155 | 2012-01-03 15:42:32 -0800 - - * Remove dead code related to record type inheritance. (Jon Siwek) - -2.0-beta-152 | 2012-01-03 14:51:34 -0800 - - * Notices now record the transport-layer protocol. (Bernhard Amann) - -2.0-beta-150 | 2012-01-03 14:42:45 -0800 - - * CMake 2.6 top-level 'install' target compat. Fixes #729. (Jon Siwek) - - * Minor fixes to test process. Addresses #298. - - * Increase timeout interval of communication-related btests. (Jon Siwek) - -2.0-beta-145 | 2011-12-19 11:37:15 -0800 - - * Empty fields are now logged as "(empty)" by default. (Robin - Sommer) - - * In log headers, only escape information when necessary. (Robin - Sommer) - -2.0-beta-139 | 2011-12-19 07:06:29 -0800 - - * The hostname notice email extension works now, plus a general - mechanism for adding delayed information to notices. (Seth Hall) - - * Fix &default fields in records not being initialized in coerced - assignments. Addresses #722. (Jon Siwek) - - * Make log headers include the type of data stored inside a set or - vector ("vector[string]"). (Bernhard Amann) - -2.0-beta-126 | 2011-12-18 15:18:05 -0800 - - * DNS updates. (Seth Hall) - - - Fixed some bugs with capturing data in the base DNS script. - - - Answers and TTLs are now vectors. - - - A warning that was being generated (dns_reply_seen_after_done) - from transaction ID reuse is fixed. - - * SSL updates. (Seth Hall) - - - Added is_orig fields to the SSL events and adapted script. - - - Added a field named last_alert to the SSL log. - - - The x509_certificate function has an is_orig field now instead - of is_server and its position in the argument list has moved. - - - A bit of reorganization and cleanup in the core analyzer. (Seth - Hall) - -2.0-beta-121 | 2011-12-18 15:10:15 -0800 - - * Enable warnings for malformed Broxygen xref roles. (Jon Siwek) - - * Fix Broxygen confusing scoped IDs at start of line as function - parameter. (Jon Siwek) - - * Allow Broxygen markup "##<" for more general use. (Jon Siwek) - -2.0-beta-116 | 2011-12-16 02:38:27 -0800 - - * Cleanup some misc Broxygen css/js stuff. (Jon Siwek) - - * Add search box to Broxygen docs. Fixes #726. (Jon Siwek) - - * Fixed major bug with cluster synchronization, which was not - working. (Seth Hall) - - * Fix missing action in notice policy for looking up GeoIP data. - (Jon Siwek) - - * Better persistent state configuration warning messages (fixes - #433). (Jon Siwek) - - * Renaming HTTP::SQL_Injection_Attack_Against to - HTTP::SQL_Injection_Victim. (Seth Hall). - - * Fixed DPD signatures for IRC. Fixes #311. (Seth Hall) - - * Removing Off_Port_Protocol_Found notice. (Seth Hall) - - * Teach Broxygen to more generally reference attribute values by name. (Jon Siwek) - - * SSH::Interesting_Hostname_Login cleanup. Fixes #664. (Seth Hall) - - * Fixed bug that was causing the malware hash registry script to - break. (Seth Hall) - - * Remove remnant of libmagic optionality. (Jon Siwek) - -2.0-beta-98 | 2011-12-07 08:12:08 -0800 - - * Adapting test-suite's diff-all so that it expands globs in both - current and baseline directory. Closes #677. (Robin Sommer) - -2.0-beta-97 | 2011-12-06 11:49:29 -0800 - - * Omit loading local-.bro scripts from base cluster framework. - Addresses #663 (Jon Siwek) - -2.0-beta-94 | 2011-12-03 15:57:19 -0800 - - * Adapting attribute serialization when talking to Broccoli. (Robin - Sommer) - -2.0-beta-92 | 2011-12-03 15:56:03 -0800 - - * Changes to Broxygen master script package index. (Jon Siwek) - - - Now only lists packages as those directories in the script hierarchy - that contain an __load__.bro file. - - - Script packages (dirs with a __load__.bro file), can now include - a README (in reST format) that will automatically be appended - under the link to a specific package in the master package - index. - -2.0-beta-88 | 2011-12-02 17:00:58 -0800 - - * Teach LogWriterAscii to use BRO_LOG_SUFFIX environemt variable. - Addresses #704. (Jon Siwek) - - * Fix double-free of DNS_Mgr_Request object. Addresses #661. - - * Add a remote_log_peer event which comes with an event_peer record - parameter. Addresses #493. (Jon Siwek) - - * Remove example redef of SMTP::entity_excerpt_len from local.bro. - Fixes error emitted when loading local.bro in bare mode. (Jon - Siwek) - - * Add missing doc targets to top Makefile; remove old doc/Makefile. - Fixes #705. (Jon Siwek) - - * Turn some globals into constants. Addresses #633. (Seth Hall) - - * Rearrange packet filter and DPD documentation. (Jon Siwek) - -2.0-beta-72 | 2011-11-30 20:16:09 -0800 - - * Fine-tuning the Sphinx layout to better match www. (Jon Siwek and - Robin Sommer) - -2.0-beta-69 | 2011-11-29 16:55:31 -0800 - - * Fixing ASCII logger to escape the unset-field place holder if - written out literally. (Robin Sommer) - -2.0-beta-68 | 2011-11-29 15:23:12 -0800 - - * Lots of documentation polishing. (Jon Siwek) - - * Teach Broxygen the ".. bro:see::" directive. (Jon Siwek) - - * Teach Broxygen :bro:see: role for referencing any identifier in - the Bro domain. (Jon Siwek) - - * Teach Broxygen to generate an index of Bro notices. (Jon Siwek) - - * Fix order of include directories. (Jon Siwek) - - * Catch if logged vectors do not contain only atomic types. - (Bernhard Amann) - -2.0-beta-47 | 2011-11-16 08:24:33 -0800 - - * Catch if logged sets do not contain only atomic types. (Bernhard - Amann) - - * Promote libz and libmagic to required dependencies. (Jon Siwek) - - * Fix parallel make from top-level to work on more platforms. (Jon - Siwek) - - * Add decode_base64_custom(). Addresses #670 (Jon Siwek) - - * A bunch of Sphinx-doc reorgs and polishing. (Jon Siwek) - -2.0-beta-28 | 2011-11-14 20:09:28 -0800 - - * Binary packaging script tweaks. We now require CMake 2.8.6. (Jon Siwek) - - * More default "weird" tuning for the "SYN_with_data" notice. (Seth - Hall) - - * Tiny bugfix for http file extraction along with test. (Seth Hall) - -2.0-beta-21 | 2011-11-06 19:27:22 -0800 - - * Quickstart doc fixes. (Jon Siwek) - -2.0-beta-19 | 2011-11-03 17:41:00 -0700 - - * Fixing packet filter test. (Robin Sommer) - -2.0-beta-12 | 2011-11-03 15:21:08 -0700 - - * No longer write to the PacketFilter::LOG stream if not reading - traffic. (Seth Hall) - -2.0-beta-10 | 2011-11-03 15:17:08 -0700 - - * Notice framework documentation update. (Seth Hall) - - * Fixing compiler warnings (addresses #388) (Jon Siwek) - -2.0-beta | 2011-10-27 17:46:28 -0700 - - * Preliminary fix for SSH login detection: we need a counted measure - of payload bytes (not ack tracking and not with the IP header - which is what we have now). (Seth Hall) - - * Fixing send_id() problem. We no longer update &redef functions. - Updating code on the fly isn't fully supported. (Robin Sommer) - - * Tuning the format of the pretty-printed alarm summaries. (Robin - Sommer) - -1.6-dev-1508 | 2011-10-26 17:24:50 -0700 - - * Updating submodule(s). (Robin Sommer) - -1.6-dev-1507 | 2011-10-26 15:10:18 -0700 - - * Baseline updates. (Robin Sommer) - -1.6-dev-1506 | 2011-10-26 14:48:43 -0700 - - * Updating submodule(s). (Robin Sommer) - -1.6-dev-1505 | 2011-10-26 14:43:58 -0700 - - * A new base script that pretty-prints alarms in the regular - summary. (Robin Sommer) - - * Adding a dummy log writer WRITER_NONE that just discards - everything. (Robin Sommer) - -1.6-dev-1498 | 2011-10-26 14:30:15 -0700 - - * Adding instructions to local.bro how to do ACTION_ALARM by - default. (Seth Hall) - -1.6-dev-1495 | 2011-10-26 10:15:58 -0500 - - * Updated unit test baselines. (Seth Hall) - -1.6-dev-1491 | 2011-10-25 20:22:56 -0700 - - * Updating submodule(s). (Robin Sommer) - -1.6-dev-1482 | 2011-10-25 19:08:32 -0700 - - * Fixing bug in log managers predicate evaluation. (Robin Sommer) - -1.6-dev-1481 | 2011-10-25 18:17:03 -0700 - - * Fix a problem with DNS servers being logged that aren't actually - servers. (Seth Hall) - - * Changed generated root cert DN format for RFC2253 compliance. (Jon - Siwek) - - * Removed :bro doc directives from notice documentation. (Seth Hall) - - * New notice framework docs. (Seth Hall) - - * Adding sub messages to emails. (Seth Hall) - - * Adding extra fields to smtp and http to track transaction depth. - (Seth Hall) - - * Fix for SSH login detection heuristic. (Seth Hall) - - * Removed some fields from http analysis that weren't commonly - needed or were wrong. (Seth Hall) - - * Updated/fixed MSIE version parsing in the software framework. - (Seth Hall) - - * Update Mozilla trust roots to index certs by subject distinguished - name. (Jon Siwek) - - * weird.bro rewrite. (Seth Hall) - - * More notice email tuning. (Seth Hall) - - * Slightly restructured http file hashing to fix a bug. (Seth Hall) - - * Changed the notice name for interesting ssh logins to correctly - reflect semantics of the notice. (Seth Hall) - - * Field name change to notice framwork. $result -> $action - - - $result is renamed to $action to reflect changes to the notice - framework since there is already another result-like field - ($suppress_for) and there may be more in the future. - - - Slipped in a change to add connection information to notice - emails too. (Seth Hall) - - * Small script refinements and documentation updates. (Seth Hall) - - * Pass over upgrade guide. (Robin Sommer) - - -1.6-dev-1430 | 2011-10-21 10:39:09 -0700 - - * Fixing crash with unknown debug streams. Closes #643. (Robin - Sommer) - - * Code to better handle interpreter errors, which can now be turned - into non-fatal runtime errors rather than immediate aborts. (Robin - Sommer). - - * Remove old make-src-packages script. (Jon Siwek) - - * Fixing a bunch of format strings. Closes #567. (Robin Sommer) - - * Cleaning up some distribution files. (Robin Sommer) - - * Various test, doc, and installation fixes/tweaks. (Seth Hall, Jon - Siwek and Robin Sommer). - - * Varios smaller policy fixes and tweaks (Seth Hall). - - * Moving docs from web server into distribution. (Robin Sommer) - - * Fixing more (small) memory leaks. (Robin Sommer) - - * Profiling support for DNS_Mgr and triggers. With - misc/profiling.bro, both now report a line in prof.log with some - counters on usage. (Robin Sommer) - - * Fixing DNS memory leaks. Closes #534. (Robin Sommer) - - * Fix code for disabling analyzers. Closes #577. (Robin Sommer) - - * Changed communication option from listen_encrypted to listen_ssl. - (Seth Hall) - - * Modification to the Communication framework API. (Seth Hall) - - - Simplified the communication API and made it easier to change - to encrypted connections by not having separate variables to - define encrypted and unencrypted ports. - - - Now, to enable listening without configuring nodes just - load the frameworks/communication/listen script. - - - If encrypted listening is desired set the following: - redef Communication::listen_encrypted=T; - - * Connection compressor now disabled by default. Addresses #559. - (Robin Sommer) - - -1.6-dev-1372 | 2011-10-06 18:09:17 -0700 - - * Filtering some potentially high-volume DNS weirds. (Robin Sommer) - - * DNS now raises DPD events. Closes #577. (Robin Sommer) - - * Fixing a bunch of compiler warnings. (Robin Sommer) - - * Remote logs are auto-flushed if the last write was longer than a - second ago. Addresses #498. (Robin Sommer) - - * Fix missing from previous MIME commit. (Robin Sommer) - -1.6-dev-1366 | 2011-10-06 17:05:21 -0700 - - * Make CompHash computation/recovery for functions deterministic. - Closes #636. (Jon Siwek) - - * Removing unnecessary @load in local.bro. (Robin Sommer) - - * Optimizing some MIME code. (Robin Sommer) - - * Speed improvements in logging code. (Robin Sommer) - - * Consolidating some node-specific functionality from scripts in - broctl repo. (Jon Siwek) - - * Another fix the for 1xx script code. (Robin Sommer) - -1.6-dev-1352 | 2011-10-05 16:20:51 -0700 - - * Fix for optional HTTP::Info status_code. (Jon Siwek) - - * Teaking some external testing scripts. (Jon Siwek) - - * HTTP bug fix reported by Martin Holste. (Seth Hall) - - * More script tuning. (Seth Hall) - - - Moved some of the weird events back to the base/ directory. - - - SSL fixes, updates, and performance optimization. - - * More adjustment to reduce Weird volumes. (Seth Hall) - - * Fixed an error when calculating x509 certificate hashes (reported - by Martin Holste). (Seth Hall) - - * Clean up to cluster framework to make event handling clearer. - (Seth Hall) - - * Fixed a bug in the notice framework. (Seth Hall) - - * Bug fix for FTP analysis script. (Seth Hall) - -1.6-dev-1333 | 2011-09-29 22:29:51 -0700 - - * Fixing a number of memory leaks. (Robin Sommer) - - * Loaded_scripts.log is indented with spaces now and makes more - sense to look at. (Seth Hall) - - * Teach HTTP parser to derive content length of multipart/byteranges - bodies. Addresses #488. (Jon Siwek) - - * Change logging of HTTP 1xx responses to occur in their own - columns. Addresses #411. (Jon Siwek) - - * Fix handling of HTTP 1xx response codes. Addresses #411). - - * Taking advantage of yet another trick to get installed browser - plugins. (Seth Hall) - - - With the software-browser-plugins script you can watch for Omniture - advertising servers to grab the list of installed plugins. - - - I reorganized the plugin detection a bit too to abstract it better. - - - Removed the WEB_ prefix from all of the Software::Type HTTP enums. - They were essentially redundant due to the full name already being - HTTP::SERVER (for example). - -1.6-dev-1316 | 2011-09-28 16:50:05 -0700 - - * Unit test cleanup. Updated README and collected coverage-related - tests in a common dir. (Jon Siwek) - - * Fixes for known-services. (Seth Hall) - - * Ported and 2.0ized the capture-loss script. (Seth Hall) - - * Communication fix and extension.(Robin Sommer) - - - Removing unnecessary log flushing. Closes #498. - - - Adding new BiF disconnect() that shuts a connection to a peer down. - - - terminate_connection() now first flushes any still buffered log - messages. - - * Fix for high SSL memory usage by adding &transient attribute to - top-level SSL pac array type. Closes #574. (Robin Sommer) - - * Fix a small bug in the metrics framework. (Seth Hall) - - * Temporarily removing scripts that aren't ready to be included. - Will return before next release. (Seth Hall) - - * New SSL policy scripts. (Seth Hall) - - - protocols/ssl/expiring-certs uses time based information from - certificates to determine if they will expire soon, have already - expired, or haven't yet become valid. - - - protocols/ssl/extract-certs-pem is a script for taking certs off - the line and converting them to PEM certificates with the openssl - command line tool then dumping them to a file. - - * Notice::type_suppression_intervals: table[Notice::Type] of - interval can be used to modify the suppression intervals for - entire types of notices. (Seth Hall) - - * EOF SSL protocol violations are only generated a single time now. - (Seth Hall) - - * Script level fixes. (Seth Hall) - - - Fixed a type name conflict in the Known namespace. - - - Fixed a DPD framework bug that was causing Reporter messages. - - - Fixed the notice_policy log. - - - Predicate functions are now logged. - - - Predicate functions are now optional. If not given, it's assumed that - the result should always apply. (Seth Hall) - - - Fix a problem with accidental and mistaken HTTP log lines. - -1.6-dev-1293 | 2011-09-22 19:44:37 -0700 - - * Smaller script tweaks. (Seth Hall) - - * Duplicate notice suppression. (Seth Hall) - - - Duplicate notices are discovered with the new Notice::Info - field $identifier. It's a string that is left up to the - notice implementor to define which would indicate a - fundamentally duplicate notice. The field is optional and - if it's not included it's not possible for notice - suppression to take place. - - - Duplicate notices are suppressed by default for the interval - defined by the Notice::default_suppression_interval variable - (1 hour by default). - - - A new notice action was defined ACTION_NO_SUPPRESS to prevent - suppression for a specific notice instance. A convenience set - named not_suppressed_types was also created to not suppress - entire notice types. - - - A new field was added to the PolicyItem type to modify the length - of time a notice should be suppressed if the predicate matches. - The field is named $suppress_for. This name makes the code more - readable like this: $suppress_for = 1day - - - New events were created to give visibility into the notice - framework's suppression activity. - - event Notice::begin_suppression(n: Notice::Info) - - event Notice::suppressed(n: Notice::Info) - - event Notice::end_suppression(n: Notice::Info) - - - The suppression.bro script doesn't have a baseline because - it is causing a segfault in Bro. This one test is the - reason that this is being integrated into a branch instead - of master. (Seth Hall) - - * Fix crash on exit. Addresses #607. (Jon Siwek) - - * Fix PktSrc setting next_timestamp even when no packet available. - (Jon Siwek) - - * Fix lack of NUL-termination in to_upper/to_lower BIF's return val. - (Jon Siwek) - - * Fixing unit tests and some minor bugs. (Jon Siwek) - - * Fix broctl cluster log rotation. Addresses #619. (Jon Siwek) - - * Added session ID to the SSL logging. (Seth Hall) - - * Adding "install-aux" target + updating bro-aux submodule. (Jon - Siwek) - - * Cleaning up INSTALL and README. (Jon Siwek) - - * Remove $Id$ tags. (Jon Siwek) - - * Remove policy.old directory. Addresses #511. (Jon Siwek) - - * Small rework with ssl base script to reduce memory usage. (Seth - Hall) - - * Updated the mozilla root certs. (Seth Hall) - -1.6-dev-1261 | 2011-09-15 17:13:55 -0700 - - * Memory leak fixes. Addresses #574 (Jon Siwek) - - * Add configure options for ruby/bindings integration. (Jon Siwek) - - * Fix filter path_func to allow record argument as a subset of - stream's columns. Addresses #600. (Jon Siwek) - - * Log rotation is now controlled directly through Filter records. (Jon Siwek) - - * Fix indexing for record types with optional fields. Addresses #378 - (Jon Siwek) - -1.6-dev-1248 | 2011-09-15 16:01:32 -0700 - - * Removed custom malloc() implementation for FreeBSD. Closes #557. - (Jon Siwek) - - * Testing/external scripts no longer compute MD5 checksums for SMTP - entities. (Robin Sommer) - - * External tests no longer include the full content of mismatching - files in the diagnostics output. (Robin Sommer) - -1.6-dev-1241 | 2011-09-14 22:51:52 -0400 - - * Fixing a major memory utilization issues with SSL analysis. (Seth - Hall) - - * Enhancements to HTTP analysis: (Seth Hall) - - - More options for the header-names.bro script. - - - New script for logging header names and values. Closes #519. - (Seth Hall) - - - HTTP body size measurement added to http.log. - - - The value of the content-length headers has now been removed - in the default output but it could be added back locally at an - installation by a user. - - - Added fields to indicate if some parsing interruption happened - during the body transfer. Closes #581 (Seth Hall) - - * Misc smaller usability and correctness updates: (Seth Hall) - - - Removed an notice definition from the base SSL scripts. - - - Moved a logging stream ID into the export section for known-services - and bumped priority for creating the stream. - - - Adding configuration knobs for the SQL injection attack detection - script and renaming the HTTP::SQL_Injection_Attack notice to - HTTP::SQL_Injection_Attack_Against - - - Bumped priority when creating Known::CERTS_LOG. - - - Fixing a warning from the cluster framework. (Seth Hall) - - * Bugfix for log writer, which didn't escape binary stuff in some - situations. Closes #585. (Robin Sommer) - - * A larget set of changes to the testing/external infrastructure. - The traces for external test-suites are no longer kept inside the - repositories themselves but downloaded separately via curl. This - is because git is pretty bad at dealing with large files. See the - README for more information. (Robin Sommer) - -1.6-dev-1221 | 2011-09-08 08:41:17 -0700 - - * Updates for documentation framework and script docs. (Jon Siwek) - - * The script level PF_RING support isn't working so removing it. - (Seth Hall) - - * Delete SSL certificates from memory after ssl_established event. - (Seth Hall) - - * Small fixes for SSL analysis. (Seth Hall) - -1.6-dev-1212 | 2011-09-07 16:15:28 -0700 - - * Internally, the UID generation can now return values from - different pool for better reproducability in testing mode. - (Gilbert Clark). - - * Added new BiF unique_id_from(pool: string, prefix: string) that - allows the user to specify a randomness pool. (Gilbert Clark) - -1.6-dev-1198 | 2011-09-07 11:03:36 -0700 - - * Extended header for ASCII log that make it easier for scripts to - parse Bro log files. (Gilbert Clark) - - * Potential fix for rotation crashes. Addresses #588. (Robin Sommer) - - * Added PF_RING load balancing support to the scripting layer, - enabled by loading the misc/pf-ring-load-balancing script. (Seth - Hall) - - * Added a BiF setenv() for setting environment variables. (Seth - Hall) - -1.6-dev-1184 | 2011-09-04 09:34:50 -0700 - - * FindPCAP now links against thread library when necessary (e.g. - PF_RING's libpcap). (Jon Siwek) - - * Install binaries with an RPATH. (Jon Siwek) - - * Fix for a case where nested records weren't coerced even though - possible. (Jon Siwek) - - * Changed ASCII writer to delay creation of log after rotation until - next write. - - * Changed default snaplen to 65535 and added a -l/--snaplen command - line option to set it explicitly. Addresses #447. (Jon Siwek) - - * Various updates to logging framework. (Seth Hall) - - * Changed presentation of enum labels to include namespace. (Jon - Siwek) - - * HTTP analyzer is now enabled with any of the HTTP events. (Seth - Hall) - - * Fixed missing format string that caused some segfaults. (Gregor - Maier) - - * ASCII writer nows prints time interval with 6 decimal places. - (Gregor Maier) - - * Added a Reporter::fatal BIF. (Jon Siwek) - - * Fixes for GeoIP support. Addresses #538. (Jon Siwek) - - * Fixed excessive memory usage of SSL analyzer on connections with - gaps. (Gregor Maier) - - * Added a log postprocessing function that can SCP rotated logs to - remote hosts. (Jon Siwek) - - * Added a BiF for getting the current Bro version string. (Jon - Siwek) - - * Misc. doc/script/test cleanup. (Jon Siwek) - - * Fixed bare-mode @load dependency problems. (Jon Siwek) - - * Fixed check_for_unused_event_handlers option. (Jon Siwek) - - * Fixing some more bare-mode @load dependency issues (Jon Siwek) - - * Reorganizing btest/policy directory to match new scripts/ - organization. Addresses #545 (Jon Siwek) - - * bro scripts generated from bifs now install to - $prefix/share/bro/base. Addresses #545 (Jon Siwek) - - * Changeed/fixed some cluster script error reporting. (Jon Siwek) - - * Various script normalization. (Jon Siwek) - - * Add a test that checks each individual script can be loaded in - bare-mode. Adressess #545. (Jon Siwek) - - * Tune when c$conn is set. Addresses #554. (Gregor Maier) - - * Add ConnSize_Analyzer's fields to conn.log. (Gregor Maier) - - * Fixing bug in "interesting hostnames" detection. (Seth Hall) - - * Adding metrics framework intermediate updates. (Seth Hall) - -1.6-dev-1120 | 2011-08-19 19:00:15 -0700 - - * Fix for the CompHash fix. (Robin Sommer) - -1.6-dev-1118 | 2011-08-18 14:11:55 -0700 - - * Fixing key size calculation in composite hash code. (Robin Sommer) - -1.6-dev-1116 | 2011-08-18 10:05:07 -0700 - - * Remove the 'net' type from Bro (addresses #535). - - * Fix H3 assumption of an 8-bit byte/char. (Jon Siwek) - - * Allow reading from interface without additional script arguments. - Explicitly passing in '-' as an additional command line argument - still allows reading a script from stdin. (Jon Siwek) - - * SSH bruteforcing detection now done with metrics framework. (Seth - Hall) - - * Updates for SQL injection attack detection to match the metrics - framework updates. (Seth Hall) - - * Metrics framework now works on cluster setups. (Seth Hall) - - * Reclassifying more DNS manager errors as non-fatal errors. (Robin - Sommer) - - * Fix ConnSize_Analyzer when used in conjunction with connection - compressor. (Gregor Maier) - - * Fix reporter using part of the actual message as a format string. - (Jon Siwek) - -1.6-dev-1095 | 2011-08-13 11:59:07 -0700 - - * A larger number of script documentation updates. Closes #543. (Jon - Siwek) - - * Workaround for FreeBSD CMake port missing debug flags. (Jon Siwek) - - * piped_exec() can now deal with null bytes. (Seth Hall) - - * Fix vector initialization for lists of records with optional - types. Closes #485. (Jon Siwek) - - * Fix redef'ing records with &default empty set fields. Closes #460. - (Jon Siwek) - - * Fix ConnSize_Analyzer when used in conjunction with the connection - compressor. (Gregor Maier) - - * Fix reporter using part of the actual message as a format string. - (Jon Siwek) - - * Fixing reporter's location tracking. Closes #492. (Robin Sommer) - - * Turning DNS errors into warnings. Closes #255. (Robin Sommer) - - * Logging's path_func now receives the log record as argument. - Closes #555. (Robin Sommer) - - * Functions can now be logged; their full body gets recorded. - Closes #506. (Robin Sommer) - - * Bugfix for hostname notice email extension. (Seth Hall) - - * Updates for notice framework. (Seth Hall) - - - New ACTION_ADD_GEODATA to add geodata to notices in an extension - field named remote_location. - - - Loading extend-email/hostnames by default now that it only does - anything when the ACTION_EMAIL action is applied (finally). - - * Updates to local.bro (Seth Hall) - - * Added the profiling script. (Seth Hall) - - * Updates for SSH scripts. (Seth Hall) - - * ConnSize analyzer is turned on by default now. (Seth Hall) - - * Updates for the build system and site local scripts for cluster. - (Seth Hall) - - * HTTP now uses the extract_filename_from_content_disposition function. (Seth Hall) - - * Major SMTP script refactor. Closes #509. (Jon Siwek and Seth Hall) - - * New variable Site::local_nets_table in utils/site for mapping - address to defined local subnet. - - * Metrics framework updates, more to come. (Seth Hall) - - -1.6-dev-1061 | 2011-08-08 18:25:27 -0700 - - * A set of new/changed tests regarding the new policy script - organisation. (Robin Sommer) - -1.6-dev-1058 | 2011-08-08 16:15:18 -0700 - - * Reorganisation of the scripts that Bro loads by default. (Seth - Hall) - - - policy/ renamed to scripts/ - - - By default BROPATH now contains: - - scripts/ - - scripts/policy - - scripts/site - - - The scripts in scripts/base/protocols/ only do logging and state - building. - - - All of scripts/base/ is loaded by by default. This can however - be disabled by switching Bro into "bare mode" using the new - command-line option --bare-mode (or -b). The cripts in - scripts/base/ don't use relative path loading to ease use of - bare mode (to copy and paste that script). - - - The scripts in scripts/base/frameworks/ add functionality - without causing any additional overhead. - - - All "detection" activity happens through scripts in - scripts/policy/. - - - bro.init was renamed to base/init-bare.bro, and base/all.bro was - renamed to init-default.bro. - - - local.bro now loads more functionality from policy/ and adds - more documentation. (Seth Hall) - - * Adding default_path_func() to the logging framework that makes the - default naming scheme script-level controlled. (Robin Sommer) - - * Reworking logging's postprocessor logic so that postprocessor - commands are no longer run by the log writers themselves, but - instead by a script level function. (Robin Sommer) - - * The communication subsystem is now by default off and must be - enabled explicitly with a new BiF, enable_communication(). Closes - #540. (Robin Sommer) - - * The hostname notice email extension now only add hostnames for - emailed noticed. (Seth Hall) - - * Cleaning up doc generation. (Seth Hall) - -1.6-dev-1044 | 2011-08-05 19:07:32 -0700 - - * Fixing memory (and CPU) leak in log writer. - - * Fixing crash in memory profiling. (Robin Sommer) - - * Fix compiler warning. (Robin Sommer) - - * Fixing missing sync in cluster setup. (Robin Sommer) - - -1.6-dev-1038 | 2011-08-05 18:25:44 -0700 - - * Smaller updates to script docs and their generation. (Jon Siwek) - - * When using a `print` statement to write to a file that has raw output - enabled, NUL characters in string are no longer interpreted into "\0", - no newline is appended afterwards, and each argument to `print` is - written to the file without any additional separation. (Jon Siwek) - - * Test portatibility tweaks. (Jon Siwek) - - * Fixing PktSrc::Statistics() which retured bogus information - offline mode. Closes #500. (Jon Siwek) - - * --with-perftools configure option now assumes --enable-perftools. - Closes #527. (Jon Siwek) - -1.6-dev-1018 | 2011-07-31 21:30:31 -0700 - - * Updating CHANGES. (Robin Sommer) - -1.6-dev-1016 | 2011-07-30 18:34:28 -0700 - - * Install example config files dynamically. They'll only get - installed when the distribution version differs from existing - version on disk. (Jon Siwek) - - * Fixed memory leak in SSL analyzer. (Seth Hall) - - * Beginning rework of metrics interface. (Seth Hall) - - * New/updated unit tests for scripts. (Jon Siwek) - - * New/updated documentstion for scripts. (Jon Siwek) - - * A number of fixes for scripts in utils/. (Jon Siwek) - -1.6-dev.244 Thu Jul 28 17:08:21 PDT 2011 - -- mask_addr() now returns subnet (addresses #512). (Jon Siwek) - -- Normalize Notice::Type identifiers per convention (closes #484). - (Jon Siwek) - -- Fixing default-loaded-scripts test for BSD systems. (Jon Siwek) - -- New piped_exec() BiF for pipeing data into an external command. (Jon - Siwek) - -1.6-dev.242 Mon Jul 25 21:42:39 PDT 2011 - -- Adding a documentation coverage test. (Jon Siwek) - -- The CMake targets for generating reST docs from policy scripts are - now automatically generated via the genDocSourcesList.sh script. - (Jon Siwek) - -- Fixed a number of script error. (Jon Siwek) - -- Fixes to relative @load'ing. (Jon Siwek) - -- Fixes to tests. (Robin Sommer) - -1.6-dev.240 Sun Jul 24 15:14:26 PDT 2011 - -- Updated tests and test baselines. (Jon Siwek) - -- ASCII log writer now prints time values w/ constant 6 digit - precision. (Jon Siwek) - -- Many policy script updates acrsso the board (Seth Hall). - -- Moving devel-tools to bro-aux. (Robin Sommer) - -- BugFix for disable_analyzer(), which could cause crashes with some - analyzers. (Robin Sommer) - -- Bugfix for potential segfault in DebugLogger. (Robin Sommer) - -1.6-dev.226 Thu Jul 21 15:23:39 PDT 2011 - -- Extensions to the @load and @unload process. (Jon Siwek) - - * Make @load statements recognize relative paths. For example a - script can do "@load ./foo" to load a script named foo.bro that - lives in the same directory or "@load ../bar" to load a script - named bar.bro in the parent directory, even if those directories - are not contained in BROPATH. - - * Reimplementation of the @prefixes statement. (Closes #486) - - Any added prefixes are now used *after* all input files have - been parsed to look for a prefixed, flattened version of the - input file somewhere in BROPATH and, if found, load it. For - example, if "lcl" is in @prefixes, and site.bro is loaded, then - a file named "lcl.site.bro" that's in BROPATH would end up being - automatically loaded as well. Packages work similarly, e.g. - loading "protocols/http" means a file named - "lcl.protocols.http.bro" in BROPATH gets loaded automatically. - - * Fix @unload'd files from generating bro_script_loaded event. - - * Updates to tests. - -1.6-dev.225 Wed Jul 20 17:10:41 PDT 2011 - -- IRC improvements (Jon Siwek). Including: - - * Shorten what's displayed in the IRC's log mime_type column for - DCC transfers. - - * Add IRC unit tests. - - * Fix IRC analyzer supplying wrong type to irc_dcc_message event. - - * Removed irc_client and irc_server events. - - * Added is_orig arguments to all other irc events. - - * Fix analyzer not recognizing Turbo DCC extension message format. - - * Fix analyzer not generating irc_dcc_message event when irc_privmsg_message - event doesn't have a handler registered. - -- Fixing tests that need a diff canonifier. (Jon Siwek) - -1.6-dev.223 Tue Jul 19 19:10:36 PDT 2011 - -- Adding a script to update CHANGES and VERSION. (Robin Sommer) - -1.6-dev.218 Tue Jul 19 18:16:44 PDT 2011 - -- Comprehensive policy script overhaul/rewrite. (Seth Hall) - - Changes are too extensive to list individually. - -- Removing undocumented -H command line flag. (Robin Sommer) - -- Fixing many tests. (Everybody) - -- Fixing 0-chunk bug in remote logging. (Robin Sommer) - -- $PATH is now appropriately set by the bro-path-dev.(sh|csh) scripts. - (Seth Hall) - -- Making valgrind a bit more happy. (Robin Sommer) - -- New BiF record_field_vals() that returns the fields of a record in a - table with meta-information. (Robin Sommer) - -- Adding a script in aux/devel-tools that extracts a connection from a - trace based on uid. (Robin Sommer) - -- Fixing bug causing crash when running without arguments. (Robin Sommer) - -- A new event bro_script_loaded() raised for each policy script - loaded. Also removing the -l command-line option as that can now be - done at the script-level. (Robin Sommer) - -- Fixing memory leaks. (Gilbert Clark, Seth Hall, Robin Sommer) - -- Many SSL analysis improvements and fixes. (Seth Hall) - -- Fixing bug with event priorities potentially being ignored for the - handler. (Robin Sommer) - -- Overhauling the internal reporting of messages to the user. The new - Reporter class is now in charge of reporting all errors, warnings, - informational messages, weirds, and syslogs; and it passes - everything through the script layer. (Robin Sommer) - -* Removed the alarm statement and the alarm_hook event. (Robin Sommer) - -- Adding new policy file test-all.bro that loads all other policies. - This is for testing only. (Robin Sommer) - -- A new framework for doing regression testing with larger traces and - more complex Bro configurations in testing/external. (Robin Sommer) - -- Many updates to script doc generation. (Jon Siwek) - -1.6-dev.146 Sat Jun 25 18:12:27 PDT 2011 - -- DNS mapping are now becoming invalid when an entry's TTL expires. - (Thomas Other) - -- Reworking how Bro tracks which scripts are already loaded. Rather - than paths, Bro now tracks inode numbers. (Jon Siwek) - -- New BiF netstats() to query packet capture statistics. The netstats - script now uses the new BiF to periocally report packets drops. The - net_stats_update() event and the heartbeat_interval global went - away. (Seth Hall) - -- Fixing bug with logging &optional records. Closes #476. (Robin - Sommer) - -- Fixing istate.events-ssl test failing because of expired cert. (Jon - Siwek) - -- A large number of improvements and fixes for Bro's doc mode. (Jon - Siwek) - -- Significant updates for RPC and NFS analyzers (Gregor Maier) - - * Unify semantics for UDP and TCP connections. - - * RPC can now log to a log file if desired. - - * Portmapper can now log general activity to a log file and also log - actual port mappings. - - * NFS analyzer now supports significantly more procedure calls as - as file name tracking and file content extraction. - -- NetBIOS fixes. (Jon Siwek) - -- A number of unit tests are more robust and portable. (Jon Siwek) - -- A new BiF unique_id() that returns a string that's unique across Bro - instaces with high probablity. (Robin Sommer) - -- Complete rewrite of the BinPAC SSL analyzer. (Seth Hall) - - * DER certificates are extracted as strings to be used with - corresponding BiFs. - - * x509_verify function to verify single certs and/or full - certificate chains. - - * Removed hand written SSL analyzer. - - * The ssl.bro script is just a place-holder for now. New version - will come with the other new scripts. - -- New syslog analyzer. (Seth Hall) - -- @load now supports loading a directory. With a directory "foo" - somewhere in BROPATH, "@load foo" now checks if there's a file - "foo/__load__.bro". If so, it reads that file in. (Robin Sommer) - -- ASCII logger now escapes non-printable characters. Closes #450. - (Robin Sommer) - -- Packaging tweaks and rewrite of 'dist' target. (Jon Siwek) - -- Changes to allow DEB packaging via CPack, addresses #458. (Jon - Siwek) - -- An extension to the ICMP analyzer to handle redirects. Julien - Sentier - -- Removing old istate test-suite. (Robin Sommer) - -- A hack to report missing GeoIP support only once. This closes #357, - but #455 captures the need for a more general solution. (Robin - Sommer) - -- Bugfix: vectors in records were not initalized. Closes #421. (Robin - Sommer) - -- If IPv6 default is not compiled in, the default BPF filters now - excludes IPv6 packets. (Robin Sommer) - -- New bif bro_has_ipv6() to check whether IPv6 support is compiled in. - (Robin Sommer) - -- Updating btests and a Makefile. "make" now runs all the tests. - (Robin Sommer) - -- Moving the test-scripts from the old test-suite over to btest. - (Robin Sommer) - -- Fix for major bug in POP3 analyzer, which didn't recognize '.' - terminators in multi-line replies if the terminator was bare (no - newline). This caused it to ignore the rest of the session that it's - analyzing. (Vern Paxson) - -- Fix compiler warning with gcc-4.4.4 (Gregor Maier) - -- Adding example documentation for a script's use of logging features. - (Jon Siwek) - -- Adding &log attribute to static attr_names array. (Jon Siwek) - -- Bro can now track packet and byte counts per connection. (Gregor - Maier) - - * If 'use_conn_size_analyzer' is true, the event engine tracks - number of packets and raw IP bytes per connection. If - report_conn_size_analyzer is true, these values are included as - four new columns into conn.log - - * I changed conn.bro so that the value of - report_conn_size_analyzer follows that of - use_conn_size_analyzer. For the new conn.log, we probably want - to get rid of report_conn_size_analyzer anyway. - -- Fixing numerous compiler warnings and portability issues. (All) - -- Switching vectors from being 1-based to 0-based. Note that this is a - change that break backwards-compatibility. (Robin Sommer) - -- Increasing serialization format version for the recent 64-bit - changes. (Robin Sommer) - -- Support for (mixed) MPLS and VLAN traffic, and a new default BPF - filter. (Seth Hall and Robin Sommer) - - * Merging in the patch from #264, which provides support for mixed - VLAN and MPLS traffic. - - * Changing Bro's default filter from being built dynamically to - being a static "ip or not ip". To get the old behaviour back - (i.e., the dynamically built filter), redef "all_packets" to - false. - - * print-filter.bro now always prints the filter that Bro is - actually using, even if overriden from the command line. (Robin - Sommer) - -- Changing the HTTP's analyzers internals to use 64-bit integers. - (Gregor Maier). - -- Fixing bug with deleting still unset record fields of table type. - (Robin Sommer) - -1.6-dev.99 Fri Apr 22 22:10:03 PDT 2011 - -- Extending the connection record with a unique identifier. (Robin - Sommer) - - type connection: record { - [...] - id: string; - }; - - These identifiers very likely unique even across independent Bro - runs. - -- Delete operator for record fields. (Robin Sommer) - - "delete x$y" now resets record field "x" back to its original state - if it is either &optional or has a &default. "delete" may not be - used with non-optional/default fields. - -- Fixing bug with nested record coercions. (Robin Sommer) - -- Fixing a do_split() bug. (Seth Hall) - - -1.6-dev.94 Thu Apr 21 19:51:38 PDT 2011 - -- Fixing generation of config.h. (Jon Siwek) - -- Updates and tests for NetBIOS name BiF. (Seth Hall) - -- Fixing do_split bug(), and adding a test. (Seth Hall) - -- When Bro is given a PRNG seed, it now uses its own internal random - number generator that produces consistent results across sytems. - Note that this internal generator isn't very good, so it should only - be used for testing purpses. (Robin Sommer) - -- The BTest configuration now sets the environemnt variables TZ=UTC - and LANG=C to ensure consistent results. (Robin Sommer) - -- Logging fixes. (Robin Sommer) - -1.6-dev.88 Wed Apr 20 20:43:48 PDT 2011 - -- Implementation of Bro's new logging framework. We will document this - separately. (Robin Sommer) - -- Already defined record types can now be further extended via the - '+=' operator. The added fields must be either &optional or have a - &default value. (Robin Sommer) - - Example: - - type Foo: record { - a: count; - b: count &optional; - }; - - redef record Foo += { - c: count &default=42; - d: count &optional; - }; - - global f: Foo = [$a=21]; - - print f; - - Output: - - [a=21, b=, c=42, d=] - -- Enabling assignment of empty vectors ("vector()"). (Robin Sommer) - -- Fixing attributes to allow &default attributes to be associated with - records fields of type tables/sets/vector. (Robin Sommer) - -- '[]' is now a valid record constructor. (Robin Sommer) - -- A instance of a record type A is now coercable into one of type B if - the fields of type A are a subset of those of type B. (Robin Sommer) - -- A number of bug fixes and enhancements for record/set/table/vector - coercion. (Robin Sommer) - -- Fixing a problem with records that have optional fields when used as - table/set indices. Addresses #367. (Robin Sommer) - -- Fixing an off-by-one error in join_string_vec(). (Seth Hall) - -- Updating to_count() to cope with 64bit ints. (Seth Hall) - -- A new BiF count_to_v4_addr() to turn a count into an IPv4 address. - (Seth Hall) - -1.6-dev.80 Mon Apr 18 14:50:54 PDT 2011 - -- New framework for generating documentation from Bro scripts. (Jon - Siwek) - - This includes: - - * Changes to Bro's scanner/parser to facilitate automatic - generation of Bro policy script documentation in - reStructuredText format. - - * New command line flags -Z/--doc-scripts to enable the new doc - generation mode. - - * Changes to bifcl to pass comments starting with "##" through - into the generated .bro script. - - * A "doc" build target for the top-level Makefile to first - generate reStructuredText for a defined set of Bro policy - scripts, and then run that through Sphinx to create HTML - documentation. - -1.6-dev.78 Mon Apr 18 12:52:55 PDT 2011 - -- Adding files to CMake build targets so they show up in generated IDE - projects. This addresses #413. (Jon Siwek) - -- Fix unnecessary config.h preprocessor (re)definitions. This - addresses #414. (Jon Siwek) - -- Updating istate tests. (Robin Sommer) - -- Adding files to CMake build targets so they show up in generated IDE - projects. - -- Adding new environment variable BRO_SEED_FILE to set the seed file - for the random number generator. (Robin Sommer) - -1.6-dev.71 Fri Apr 1 16:06:33 PDT 2011 - -- Removing code for the following no longer supported functionality. - - * Trace rewriting. - * DFA state expiration in regexp engine. - * Active mapping. - * Unused hash functions. - - (Robin Sommer) - -- Fixing crashes when SSL is not configured correctly. (Robin Sommer) - -1.6-dev.66 Tue Mar 29 21:52:01 PDT 2011 - -- Initial btest setup (Don Appleman and Robin Sommer) - -- Porting the istate tests to btest (not finished) (Robin Sommer) - -1.6-dev.63 Mon Mar 21 16:31:15 PDT 2011 - -- Changes to the way user-modifiable config files are installed (Jon Siwek) - - * Duplicates of the distribution's configuration files are now - always installed with a .example suffix - - * Added --binary-package configure option to toggle configure - logic specific to the creation of binary packages. - - * When not in binary packaging mode, `make install` never - overwrites existing configure files in case they've been - modified. The previous behavior (CMake's default) would only - avoid overwriting modified files if one consistently uses the - same build directory and doesn't reconfigure. - -- Fixed an issue with Mac package's pre-install script not preserving - ACLs. (Jon Siwek) - -- Minor cleanup/refactor of the make-mac/rpm-packages scripts. (Jon - Siwek) - -- Add explicit CMake check for compiler. (Jon Siwek) - -- Add alternative way to set BROPATH for running bro from build/ dir. - (Jon Siwek) - -- Fixing compiler warnings (Gregor Maier) - -- Remvoing leftover local variables that caused compile error on Mac - OS X. (Gregor Maier) - -1.6-dev.53 Fri Feb 25 17:03:05 PST 2011 - -- Fixing file detector leak in remote communication module. (Scott - Campbell) - -- Updating independent-state tests to work with new setup. (Robin - Sommer) - -1.6-dev.49 Fri Feb 25 15:37:28 PST 2011 - -- Enum IDs can have explicitly defined values. (Gregor Maier) - -- Extensions for the built-in function compiler, bifcl. (Gregor Maier) - - * Support for policy-layer namespaces. - * Support for type declarations in bif files (with access them - from C++) - * Extended const declarations in bif files. - - See http://bro.icir.org/devel/bif-doc for more information. - -1.6-dev.48 Fri Feb 25 10:53:04 PST 2011 - -- Preliminary TCP Reassembler fix: deliver data after 2GB by disabling - the unused seq_to_skip feature. (Gregor Maier) - -1.6-dev.47 Fri Feb 25 10:40:22 PST 2011 - -- Fixing endianess error in XDR when data is not 4-byte aligned. - (Gregor Maier) - -- Fix for Val constructor with new int64 typedefs. (Gregor Maier) - -- Updated fix for OS X 10.5 compile error wrt llabs(). (Gregor Maier) - -- Fix more compiler warning wrt printf format strings. (Gregor Maier) - -1.6-dev.45 Tue Feb 8 21:28:01 PST 2011 - -- Fixing a number of compiler warnings. (Seth Hall and Robin Sommer) - -1.6-dev.44 Tue Feb 8 20:11:44 PST 2011 - -- A number of updates to the SSL analyzer, including support for new - ciphers; SSL extensions; and bug fixes. The analyzer does not longer - throw weird for exceeding a predefined cipherspec_size anymore. - (Seth Hall and Rmkml). - -- The various split*() BiFs now handle strings containing null bytes - correctly. (Seth Hall) - -- Adding new aux/btest submodule. This is a framework we will use in - the future for doing unit tests. (Robin Sommer) - -1.6-dev.41 Mon Feb 7 13:43:56 PST 2011 - -- Smarter way to increase the parent/child pipe's socket buffer. - (Craig Leres). - -- Fixing bug with defining bro_int_t and bro_uint_t to be 64 bits wide - on some platforms. (Robin Sommer) - -1.6-dev.39 Mon Jan 31 16:42:23 PST 2011 - -- Login's confused messages now go through weird.bro. (Robin Sommer) - -1.6-dev.36 Mon Jan 31 08:45:35 PST 2011 - -- Adding more configure options for finding dependencies, (Jon Siwek) - - --with-flex=PATH path to flex executable - --with-bison=PATH path to bison executable - --with-perl=PATH path to perl executable - --with-python=PATH path to Python interpreter - --with-python-lib=PATH path to libpython - --with-python-inc=PATH path to Python headers - --with-swig=PATH path to SWIG executable - -- Fixing typo in PCAPTests.cmake (Jon Siwek) - - -1.6-dev.33 Mon Jan 24 15:29:04 PST 2011 - -- Fixing bug in SMB analyzer. (Robin Sommer) - -- Configure wrapper now deletes previous CMake cache (Jon Siwek) - -- Fix for the --with-binpac configure option. (Jon Siwek) - -1.6-dev.30 Thu Jan 20 16:32:43 PST 2011 - -- Changed configure wrapper to create config.status. (Jon Siwek) - -1.6-dev.29 Thu Jan 20 16:29:56 PST 2011 - -- Fixing little problem with initialization of Bro-to-Bro event - communication. (Christian Kreibich) - - -1.6-dev.27 Thu Jan 20 13:52:25 PST 2011 - -- Fine-tuning of the HTTP analyzer in terms of raising protocol - violations and interrupted transfers. (Gregor Maier) - - -1.6-dev.21 Wed Jan 19 17:36:02 PST 2011 - -- Added 4 new BiFs and a new record type for testing the entropy of - strings. (Seth Hall) - - find_entropy(data: string): entropy_test_result - This is a one shot function that accepts a string and - returns the result of the entropy calculations. - - entropy_test_init(index: any): bool - This and the next two functions are for calculating entropy - piece-wise. It only needs an index which can be any type of - variable. It needs to be something that uniquely identifies - the data stream that is currently having it's entropy - calculated. - - entropy_test_add(index: any, data: string): bool - This function is used to add data into the entropy - calculation. It takes the index used in the function above - and the data that you are adding and returns true if - everything seemed to work, false otherwise. - - entropy_test_finish(index: any): entropy_test_result - Calling this function indicates that all of the desired data - has been inserted into the entropy_test_add function and the - entropy should be calculated. This function *must* be called - in order to clean up an internal state tracking variable. - If this is never called on an index, it will result in a - memory leak. - - The entropy_test_result values have several measures of the - entropy, but a good one to work with is the "entropy" attribute. - It's a double and as the value approaches 8.0 it can be considered - more and more random. For example, a value of 7.832 would be - quite random but a value of 4.671 is not very random. - -1.6-dev.20 Wed Jan 19 17:30:11 PST 2011 - -- BRO_DNS_FAKE is now listed in the --help output. (Seth Hall) - - -1.6-dev.18 Wed Jan 19 16:37:13 PST 2011 - -- Removing unnecessary expire timer from http_sessions. (Gregor - Maier) - - -1.6-dev.16 Sat Jan 15 14:14:21 PST 2011 - -- Updates to the build system. (Jonathan Siwek) - - * ``make dist`` is now available to be used with the top-level - Makefile for creating source packages according to #344. - - * ``make-rpm-packages`` and ``make-mac-packages`` scripts can - now generate binary packages according to #295. - - * Additional configure options to change packaging behavior. - - * OS X builds will now prefer to link static libraries of - optional dependencies that don't come with the vanilla - operating system. - - * Fix for OS X 10.5 compile error dealing with the llabs() - function from stdlib. - - * Installing as a different user than the one that - configured/built now works (although, a harmless error message - about not being able to write the install manifest may occur). - - -1.6-dev.3 Wed Dec 8 04:09:38 PST 2010 - -- Merge with Subversion repository as of r7137. Incorporated change: - - * Fix for packet processing resumption when a remote Bro dies - during state synchronization (Robin Sommer). - -1.6-dev.2 Wed Dec 8 03:57:03 PST 2010 - -- Compatibility fix for OpenSSL 1.0.0 (Christian Kreibich, Gregor - Maier). - -1.6-dev.1 Sat Nov 27 12:19:47 PST 2010 - -- Merge with Subversion repository as of r7098. Incorporated changes: - - * Rotation post-processors are now passed an additional argument - indicating whether Bro is terminating (Robin Sommer). - - * Bro now consistently generates a file_opened event for all - fopen() calls. (Robin Sommer). - - * You can now redefine the email_notice_to function (Robin - Sommer). - -1.6-dev.0 Fri Nov 26 13:48:11 PST 2010 - -- The Bro source code is now developed in the new git repositories. - See the developer pages at http://www.bro-ids.org for more - information on the new development process. - -- Bro's build and installation setup has been moved from GNU - autotools to CMake. As a result of that, layout and specifics of - the distribution has changed significantly. - -- Lots of pieces have been removed from the distribution that are - either now unnecessary or are no longer maintained. - -- As part of the cleanup, a numbef of Bro configure options and - their corresponding functionality have been removed, including: - - * --disable-select-loop - * --with-dag - * --disable-nbdns - * --enable-activemapping - * --enable-activemapping - * --enable-shippedpcap - -- The previous configure option --enable-int64 is now enabled by default, - and can no longer be disabled. - -- ClamAV support has been removed, which has been non-functional for - a while already. - -1.5.2.7 Sun Sep 12 19:39:49 PDT 2010 - -- Addressed a number of lint nits (Vern Paxson). - - -1.5.2.6 Sun Sep 12 17:00:13 PDT 2010 - -- The SWIG file now explicitly lists those pieces from broccoli.h which it - wants to wrap, rather than just including all of broccoli.h (Robin Sommer). - This fixes the problem that the SWIG bindings depend on what configure - finds out about the availability of libpcap even though the corresponding - functions don't need to be wrapped anyway. - -- http-header.bro now includes a global include_header: set[string] - (Robin Sommer). If it contains any strings, then only those headers - will be processed. If left empty, then you continue to get the current - behavior of processing all headers. - -- Several changes to drop.bro (Robin Sommer): - - * If true, the new flag Drop::dont_drop_locals indicates that - local hosts should never be dropped. On by default. - - * If true, the new flag Drop::debugging activates extensive debugging - output for the catch-and-release logic. Off by default. - - * The timeout for tracking dropping information is now 1 day - rather than 7 days, to better echo the one-restart-a-day semantics - used in the past. - - * Bug fix for hosts once dropped by later cleared; some state - for them persisted. - -- Portability fix for Broccoli Python bindings on 64-bit platforms (Robin - Sommer). - -- The HTTP analyzer no longer attempts to track Server/User-Agent - versions, as these are hugely voluminous (Seth Hall). Ideally this - would still be available as an option for someone who truly wants - the full set. - -- HTTP and SMTP no longer have extra-short inactivity timeouts, as - these were too often leading to premature expiration of a connection - (Robin Sommer). - -- The "rst" tool (aux/rst/) now takes an optional "-I " argument - that instructs it to inject as payload rather than sending a RST - packet (Vern Paxson). must be NUL-terminated, and the NUL is not - included. - -- Bug fix for crashes in the DNS analyzer when processing replies for - which no request was seen (Robin Sommer). - - -1.5.2.5 Mon Jul 19 16:20:58 PDT 2010 - -- Removed now-quite-stale SSHv1 overflow detection, as it's more prone - to false positives than useful detection (Vern Paxson). - - -1.5.2.4 Fri Jun 4 16:02:11 PDT 2010 - -- Bug fixes for terminating connections (Tyler Schoenke and Vern Paxson). - - -1.5.2.3 Wed Mar 24 18:23:57 PDT 2010 - -- Bug fixes for --enable-int64 and for avoiding bogus statistics / - bad memory references when generating profiling information upon - exit (Vern Paxson). - - -1.5.2.2 Tue Jan 12 12:33:42 PST 2010 - -- Broccoli compiler warning fixes (Kevin Lo). - - -1.5.2.1 Sun Jan 10 16:59:01 PST 2010 - -- Bug fix for Active Mapping support (Kevin Lo). - - -1.5.2 Sat Dec 26 18:38:37 PST 2009 - -- Portability fixes for --enable-int64 (Vern Paxson). - - -1.5.1 Fri Dec 18 15:17:12 PST 2009 - -- Due to a Python configuration problem, the original 1.5 distribution - did not include the BroControl component, which also introduced a - portability problem for CentOS. These issues have now been fixed (Robin - Sommer and Vern Paxson). - - -1.5 Wed Dec 16 21:28:47 PST 2009 - -- Bro now comes with a new framework, BroControl, for managing an - operational Bro setup, including support for installation, configuration, - and maintainance tasks such a log archival and mail notification. The - framework transparently supports both traditional standalone setups as - well as cluster installations in which multiple Bro boxes coordinate to - analyze a high-volume network link. - - See aux/broctl/README for more information about BroControl. - - Note, BroControl supersedes the older BroLite system, which is no longer - supported and has been deprecated for a while now. - -- Numerous adjustments to DPD = dynamic protocol detection (Robin Sommer): - - o The Analyzer::ProtocolViolation?() method can now be passed the - offending data (which POP3, SMTP, and FTP now do). This information - is added to the "reason" string passed to the script level. - - o SMTP now more accurately reports violations. - - o FTP stops processing when client & server successfully negotiate - an AUTH scheme (leading to subsequent encryption). - - o Analyzer::ProtocolViolation() is virtual, and - TCP_ApplicationAnalyzer() overrides it to not report violations - for any partial connections, because very likely these arise just - due to the analyzer getting confused. - - o TCP::IsPartial() returns true if any side did not start with - a SYN packet (used to be just be for the originator). - - o The connection_state_remove handler in conn.bro now has a higher - &priority so that other handlers for the same event can use - determine_service() and see any changes it performs. - - o DynDisable:max_volume specifies a volume limit (default 10K). - Once a connection exceeds this limit, further protocol - limitations will neither raise ProtocolViolation notices nor - cause the analyzer to be disabled. - - o The event engine no longer raises protocol_violation events for - TCP connections which had gaps, as these have proven too unreliable. - (Note that, ideally, the *analyzers* should avoid reporting - protocol_violations when they can't reliably parse a connection - anymore after a gap; but many don't.) - -- A set of new script functions provide support for incrementally computing - MD5 checksums (Seth Hall). - - md5_hash_init(index: any): bool - Initializes an incremental hashing instance. "index" is - a value of arbitrary type, used to identify this particular - instance (you can have multiple concurrent instances by - using different index values). Returns T on success, - F on failure (such as the index is already in use). - - md5_hash_update(index: any, data: string): bool - For the given hashing instance, updates the hash - based on the given data. Returns T on success, F on - failure (such as the index has not been initialized). - - md5_hash_finish(index: any): string - Returns the MD5-printable hash for the given index - and terminates the instance, or the string "" if the - index was not active. - -- Bro now supports a believed-to-be-robust mechanism for estimating the - proportion of traffic that it failed to capture ("measurement drops"), - which can arise due to overload in either Bro itself, the kernel's - packet filter, or problems with the link tapping mechanism (Vern Paxson). - The event engine can generate estimates for either live traffic or what - was previously recorded in a trace file, though traces subject to some - forms of selective omission (such as skipping over parts of a connection - to reduce storage) can lead to erroneous values. - - The estimates are based on observing gaps in TCP data streams, and - come in two forms: the rate at which such gaps appear, and the relative - volume of data missing due to the gaps. (We've found however that the - volume-based estimator is not robust due to occasional packets with - incorrect sequence numbers, so this estimator is off by default.) - - The easy way to get the estimates is to load capture-loss.bro. - By default, it generates a CaptureLossSummary notice upon Bro's exit, - which can look like: - - 1130222759.344066 CaptureLossSummary estimated rate = 0.00089124 / 0.000970997 (events/bytes) - - If the estimated loss is none, however, it suppresses this notice, - unless you redef CaptureLoss::summary_if_none to T. - - You can also get finer-grained access by defining a "gap_report" - event handler and redef'ing gap_report_freq to a non-zero interval - (such as "10 sec"). This event allows you to pinpoint regions in - time that exhibit significant capture loss. See capture-loss.bro - for an example of a handler for this event. - - Finally, these changes include a number of fixes to Bro's - ack_above_hole/content_gap analysis, which is now significantly - more robust. - -- GeoIP support now supports ASN lookups via the built-in - function lookup_asn(a: addr): count (Scott Campbell and Seth Hall). - -- The GeoIP built-in's lookup_location() and lookup_asn() now - support IPv6 (Seth Hall). Note, the current GeoIP distribution - doesn't include any IPv6 databases, so for now these won't succeed, - but the hooks are in place for when databases become available. - -- lookup_location() now falls back back to the country database if - the city database isn't available (Seth Hall). - -- The new SuccessfulPasswordGuessing Notice is generated when a host - has been seen attempting password guessing (currently only for FTP - sessions) and then successfully logs in (Royal Chan). You can control the - threshold for such reports in terms of how many attempts the host must - have made by redef'ing the variable password_guessing_success_threshhold, - which defaults to 20. - -- The new script http-detect-passwd.bro analyzes the Web items returned - for fetches that appear to be accessing the passwd file (Akhil Dhar). - It generates a PasswordFullFetch Notice if it appears that the item - includes a full password file, and PasswordShadowFetch if it looks like - a shadowed password file. - -- The new built-in - - system_env(cmd: string, env: table[string] of string) - - works like system(), but puts the table entries into the environment - before invoking the command (Robin Sommer). Each in the table - creates an environment variable of the form "BRO_ARG_", whose - value is the corresponding table entry. - -- The new script function - - execute_with_notice(cmd: string, notice_info) - - executes "cmd" with an environment containing the fields of the - notice_info, i.e., the information associated with a Notice (Robin Sommer). - Per the new system_env() function above, the environment variables appear - as "BRO_ARG_", where is the field tag as it appears in - notice.log when you enable use_tagging. - -- The new built-in enable_raw_output(file) acts the same as - the attribute &raw_output (Seth Hall). - -- The new built-in file_opened(f: file) event is generated any time Bro - opens a script-level file (Justin Azoff). You can use this, for example, - if you want to ensure that a given file has a prelude in it such as - human-readable headers, even when the file is rotated. - -- The notice_info record has a new field - - aux: table[string] of string &optional - - which you can use for information specific to a given type of notice - (Robin Sommer). Entries in $aux appear as "aux_" tags in notice.log. - -- Another new notice_info record field is the boolean do_alarm (default=T), - which, if set to F, overides a notice action otherwise specifying to - generate an alarm (Robin Sommer). In other words, if do_alarm is F, no - alarm will be generated independent of the notice action. - - This is a work-around for the fact that we can't specify more than one - action. In particular, we couldn't NOTICE_DROP but then *not* alarm, - which we now can by returning NOTICE_DROP yet setting do_alarm to F. - -- The notice_info record field $dropped now appears in the tagged output - format if true (Robin Sommer). - -- NOTICEs relating to scan detection now no longer include the connection - that triggered the notice, as it really doesn't contain any useful - information, given that the particular trigger simply depends on the - detection algorithm and its parameters (Robin Sommer). However, we do - explicitly set $p (port number) in the notice, and also $n with the - number of attempts. - -- drop.bro now hardwires a Catch-and-Release redrop after seeing one - connection from a previously-dropped-but-already-released host - (Robin Sommer). - -- drop.bro now provides some new hooks (Robin Sommer): - - event address_dropped(a: addr) - Generated when an address has been dropped. - - event address_restored(a: addr) - Generated when connectivity to an address has been restored, - such as using the Catch-and-Release mechanism. - - event address_cleared(a: addr) - Generated when an address that was dropped in the past is - no longer being monitored looking for new connections - (as part of the Catch-and-Release mechanism). - -- The new built-in function - - hexdump(data_str: string) : string - - returns a hex dump representation of the given input data (Christian - Kreibich). The dump renders 16 bytes per line, with hex on the left and - ASCII (where printable) on the right. - -- Bro's notion of when a TCP connection begins now dastes to the first - instance of an initial SYN packet seen, rather than the last (Gregor Maier). - -- The Time Machine script tm-contents.bro now generates - - event contents_saved: event(c: connection, orig_file: string, - resp_file: string) - - when the content of a connection has been completely saved to disk - (Robin Sommer). - -- The mime.bro script now exports the MIME header callback table, and also - marks it as &redef'able so you can modify its entries (Matthias Vallentin). - The mime_log file is also now exported. - -- A new signature file, policy/sigs/http-bots.sig, contains signatures - to detect some of the current HTTP based controlled bot families (Seth Hall). - -- The signature engine's HTTP pattern matching has been fixed (Seth Hall) - to align with the documentation at: - - http://www.bro-ids.org/wiki/index.php/Reference_Manual:_Signatures#Content_conditions - - In particular, the content condition "http" is now referred to as - "http-request" (though "http" still works for backward compatibility), - "http-request-header" and "http-reply-header" now provide access to - headers seen in only one direction, and similarly for "http-request-body" - and "http-reply-body". (This latter is still accessible as "http-body" - for backwards compatibility.) - -- The new script variable max_remote_events_processed: count (default 10) - sets a limit on the number of remote events processed in each round, - before tending to other inputs (Robin Sommer). - -- If you set the new script variable dump_used_event_handlers to T, - then on startup Bro dumps out all of the event handlers that the - loaded set of scripts can invoke (Matthias Vallenti). - -- Summaries for DNS PTR scanning now use a separate Notice, - DNS_PTR_Scan_Summary, rather than overloading DNS_PTR_Scan (Robin Sommer). - -- scan.bro now provides a table skip_dest_server_ports: set[addr, port] - which lists servers (defined as an address and a port) excluded from - scan detection computations (Craig Leres and Jay Krous). - -- When redefining values on the command line directly (using var=value), - quotation marks are now implicit only if "var" is a variable of type - string (Christian Kreibich). This allows other string-like values - (such as enum's) to be passed as well. - -- scan.bro now explicitly loads conn.bro so that it can itself - be loaded independently (Robin Sommer). - -- login.bro depends on scan.bro (because of tracking authentication - "scans"), so now it explicitly loads it (Vern Paxson). - -- UDP_datagram_length_mismatch is now by default flagged just once per - originating host rather than once per connection, as it can generate - tons of messages (Vern Paxson). - -- Removed now-long-boring flagging of access to Solaris "listen" - service as "hot" (Vern Paxson). - -- Removal of libedit, since libreadline provides similar functionality - (Christian Kreibich). - -- Added scripts missing from distribution: dce.bro, ncp.bro, and smb.bro - (Vern Paxson). - -- ssh.bro now exports ssh_ports (Seth Hall) - -- A number of improvements to inter-Bro communication (Robin Sommer). - - (1) Remote communication now no longer includes location information for - serialized objects; that removes quite a bit of redundacy from the network - traffic. - - (2) The new option 'remote_check_sync_consistency" disables the cross-check - on the receiving side of &synchronized state of whether the current value - of a variable has the value expected by the sender. Transmitting the - original values in addition to the updates generates quite a bit CPU & - network load in some cases (in particular, a table of tables). The default - for remote_check_sync_consistency is off, and so far that in particular - seems to reduce the proxy's load quite a bit. - - (3) Complete overhaul of the internal caching of serialized objects. The - objective of the caching is avoid retransmitting already sent values over - and over again. It turns out, however, that some objects are very stable - and hardly change or get replaced (e.g., Bro types); while other change - all the time and are hardly reused some time later (e.g., Vals). Now - we maintain *two* caches independently for these types of objects; one - with a low turn-over one and another with a high one. This should reduce - CPU load on both sender and receiver sides. - - The new scheme is only used if both communicating Bros support it; with - older Bros, as well as with Broccoli, we continue using the old scheme. - -- Some reworking of remote printing (Robin Sommer), as follows. Bro now - uses a new interprocess message rather than print_hook events, to better - manage buffering and associated load (these can produce failures depending - on system configuration; see remote.log). A number of timeouts and - buffer sizes have been tuned. Internally, EINTR errors are now treated - separately from EAGAIN. Finally, even with remote_check_sync_consistency=F, - one type of consistency check was still being done; this is no longer - the case. - -- The DNS analyzer now generates events (dns_query_reply/dns_rejected) - for replies with zero questions (Robin Sommer). - -- Perftools support for incompatible changes in the 1.0 API (Robin Sommer). - -- Rearranged (generally reducing, though not always) some state timeouts - associated with scan detection (Robin Sommer). In addition, when a - scanning address crosses ignore_scanners_threshold (meaning that it will - be ignored from now on anyway), it gets discarded from all state-tracking - tables. Finally, the ignore_scanners_threshold now applies all kinds - of scans, not just address scans. - -- Substantial Broccoli updates, including a new initialization requirement - that breaks backward compatibility, support for enqueueing serialized - event data for transmission, and OpenSSL threadsafe initialization. - See aux/broccoli/ChangeLog for details (Christian Kreibich, Robin - Sommer, and Matthias Vallentin). - -- Broccoli hashtable optimisation. See aux/broccoli/ChangeLog for - details (Christian Kreibich & Matthias Vallentin). - -- Broccoli memory leak fixed, see aux/broccoli/ChangeLog for details - (Christian Kreibich). - -- Broccoli: updates to bropipe tool (Steve Chan and Robin Sommer). - -- Bug fixes for Broccoli Python bindings (Robin Sommer and Matthias Vallentin). - -- Fixed nasty bug due to module scoping that completely kept stepping-stone - detection from working (Vern Paxson). - -- A serious bug in the packet sorter has been fixed (Robin Sommer). - -- Bug fix for extra NULs getting embedded in escaped strings (Seth Hall). - -- Bug fix for HTTP messages that use "Connection: close" rather than length - headers, which yielded erroneous reassembled messages with \r\n's when - only \n's were present (Bernhard Ager). - -- Fix for reporting on ICMP flows that are expired from the flow table - (Vern Paxson). Previously there was a race condition if the flow - was flushed prior to its summary timer expiring. - -- The -l option (list the scripts that Bro loads) now correctly prints - scripts loaded by the prefix mechanism, and uses indentation to indicate - the load hierarchy (Robin Sommer). - -- A bug has been fixed (really, worked around) in drop.bro that prevented - dropped addresses from being properly restored (Robin Sommer). - -- Fixes for deadlocking problems in the Broccoli protocol. See - aux/broccoli/ChangeLog for details (Christian Kreibich & Robin Sommer). - -- Bug fix for DNS analyzer on 64-bit machines (Gregor Maier). - -- Bug fix for asynchronous DNS lookups to prevent some successful lookups - being reported as timed out (Robin Sommer). - -- Bug fix for tracking line numbers associated with compound statements - (Po-Ching Lin). - -- Fix for a rare condition in which the main Bro process couldn't kill - its child process (Robin Sommer). - -- Fix for file rotation when the underlying file is deleted before the - timer expires (Robin Sommer). - -- Fix for potential crash when communication connections break down, - and also for releasing cached objects (Robin Sommer). - -- Fix for default table entries computed by function invocation to not - cache previous results (Robin Sommer). - -- Fix for Bro's internal DNS resolution (Scott Campbell and Robin Sommer). - -- Portability fix for DAG packet capture (Gregor Maier). - -- Portability fix for --enable-brov6 (Robin Sommer). - -- Portability fixes for FreeBSD (Vern Paxson). - -- A work around for new_packet() crashing on IPv6 packets (Vern Paxson). - For now, IPv6 packets are skipped. Also, for fragments the event handler - is now only called for the fully reassembled packet. - -- The new configuration option --disable-nbdns supports disabling non-blocking - DNS at configure time (Sean McCreary). Note, there are some known problems - with it in some environments. - -- A number of configuration fixes and enhancements (Christian Kreibich - and Robin Sommer). - -- Consistency nit for the configuration process (Seth Hall). - -- A number of reference-counting and other memory management fixes - (Robin Sommer). - -- Bug fix for inter-Bro communication lockup (Seth Hall and Robin Sommer). - -- Bug fix for computing TCP payload length in new_packet event (Lothar Braun). - -- Bug fix for sending boolean True values via Broccoli (Seth Hall). - -- make distcheck fix to clean up .bif.bro files (Christian Kreibich). - -- Bug fix for DPD's recognition of SSLv2 connections (Seth Hall). - -- Bug fix for &default for tables indexed by subnets (Seth Hall). - -- A bug has been fixed that could crash Bro when you called get_event_peer() - after a remote connection had already disppeared (Robin Sommer). - -- Introduced a work-around for crashes that occur when Bro exits - due to handling a signal (Robin Sommer). - -- Bug fix for checkpoint.bro - don't schedule timers for times that - aren't actually in the future (Robin Sommer). - -- Hostname formatting fix for anon.bro (Fabian Schneider). - -- Bug fix for redundant .log extension in Time Machine log file - (reported by CS Lee). - -- Removed now-outdated special-casing of Linux reporting of packet filter - statistics (Peter Wurzinger and Robin Sommer). - -- A number of memory leaks fixed (Robin Sommer). - -- Addressed warnings from newer versions of g++ (Robin Sommer and Vern Paxson). - -- Fixed an invocation issue in the ca-create script that prevented it from - working with recent OpenSSL versions (Craig Leres & Christian Kreibich). - -- Comment fixed in drop-adapt (Justin Azoff). - -- Duplicate code removed from Val (Seth Hall). - - -1.4 Fri Oct 17 11:08:52 PDT 2008 - -- We are no longer supporting a previous Bro release as the "stable" - version. Rather, the model now is that the current public release will - aim for increasing stability (occasionally updated with fixes), and those - who wish to use a "bleeding-edge" snapshot can do so via access to the - public SVN source code repository, as explained at - - http://bro-ids.org/wiki/index.php/Subversion#Public_Access - - Note that all previous releases remain available from the download page; - what is changing is that we no longer commit to support for the most - recent of these. - -- We have clarified the copyright statement that covers most of the - code to remove the "advertising clause" that derived from older - BSD licenses, and we have removed copyright wording from most source - code files. See COPYING for the current wording and a list of - files that retain their own copyright notices. - -- Bro now supports analyzing NetFlow v5 data, i.e., from Cisco routers - (Bernhard Ager). NetFlow can be useful for intrusion detection as it - allows analysis of traffic from many different points in the network. - Bro can now read NetFlow data from a UDP socket, as well as (mostly - for debugging purposes) from a file in a specialized format. You can - create these files with the programs given in aux/nftools. - - Command line switches: - - -Y|--netflow :[=] | read flow from socket - - This is the usual way of getting NetFlow data into Bro by - opening a UDP socket on : and reading all incoming - packets. Setting the to 0.0.0.0 should work on most - platforms. Optionally you may set an identifier for the - source - useful if there are many different sources you want - to analyze in parallel. This might also be necessary if you - want to use this feature with a clustered Bro. - - Examples: - bro -Y 0.0.0.0:5555 netflow - bro -i eth0 -Y 10.0.0.1:1234=src1 brolite netflow - - -y|--flowfile [=] - - Used to read from a file. You can optionally include an - identifier for the source. - - Examples: - bro -y myflowfile netflow - bro -y myflowfile=src1 otherflowfile=src2 netflow - - Netflow Events: - - event netflow_v5_header(h: nf_v5_header) - - Generated upon reading a new NetFlow PDU, as summarized in the - argument. The field h_id gives the flow source identifier and - a serial number. You can use this field to associate subsequent - netflow_v5_record events with their header. - - event netflow_v5_record (r: nf_v5_record) - - Every record within a NFv5 PDU generates a corresponding - netflow_v5_record() event. The relatively complex timestamp - format of NFv5 is already converted to Bro's time type, and - the TCP header flags are separated into bools. - - The distribution includes an example analysis script, netflow.bro. - It simply dumps received NetFlow records. If netflow_restitch is T - (the default), then Bro performs flow restitching as well, and two - script variables become relevant: - - global netflow_finished_conn_expire = 310 sec &redef; - - specifies how long to wait for additional flow records after - a RST or FIN for - - const netflow_table_expire = 31 min; - - Its setting only affects table declarations, and therefore - cannot be usefully redef'd. - - Auxiliary programs: - - Bro uses a custom format for flow data stored in files, - to enable preserving timestamps of the PDU arrivals and the - exporter's IP address. The tools nfcollector and ftwire2bro - in aux/nftools/ provide ways to manipulate the Bro NF file - format. The first dumps NetFlow data from a UDP socket to - stdout or to a file in Bro format. The second converts NetFlow - data in "wire" format to Bro format, and, while doing so, - fakes up the exporter's IP address and timestamp. You can get - "wire" format from normal flow-tools files, e.g., by using - 'flow-export -f 4'. Please note that the Bro format is just - a hack to allow for easier debugging. Therefore the format - is not in fact platform independent, and not suitable for data - storage. - -- A new DHCP analyzer generates the following events (Po-Ching Lin): - - event dhcp_discover(c: connection, msg: dhcp_msg, req_addr: addr) - event dhcp_offer(c: connection, msg: dhcp_msg, mask: addr, - event dhcp_request(c: connection, msg: dhcp_msg, - event dhcp_decline(c: connection, msg: dhcp_msg) - event dhcp_ack(c: connection, msg: dhcp_msg, mask: addr, - event dhcp_nak(c: connection, msg: dhcp_msg) - event dhcp_release(c: connection, msg: dhcp_msg) - event dhcp_inform(c: connection, msg: dhcp_msg) - - where dhcp_msg values look like: - - type dhcp_msg: record { - op: count; # 1 = BOOTREQUEST, 2 = BOOTREPLY - m_type: count; # the type of DHCP message - xid: count; # transaction ID of a DHCP session - h_addr: string; # hardware address of the client - ciaddr: addr; # original IP address of the client - yiaddr: addr; # IP address assigned to the client - }; - - See dhcp.bro for the corresponding analysis script (which could - probably use some refinements). - - Note, this analyzer is implemented using BinPAC, so you will need - to specify --use-binpac to activate it. - -- A BitTorrent analyzer is now available (Nadi Sarrar). See the policy - scripts bittorrent.bro and bt-tracker.bro for the events generated for - analyzing transfers and tracker dialogs, respectively. - -- The "Bro Lite" configuration is now deprecated and will not in - general be supported (Robin Sommer & Vern Paxson). - -- "make install" now only installs a core set of files (Robin Sommer). - Policy files are now installed in /share/bro/* (or whatever - configure determines $datadir to be), which is now in Bro's default - search path. It creates a directory /share/bro/site for local - policy files, and the default BROPATH is extended to include this. The - default path no longer includes policy/local. You can install the - additional files used by the (now deprecated) "Bro Lite" configuration - using "make install-brolite". - -- Substantial updates to Broccoli, including support for container - types (tables and sets) as well as a new metadata structure for event - callbacks, facilitating truly generic event handler implementations - (Christian Kreibich, Seth Hall and Robin Sommer). See aux/broccoli/ChangeLog - for details. - -- Extensive changes to allow Bro to process packets captured in the - past intermingled with those captured in real-time (Matthias Vallentin - and Robin Sommer). This operation reflects combining Bro with use of - "Time Machine" functionality for packet capture. - -- We have unfortunately had to disable support for configuring Bro - to use ClamAV, since it turns out that the key interface we need - for processing blocks of memory directly rather than whole files - is no longer supported by the package, and in fact was buggy even - when it was (Robin Sommer). - -- The new signature option "http-body //" matches - on the body data of HTTP entities (Robin Sommer). The matching is - done after decompressing the body, if necessary. - -- The new built-in function identify_data(data: string, return_mime: bool) - analyzes the string "data" and returns its type according to libmagic, - if installed (Seth Hall). The second argument controls whether it should - be returned as a MIME-type or just an identifying string. For example, - identify_data("MZpofigu", F) returns the string "MS-DOS executable", and - print identify_data("MZpofigu", T) returns "application/x-dosexec". - -- The new analysis script http-identified-files.bro identifies the - type of items returned by Web servers using libMagic (if available) - and generates notices for interesting types and mismatches between - URLs and types (Seth Hall). - - You configure it using two variables. watched_mime_types is a pattern - (default /application\/x-dosexec/ | /application\/x-executable/ ) for - which any MIME type matching the pattern generates a HTTP_WatchedMIMEType - notice. - - mime_types_extensions is a table mapping strings to patterns specifying - how URLs for the given MIME type should appear. (Ideally, this would - be a table mapping patterns to patterns, but Bro doesn't currently support - that.) It defaults to: - - ["application/x-dosexec"] = /\.([eE][xX][eE]|[dD][lL][lL])/ - - i.e., do Windows executables end in .exe or .dll. - - You can also redef the pattern ignored_urls to specify URLs that should - not generate complaints. It defaults to matching Windows Update. - -- The new script http-extract-items.bro extracts the items from HTTP - traffic into individual files (Vern Paxson). Files are named: - - .._._. - - where is a redef'able prefix (default: "http-item"), is a - number uniquely identifying the item, the next four are describe the - connection tuple, and is "orig" if the item was transferred - from the originator to the responder, "resp" otherwise. - -- The workings of how Bro interfaces to external programs for dropping/ - restoring connectivity of misbehaving hosts has been significantly - reworked (Brian Tierney and Robin Sommer). - - First, dropping decisions used to be made directly by analyzer scripts, - such as scan.bro directly calling drop_address(). Now instead the - scripts generate Notices and then the notice policy can have an - action of NOTICE_DROP to codify that the response to the given Notice - is to drop the source. The new notice_action_filter of drop_source - drops the source of notices, and drop_source_and_terminate both - drops the source and terminates the corresponding connection. - - So, to drop all sources triggering a specific notice, one can now, e.g., - write: - - redef notice_action_filters += { [Hot::SSH_Overflow] = drop_source }; - - Related to this change, notice_info has a new field $dropped, set to - true if the Notice triggered a (successful) drop. - - Second, by redef'ing Drop::use_catch_release to T (default F) you can - activate "catch-and-release" logic. You use this mode when you need to - manage a limited number of possible blocks, or to build in automatic - "forgiveness" in situations where blocked sources might become benign - (such as due to dynamic IP addresses). If a source has been idle for - Drop::drop_time, then it is unblocked. However, if it is again seen as - block-worthy, then it is blocked for an interval of Drop::long_drop_time. - - Third, ICMP scanning is now reported by its own notice, ICMPAddressScan, - rather than Scan::AddressScan. - -- Google's perftools have replaced mpatrol for leak-checking and - heap-profiling (Robin Sommer). If Bro is compiled with --enable-perftools - and configure finds the perftools, there are two command-line options - available: - - -m turns on leak checking of the main packet loop, with some - uninteresting leaks are suppressed. Currently, with one - exception (the RPC analyzer; problem not yet found), it reports - no leaks when running the test suite. - - -M turns on heap profiling: Bro will take a snapshot of the heap - before starting the main packet loop and another one when - finished. These snapshots can then be analyzed with pprof. - - For more information about the perftools see - - http://code.google.com/p/google-perftools - -- Notice tags are now generated in a pseudo-unique fashion that, with high - probability, ensures that tags generated by separate Bro processes don't - clash when logged to a common location, such as for a Bro cluster (Robin - Sommer). Tags are now string's rather than count's, and are associated - with all notices, not just that are connection-related. You can however - redef the string notice_tag_prefix or the function new_notice_tag to - further control how such tags are generated. - -- Four new built-ins for type conversion (Robin Sommer): - - function double_to_interval(d: double): interval - function addr_to_count(a: addr): count - function port_to_count(p: port): count - function count_to_port(c: count, t: transport_proto): port - -- Many policy scripts have been modified to use modules & scoping - (Robin Sommer and Matthias Vallentin), which may require updates to - existing scripts/refinements. - -- The new script variable dpd_conn_logs (default F), if true, changes the - semantics of the service field in connection logs written to conn.log, - as follows (Robin Sommer). It becomes a comma-separated list of analyzers - confirmed by DPD to parse the connection's payload. If no analyzer could - confirm its protocol, but the connection uses a well-known port, the - service is the name of the port with "?" appended (e.g., "http?"), as - long as the corresponding analyzer has not declined the connection. - In addition, ftp-data sessions are labeled "ftp-data" and portmapper - connections are labeled with the specific method-call (just as before). - - dpd_conn_logs defaults to F because the change in semantics may break - scripts that parse conn.logs; but it will likely change to the default - in the future. With dpd_conn_logs turned off, conn logs are generated - as they used to be, with a few rare exceptions (with previous versions, - the service field was sometimes determined while the connection was still - alive; now it's always determined at the time when the conn.log entry - is written out). - -- The SSL analyzer has been rewritten using BinPAC, with a number of - robustness improvements (Tobias Kiesling). It currently is only used - if you execute with --use-binpac. - -- Python bindings for Broccoli are now available in - aux/broccoli/bindings/python/ (Robin Sommer). See README/README.html - in that director for details. - -- The new "auth" option in remote.bro indicates whether a given side is - considered "authoritative" for shared state, in which case it sends its - initial state to &sync'ed peers (Robin Sommer). When two peers synchronize - their state, one side sends its current set of state to the other as - soon as the remote connection is established. The one sending the state - used to be the one who has been running longer; now it can also be - explicitly set via the "auth" flag in the Remote::Destination. - -- Two new tuning parameters for scan.bro (Robin Sommer): - - ignore_scanners_threshold (default 0): - - If a host has scanned more than this many hosts, it is completely - excluded from further scan detection. 0 disables. - - addr_scan_trigger (default 0): - - A host is only tracked for address scanning once it has contacted - this many different hosts. Primarily intended for using a two-stage - scan detection with a Bro cluster: first, each node searches locally - for scanners by looking for hosts contacting more than - addr_scan_trigger destinations. Those hosts which do are then - globally tracked throughout the cluster by &synchronizing the scan - detector tables. - -- When Bro serializes functions, it now does so by default using only - their name, rather than their full value (Robin Sommer). This prevents - propagation of expiration functions associated with tables and sets. - Note, currently there is no mechanism provided to switch from the - default behavior, but the internal hooks are in place to do so. - -- The new built-in variable trace_output_file gives the name of the -w - output trace file (Robin Sommer). - -- Bro no longer installs new file rotation timers when shutting down - (Robin Sommer). - -- The new policy scripts remote-print-id{,-reply}.bro support convenient - access to printing the identifiers of a remote Bro (Robin Sommer). - You use the script remote-print-id.bro to request and receive the - printing; the remote Bro must have loaded remote-print-id-reply.bro - in order to process the request. - - Example use: - - bro -e 'redef PrintID::dst="" PrintID::id=""' - remote-print-id - -- scan.bro has been heavily modified to better support distributed scan - analysis (Matthias Vallentin and Robin Sommer). - -- The check for unused event handlers is now turned off by default - (Robin Sommer). To enable, use "redef check_for_unused_event_handlers = T". - -- The new script drop.bro has been split off from scan.bro to isolate - the logic concerning dropping addresses to block scans (Robin Sommer). - -- The new -l flag lists each script as it is loaded (Robin Sommer). - -- Textual descriptions of identifiers now include their attributes - (Robin Sommer). - -- The new predefined function prefixed_id() returns a session identifier with - its peer-ID prepended if it's associated with a remote Bro (Robin Sommer). - This is now used when generating writing log files. - -- remote.bro now assigns a priority of -10 to its bro_init() event handler - to allow others a chance to modify destinations (Robin Sommer). - -- A large number of BinPAC updates (Ruoming Pang and Robin Sommer). - -- The new built-in type_name(v): string returns the name of the type - of the value v (Vern Paxson). For example, "typename(5.2)" returns - "double". This function is mainly for internal debugging (i.e., - finding mismatches between values generated by the event engine - versus how their type is expected by the script layer). - -- The new built-in str_shell_escape() does some basic escaping on strings - that will be passed to system() (Christian Kreibich). Note, this function - isn't ready (robust enough) for routine use, however. - -- The new built-in disable_print_hook(file) acts the same as - the attribute &disable_print_hook (Robin Sommer). - -- The new script terminate-connection.bro factors out the terminate_connection() - functionality that used to be in conn.bro (Robin Sommer). - -- The new attribute &group= can be associated with event handlers - to group them together into a set that can be manipulated as a whole - (Robin Sommer). is a string reflecting the name given to the group. - - The built-in enable_event_group(group: string) turns on all the analyzers - in a given group, and disable_event_group(group: string) deactivates them. - -- The new attribute &raw_output applies to variables of type file, disabling - escaping of non-printable characters (Seth Hall). - -- You can now iterate over the characters in a string value using - a "for" loop, e.g., "for ( c in str ) ..." (Robin Sommer). - -- The new built-in - - function cat_sep%(sep: string, def: string, ...%): string - - works similarly to cat(), except that it (a) separates the values - by "sep" and (b) substitutes "def" for empty strings (Seth Hall). - -- The function string_escape() now takes a string of characters to escape - rather than a single character (Robin Sommer). Each character in the - string is preceded by '\' in the return value (also any embedded '\'s, - as before). - -- The new built-in function global_ids() returns a table of all global - identifiers along with associated information (Robin Sommer). The - return value has type table[string] of script_id, indexed by the name - of the identifier and yielding records with the following fields: - - type script_id: record { - type_name: string; - exported: bool; - constant: bool; - enum_constant: bool; - redefinable: bool; - value: any &optional; - }; - -- The new script function find_last(str: string, re: pattern) returns - the last occurrence of the given pattern in the given string, or - an empty string if no match (Robin Sommer). Note that this function - returns the match that starts at the largest index in the string, which - is not necessarily the longest match. For example, a pattern of /.*/ - will return just the final character in the string. - -- The new script variable record_all_packets, if redef'd to T (default F), - instructs Bro to record every packet it processes (Robin Sommer). - Prior to introducing this variable, Bro applied a few heuristics to - reduce recording volume. Setting this variable also causes packets - to be recorded very early in processing, which can be helpful for - debugging crashes. - -- If the new script flag ssl_log_ciphers is set to T (default), ssl.bro - logs the ciphers seen (Robin Sommer). - -- Much more expanded Time Machine support, now located in - policy/time-machine/ (Robin Sommer), - -- The new command line option --status-file (alias -U) specifies - the name of a file into which Bro will write an indicator of its current - processing status (Robin Sommer). Possible values include "INITIALIZING", - "RUNNING", "TERMINATING", "TERMINATED". - -- The new policy script targeted-scan.bro looks for repeated access from - the same source to the same server, to detect things like SSH - password-guessing attacks (Jim Mellander). - -- The "alternative" style for printing strings (i.e., a fmt() argument - of "%As") now renders the raw string, other than escape-expanding - embedded NULs (Vern Paxson). This change may be temporary, pending - development of more fine-grained control over string rendering. - -- For now we have removed the %S functionality for fmt() (Robin Sommer). - %S was meant to print "raw" strings, but later processing of such - printing still introduces artifacts. - -- GeoIP information now includes latitude and longitude (Seth Hall). - -- ssh.bro now supports the variable skip_processing_after_handshake - which directs the event engine to omit any further processing of an - SSH connection after its initial handshake (Seth Hall and Robin Sommer). - This can help with performance for large file transfers but precludes - some kinds of analyses (e.g., tracking connection size). This change - also adds a scope of "SSH". - -- Email notification of notices now allows for separate destinations - depending on notice type (in particular, a regular mail destination - versus a pager destination), and also escapes the notice to prevent - injection attacks (Seth Hall and Robin Sommer). - -- The new policy script conn-flood.bro is a simple connection-flooding - detector, mainly meant as a demonstration (Robin Sommer). - -- A large number of additions to the TLS/SSL known-ciphers suite (Seth Hall). - -- Serialization now uses 64-bit IDs to cache items rather than 32-bit, - for robustness during long-running execution (Robin Sommer). - -- The new script variable tcp_max_initial_window specifies, for flows - for which ACKs have never been seen, the maximum volume of initial - data after which Bro will assume that it is seeing only one side - of the connection and will not buffer data for consistency checking - awaiting the later arrival of ACKs (Robin Sommer). It defaults to 4 KB. - (Note, this used to be an internal value, so the behavior is not new.) - Set to 0 to turn off this functionality and have Bro attempt to - track all such flows. - -- The new script variable tcp_max_above_hole_without_any_acks specifies, - for flows for which ACKs have never been seen, the maximum volume of - data above a sequence hole that Bro will tolerate for a connection - before giving up on tracking the flow (Robin Sommer). It defaults to 4 KB. - (Note, this differs from tcp_max_initial_window in that this threshold - applies to sequence holes rather than the beginning of flows. Like - tcp_max_initial_window this used to be an internal value.) Set to 0 to - turn off this functionality. - -- The new script variable tcp_excessive_data_without_further_acks specifies - a threshold similar to tcp_max_above_hole_without_any_acks, but for - flows for which Bro has seen ACKs (Robin Sommer). It defaults to 10 MB. - Set to 0 to turn off the functionality. - -- Equal signs ("=") in text for notices are now escaped when using the - tagged format to keep them unambiguous from the "=" delimiters - (Robin Sommer). - -- The final tallies for notices are now processed as NoticeTally - NOTICE's rather than directly alarm'd (Robin Sommer). - -- WeirdActivity notices now include an associated connection when appropriate - (Robin Sommer). - -- Support for large (> 2^32 bytes) pcap trace files (Po-Ching Lin). - -- Scoped names ("...::...") are now allowed in signature "eval" - constructs (Christian Kreibich). - -- scan.bro is now decoupled from conn.bro, i.e., you can @load the - latter without getting the former (Vern Paxson). As part of this - change, the logic to invoke TRW is now in scan.bro. - -- weird.bro has been updated with a number of missing Weird's (Vern Paxson). - -- If when using inter-Bro communication the child Bro process terminates, - it now also terminates the parent process (Robin Sommer). - -- BinPAC analyzers now interoperate with DPD (Robin Sommer). - -- Some http.bro processing options are now exported so they can be - accessed in other scripts (Robin Sommer). - -- SMTP analysis now applies to port 587/tcp as well as 25/tcp (Robin Sommer). - -- $conn is now set in ServerFound notices (Robin Sommer). - -- You can now create empty sets and tables using set() and table(), - i.e., the usual set/table constructors with no arguments (Vern Paxson). - By themselves, these have an unspecified type - you can't use them - directly other than to assign them. For example, - - local bad_guys: set[addr]; - ... - bad_guys = set(); # start over assuming no bad guys - -- A number of scripts have been (slightly) simplified to use the - new empty set()/table() constructors (Vern Paxson). Note that - these still aren't usable for field assignments in record constructors, - nor for attributes like &default = ... - -- Removed unused syntax for declaring sets based on a list of initial - values (Vern Paxson). - -- set() and table() can now be used as arguments to function calls - (Vern Paxson). - -- The vestigial &match attribute has been removed. - -- POP3 is now recognized using Dynamic Protocol Detection (Seth Hall). - -- The new event expected_connection_seen(c: connection, a: AnalyzerTag) - is generated whenever a connection is seen for which we have previously - scheduled an analyzer via expect_connection() (Robin Sommer). - -- The new built-in capture_state_updates logs all changes applied to - &synchronized variables, in a fashion similar to the capture_events() - built-in (Robin Sommer). An accompanying policy script, - capture-state-updates.bro, turns this on to the file state-updates.bst. - -- If the new script variable suppress_local_output is set (default: F), - Bro suppresses printing to local files if there's a receiver for - print_hook events (Robin Sommer). This option is however ignored - for files with a &disable_print_hook attribute. - -- The new notice action filter function file_if_remote specifies - that notices from sent from remote source addresses should - have an action NOTICE_FILE (Robin Sommer). - -- The new notice action filter function file_local_bro_notices specifies - that notices generated by the local Bro instance (as opposed to a - remote peer) should have an action NOTICE_FILE (Robin Sommer). - -- An arbitrary tag can now be past to post-processors for log rotation - (Robin Sommer). - -- Default inactivity timeouts for interactive services shortened to - 1 hour (Robin Sommer). - -- The scanning variables distinct_{peers,ports,low_ports} are now - redef'able (Robin Sommer). - -- The new -S (--summary-only) option for site-report.pl directs to - only generate connection summaries (Brian Tierney) - -- More useful default config file for edit-brorule.pl (Brian Tierney). - -- Bro now includes a test suite in testing/istate/ for its "independent - state" functionality (Robin Sommer). - -- Support for parallel builds via make -j (Christian Kreibich). - -- Bro's default search path now includes includes policy/sigs/ and - policy/time-machine/ (Robin Sommer). - -- Bro's internal processing of interprocess communication has been - significantly overhauled to prevent potentially fatal race conditions - (Robin Sommer). - -- Bro now checks calls to fmt() at compile-time to ensure that the - correct number of arguments are present (Vern Paxson). This is useful - in addition to Bro's run-time checking for arguments matching their - corresponding format-specifiers in the case of rarely-executed statements - that might not generate such run-time checks in routine testing. - -- The ports associated with Telnet and Rlogin are now redef'able (Robin Sommer). - -- MIME processing now removes leading whitespace from MIME headers - (Sanmeet Bhatia and Robin Sommer). - -- TCP "weird" events reported by the connection compressor now match - (other than a few rare corner-cases) those produced for normal TCP - processing (rmkml and Robin Sommer). - -- Added Scan::suppress_UDP_scan_checks to control false positives - on scan detection in environments with P2P protocols that use UDP - (Vern Paxson). - -- The internal analyzer interface now includes an EndOfData() method that - analyzers can use to report that all of a message has been delivered - (Robin Sommer). - -- Fix for a significant memory leak in processing UDP when using -w - (Robin Sommer). Note: this change turns off by default trace rewriting - for generic UDP traffic. - -- Two serious regular expression bugs fixed (Vern Paxson). In the - first, searching for a regular expression inside a string would - fail if the pattern occurred only after an embedded newline. In - the second, insufficient buffer was allocated when compiling regular - expressions, leading to memory corruption. - -- Base64 decoding bug fixes (Christian Kreibich and Ruoming Pang). - -- Automatic rotation of files is now disabled for contents files written - by the TCP reassembler, which otherwise leads to mangled files - (Robin Sommer). - -- Bro now ships with an updated version of libpcap (0.9.8), which hopefully - fixes problems managing trace files > 4 GB in size. - -- Significant bug fixes for gzip- and deflate-encoded Web items (Robin Sommer). - -- Bug fix for secondary-filter.bro (Vern Paxson). - -- Removed a naming ambiguity regarding TCP states (Vern Paxson). - -- Bug fix for signature scanner not matching all of its input (Vern Paxson). - -- Bug fix for using port values in signatures (Robin Sommer). - -- Minor policy script tweaks: state management for weird's, processing - of Notice tags associated with connections, and dependencies for - irc-bot.bro (Robin Sommer). - -- aux/ portability fixes (Vern Paxson). - -- Workarounds added for a BinPAC deficiency, which is that code in %cleanup - clauses can also be executed during recovery from exceptions when parsing - new data. This means that any delete's or Unref()'s need to also set the - corresponding pointer to nil (Vern Paxson). - -- Bug fix for crashes with the non-BinPAC SSL analyzer (Robin Sommer). - -- Tweak to peer-status.bro since Bro now requires events to be - declared prior to reference in a "schedule" statement (Robin Sommer). - -- The signature keyword "enable" now optionally accepts the syntax - "foo:bar" to specify "activate analyzer bar as a child of analyzer foo" - (Robin Sommer). This is used for example for an XML-over-HTTP analyzer - that's in the works. - -- irc-bot-syslog.bro now uses open_log_file() for its log file (including - the logging suffix) rather than a direct open (Vern Paxson). - -- Bug fix for tracking Blaster across a Bro Cluster (Robin Sommer). - -- Bug fix for the HTTP BinPAC analyzer chopping the trailing character - off of HTTP headers when generating the http_all_headers event (Gregor Maier). - -- Bug fix for HTTP chunked items for which the chunk size line was terminated - by CRLF but the CR and LF came in separate packets (Gregor Maier). - -- A bug has been fixed that would cause partial lines (for line-oriented - protocols) to fail to be processed when a connection terminated - (Robin Sommer). - -- Bro no longer treats a signal arriving before a previous signal has - been processed as fatal, nor does it attempt processing of a termination - signal if seemingly there are no race conditions to worry about - (Robin Sommer). Both of these changes are an attempt to improve - Bro's robustness. - -- Fix for attributes such as &encrypt not working in initial declarations - but only in later redef's (Seth Hall and Robin Sommer). - -- Fixes for memory leaks in SSL processing (Seth Hall and Robin Sommer). - -- Fix for POP3 analyzer to not treat lines like "." as message - terminators (Robin Sommer). - -- Bug fix for crashes arising from nil pointers in list expressions - (Seth Hall and Robin Sommer). - -- Bug fix: a signature's "enable" would activate the corresponding analyzer - even if no event handlers were defined for it (Robin Sommer). - -- Bug fixes to prevent crashes when mixing set_contents_file() with - subsequent explicit close(), and to ensure all data written to - file upon connection tear-down (Gert Doering and Robin Sommer). - -- Configuration support for MacPorts and Fink package management systems - (Christian Kreibich & Vern Paxson). - -- Communication-only Bro's now send out email alarms (Robin Sommer). - -- Writes to a file that fail due are now run-time errors rather than - fatal internal errors, since often these occur due to the disk - being full (Robin Sommer). - -- Byte-order bug fix for lookup_location() (Robin Sommer). - -- BinPAC portability fix for 64-bit machines (Bernhard Ager and Robin Sommer). - -- Portability fixes for newer versions of gcc (Jan Gerrit Goebel and - Robin Sommer). - -- Some support for porting to Solaris (Stephan Toggweiler). - -- Connection compressor bug fix for source and destination having the - same IP address, such as when monitoring loopback (Robin Sommer). - -- Connection compressor bug fix for connections with multiple SYNs - (Robin Sommer). - -- Bug fix for using already-declared local variables for looping - over vectors in a "for" loop (Robin Sommer & Vern Paxson). - -- Bug fix for not processing truncated UDP packets (Tom Kho and Robin Sommer). - -- Bounds-check added to BinPAC-generated code (Tom Kho and Robin Sommer). - -- Bug fix for checking whether an IPv6 address is part of a subnet - (Seth Hall). - -- Bug fixes for crashes relating to asynchronous DNS lookups performed - at start-up (Robin Sommer). These changes also lowered the timeout - before assuming failure from 20 seconds down to 5 seconds. - -- Portability and const-ness fixes (Kevin Lo and Robin Sommer). - -- Suppression of some content-gap complaints when running on traces - that have been filtered down to only TCP control packets (Robin Sommer). - -- Removed unnecessary dependency in notice-action-filters.bro - that led to errors when loading icmp.bro by itself (Vern Paxson). - -- Bug fix for potential infinite loop in client communiation (Robin Sommer). - -- Bug fix in reference counting that could eventually lead to roll-over - (Robin Sommer). - -- Bug fix in communication initialization (Robin Sommer). - -- Internal documentation fix: timers are specified using absolute time, - not relative (Robin Sommer). - -- Performance improvement for built-in find_all() function when running - on large strings (Robin Sommer). - -- Memory leak fixes (Robin Sommer, Bernhard Ager, Christian Kreibich). - -- Bug fix for error recovery when encountering an unknown link layer - (Bernhard Ager). - -- Bug fix for reversing client & server in a connection (Po-Ching Lin). - -- Bug fix for packet_contents when capture length exceeds the IP payload - length due to Ethernet frame padding (Christian Kreibich). - -- Bug fix for tcp_packet event erroneously including Ethernet padding - in its contents (Vern Paxson). - -- Bug fix for lookup_connection built-in (Seth Hall). - -- Portability nit for libedit tarball (Vern Paxson). - -- Broccoli portability fix for NetBSD (Christoph Leuzinger). - -- Type-checking for script-level event invocation was completedly broken - - now fixed (Vern Paxson). - -- Portability fixes for different versions of g++/STL (Nicholas Weaver - and Vern Paxson). - -- Fix for dynamic detection of SSL via DPD (Robin Sommer). - -- IPv6 portability fix for BinPAC-based DNS analyzer (Vern Paxson). - Note, more portability work is needed for it. - -- Bug fix for bifcl error messages (Vern Paxson). - -- Minor bug fix for remote communication, plus some improved communication - logging (Robin Sommer). - -- Bug fix for &printhook (Robin Sommer). - -- Bug fix for error message output (Robin Sommer). - -- Bug fix for termination cleanup (Robin Sommer). - -- Bug fix for some Rlogin corner cases (Robin Sommer & Vern Paxson). - -- Bug fix for bifcl generation of "interval" types (Vern Paxson). - -- Bug fix for getting connection memory statistics when Bro is - exiting (Robin Sommer). - -- Config fix: --enable-debug now turns off -O2 for gcc (Robin Sommer). - -- Bug fixes for "heavy" analysis (Vern Paxson). - -- Broccoli bug fixes for types net and port (Robin Sommer). - -- Bug fixes for Telnet environment options (Robin Sommer). - -- Bug fix for accessing remote peer description (Robin Sommer). - -- A fix for the connection compressor generating new_connection too - late (Robin Sommer). - -- Fixes for DAG support, including configuration and multiple - interfaces (Robin Sommer). - -- Bug fix for serializing time-stamps of table entries (Robin Sommer). - -- Bug fix for dealing with peer IDs for remote communication (Robin Sommer). - -- Bug fix to avoid installing timers when timers have already - been canceled (Robin Sommer). - -- Bug fix for interplay between serializing connections and - connection compressor (Robin Sommer). - -- Memory leak fix for enum's (Robin Sommer). - -- Bug fix for files being closed prior to bro_done() (Vern Paxson). - -- aux/broccoli/contrib was not included in distribution (Robin Sommer). - -- Auto-configuration bug fix for BinPAC (Craig Leres). - -- Bug fix for dynamic protocol detection (Robin Sommer). - -- A number of configuration fixes for installation and portability - (Christian Kreibich, Brian Tierney, Robin Sommer, Dan Kopecek). - - -1.3 Mon Jul 16 22:11:00 PDT 2007 - -- The Bro manual has been wikified at: - - http://www.bro-ids.org/wiki/index.php/User_Manual - - and this is the format in which it will evolve in the future - (Christian Kreibich). - -- Much more extensive support for SMB, NetBIOS and NCP (Chris Grier). - -- The new attribute &priority=n defines the order of execution for handlers - of the same event (Robin Sommer). Handlers with higher priority are - executed first. n is an integer expression that must evaluate to a - constant when the script is loaded. - - Example: - > cat foo.bro - event bro_init() &priority = -5 { print -5; } - event bro_init() &priority = 5 { print 5; } - event bro_init() { print 0; } # default priority=0 - > ./bro foo.bro - 5 - 0 - -5 - - The connection_state_remove() handler in conn.bro now has priority - -10 and therefore executes after all other handlers for this event. - This fixes a long-standing problem of sometimes $addl fields not showing - up in connection summaries. - -- The new expressions record(...), table(...), set(...) and vector(...) - are constructors for the corresponding aggregate types (Vern Paxson). - For example, - - record($foo = "hi", $bar = -6) - - is the same as the existing constructor - - [$foo = "hi", $bar = -6] - - For tables, sets, and vectors, the "..." values within the ()'s have - the same syntax as those that you can list in variable initializations. - For example, - - table([1, T] = "black", [4, F] = "red") - - returns a table of type "table[count, bool] of string". - - set(4, 3, -1) - - is a value of type "set[int]". - -- You can associate attributes with table() and set() constructors - (Robin Sommer). For example: - - local s = set(1.2.3.4) &read_expire = 5 secs; - - associates a 5-second read expiration with the set assigned to s. - -- Bro now explicitly supports port numbers reflecting a transport protocol - type of "unknown" (Christian Kreibich). Currently, this means "not TCP, - UDP or ICMP". The numerical value of such a port is the IP protocol, - so ranges from 0..255. For example: - - global p: port = 0/unknown; - - print fmt("%s", p); - print fmt("p is TCP? %s", get_port_transport_proto(p) == tcp); - print fmt("p is unknown? %s", - get_port_transport_proto(p) == unknown_transport); - - yields - - 0/unknown - p is TCP? F - p is unknown? T - - In comparisons of different protocol types, the following holds: - unknown < TCP < UDP < ICMP. - -- If your system supports "GeoIP" (see http://www.maxmind.com/app/geolitecity - for a corresponding city database), then the new script function - - lookup_location(a: addr): geo_location - - returns a record of geographic information associated with an address - (Seth Hall). The geo_location record has $country_code, $region and - $city fields. If no information is available, each of these will be - set to empty strings. - - If Bro hasn't been configured with GeoIP support, or if the address is - IPv6 that cannot be directly converted to IPv4, then Bro produces a - run-time error and likewise returns empty strings. - -- Signature-matching on HTTP components now processes the URI with - escape sequences expanded (Robin Sommer). Ideally, there would be - two signature keywords, one for decoded URIs (corresponding to this - case) and one that allows matching against the URI as originally - transmitted. - -- The connection compressor is no longer considered experimental, and - is used by default (Robin Sommer). - -- The new function lookup_hostname(host: string): addr_set asychronously - looks up the IPv4 address(es) of the given host via DNS (Robin Sommer). - Like lookup_addr(), this function can only be used within a "when" - statement. - -- The new built-in - - raw_bytes_to_v4_addr(s: string): addr - - takes a string that points to at least 4 bytes, and returns an address - corresponding to interpreting these as being an IPv4 address in network - order (Vern Paxson; suggested by Mike Dopheide). - -- Trace-rewriting support for DNS, SMB (Chris Grier). - -- The new script function find_all(str: string, re: pattern): string_set - returns a string_set giving all occurrences of the pattern "re" in - the string "str" (Robin Sommer). (Note that string_set's are unordered.) - -- The new policy script save-peer-status.bro generates a log - to peer_status.$BRO_LOG_SUFFIX of updates received from - communication peers (Robin Sommer). - -- The policy script print-filter.bro now includes two (scoped) variables, - terminate_bro and to_file, which control whether to exit after printing - the filter (default T) and whether to write to the log file - pcap_filter.$BRO_LOG_SUFFIX or (default) to stdout (Robin Sommer). - -- The new script variable check_for_unused_event_handlers controls whether - Bro checks for unused event handlers (Robin Sommer). It defaults to T, - which was the past behavior (always report). - -- Bro now terminates if the only pending activity is future timers - (Robin Sommer). It used to wait for those timers to expire, but this - can cause fundamental problems if the timers are associated with table - management (since these might never completely drain). - -- Tables and sets inside of records are now initialized to empty - values rather than uninitialized (Vern Paxson). - -- A new variable allow_services_from (in hot.bro) complements the - existing allow_service_to variable (Brian Tierney). It specifies - that access to the given service from the given originator is - allowed. - -- global_sizes() no longer reports internal variables (Robin Sommer). - -- The IRC analyzer is now activated if any of the (many) IRC event - handlers are defined (Robin Sommer). - -- The default value for tcp_close_delay is now 5 sec rather than 0 sec - (Robin Sommer). This prevents some spurious connection events. - -- Improved logic for dealing with "reversed" connections such - as backscatter (Vern Paxson). - -- You can now left-justify fields when using fmt() with "%-" like - in sprintf (Christian Kreibich). - -- Updates to DNS query types (Larry Leviton). - -- Added mechanism to http-header.bro to skip printing some HTTP headers - (Larry Leviton). - -- The IrcHotWord notice now sets the associated connection (Robin Sommer). - -- If a notice has a tag, it's no longer overridden (Robin Sommer). - -- ServerFound notices now set the port field (Robin Sommer). - -- The built-in lookup_ID() now returns the string "" if the - ID does not exist, rather than a run-time error (Robin Sommer). - -- The new tuning option ProtocolDetector::suppress_servers specifies a - set of analyzers for which Bro generates ServerFound notices, but not - ProtocolFound (Robin Sommer). This both reduces log file size and - conserves memory. - -- A new notice_action_filter, tally_notice_type_and_ignore, works the same - as tally_notice_type but returns IGNORE (Robin Sommer) - -- Setting summary_interval == 0 disables the creation of irc-bots.summary.log - (Robin Sommer). - -- If you @load foo and a directory "foo" is in your path, Bro no longer - tries to load it (Robin Sommer). - -- A number of BinPAC fixes and enhancements (Ruoming Pang, Chris Grier - and Vern Paxson). - -- BinPAC now resides in aux/binpac rather than src/binpac (Ruoming Pang - and Christian Kreibich). This reflects a decoupling of it from Bro so - that it can be used to generate protocol analyzers for other projects too. - -- Removed example Inktomi entries from skip_scan_sources initialization, - since they no longer exist (Vern Paxson). - -- The variable make notice_once_per_orig_tally_interval is now - redef'able (Brian Tierney). - -- SIGPROF to the communication child process now logs resource stats to - remote.log (Matthias Vallentin). - -- The new built-in getpid(): count returns Bro's process ID (Robin Sommer). - -- Patterns for detecting IRC-based bots updated (Robin Sommer). - -- irc-bot-syslog now logs just bots, not all IRC client/servers (Robin Sommer). - -- The new variable suppress_notice_actions in notice.bro suppresses - notice_actions events for selected notice types (Robin Sommer). - -- Files opened during operation now rotate just like those opened at - startup (Robin Sommer). - -- ResourceStats now also logs elapsed time and the reported number of - packets-on-the-link (Mark Dedlow). - -- Printing a "file" value now produces its name (Robin Sommer). - -- Removed deliberate truncation of payload in port 80 FIN packets - (Vern Paxson). - -- remote.log now includes received peer_descriptions (Robin Sommer). - -- Significant POP3 analyzer speed-ups (Vern Paxson). - -- Updated README (Vern Paxson). - -- Fix for "@load a" followed by "@load a.bro" not loading the same file - twice (Robin Sommer). - -- Bug fixes for propagating state operations to uninitialized variables - and for spurious state inconsistency messags (Robin Sommer). - -- Bug fix for sending final sync-points during pseudo-realtime mode - (Robin Sommer). - -- Fix for possible buffer overflow (Christian Kreibich). - -- Bug fix for spurious end-of-file's during inter-Bro communication - (Robin Sommer). - -- Bug fix for dpd_match_only_beginning=F (Robin Sommer). - -- Bug fix for updating timestamps (Christian Kreibich). - -- Bug fix for skipping ADU processing in adu.bro (Christian Kreibich - and Zhichun Li). - -- Fix for ICMPs that carry ICMP headers (or non-TCP/UDP/ICMP headers) - within them (Vern Paxson). - -- Fix for files being rotated after the timer queue has been deleted - (Vern Paxson). - -- Bug fix for signature-matching with IPv6 subnets (Vern Paxson). - -- Bug fix for connection compressor setting connection origin (Robin Sommer). - -- Bug fix for interconn.bro when processing peculiar connections (Vern Paxson). - -- Fix for off-by-one buffer in sscanf call (Christian Kreibich). - -- Fixed inefficiency/warning flagged by g++ (Vern Paxson). - -- Bug fix for NUL string termination in SMB processing (Zhichun Li). - -- Fix for over-ref'ing of file Val's (Vern Paxson). - -- Fixes for some g++ warnings (Christian Kreibich, Vern Paxson). - -- gcc 3.4.2 portability fixes (Robin Sommer). - -- Minor build fixes for Broccoli, including a version bump to match that - of Bro. See aux/broccoli/ChangeLog for details. - -- distcheck fixes (Christian Kreibich). - -- Configuration portability fixes (Matthias Vallentin, Jean-philippe Luiggi). - -- OpenBSD portability fixes (Jean-philippe Luiggi, Christian Kreibich). - - -1.2.1 Mon Dec 11 16:22:58 PST 2006 - -- Fixed delayed triggering of new_connection events when using the - connection compressor. - -- Fixed tracking of first packet in TCP analyzer. (Reported by Guohan Lu) - -- The syslog built-in got lost during some previous merge. - -- Fixed crash if local variable is given as timeout value for table. - (Reported by Mike Wood.) - -- Fixed using "time" values as table indices. - -- Added ssh to default brolite DPD configuration. - -- Fixed catching up to real-time in case of lull. - -- Fixed Broccoli "BRO_DATA_FORMAT_VERSION" to match version in Bro. - -- Fixed Makefile problem in doc directory. - -- Fixed Makefile dependency problem in binpac directory. - -- Added Linux tuning to brolite install script. - -- Modified Makefile to include broccoli/contrib. - -- Adding missing initialization to remote serializer. - -- Minor documentation updates for reference manual and Broccoli. - - -1.2 Tue Oct 17 12:09:49 PDT 2006 - -- Bro now supports DPD, dynamic protocol detection (Robin Sommer, Holger - Dreger, and Michael Mai). With DPD, Bro can analyze protocols regardless - of what port numbers they use: it infers the protocol based on which - application analyzers can parse it without error. Adding this functionality - involved extensive changes to Bro's internals, but also now enables - multiple Bro analyzers to work on the same connection, either concurrently - or one nested inside the other (we have not taken much advantage of this - latter capability yet, but see the FTP events discussed below). - - There are a number of new policy scripts, events, and variables associated - with DPD processing, as follows. - - Scripts: - - You activate DPD by @load'ing dpd.bro. It in turn instructs Bro - to load the signature file policy/sigs/dpd.sig. Note that Bro - uses signatures to expedite deciding which analyzers to try on - a given connection; it does *not* simply use the signatures to - make the determination of which protocol is in use, as this is - insufficiently robust. (At this point, Bro provides signatures - for FTP, IRC, HTTP, SMTP, and SSH. In the future we plan to add - other protocols.) - - Along with dpd.bro, you need to @load detect-protocols.bro or - detect-protocols-http.bro. The former enables general detection - of application-layer protocols, while the latter does further - inspection of HTTP sessions to characterize applications running - on top of HTTP such as Gnutella or SOAP. (Loading dpd.bro - is separate from loading one of these scripts because in principle - Bro could use a different means than signatures to activate - the analyzers, although currently it does not.) - - If you @load dyn-disable.bro, then once an analyzer determines - that it does not match a given connection, it is deactivated - (and a Notice is generated). Otherwise, it still proceeds to try - its best to analyze the connection (to possibly be more robust - against evasion). - - The scripts dce.bro and smb.bro enable DPD for the Windows DCE and - SMB protocols, respectively. (Note that analysis of these protocols - is undergoing a major expansion, not yet complete.) - - Events: - - event protocol_confirmation(c: connection, atype: count, aid: count) - Generated when the given connection has been confirmed as - conforming with the application type (protocol) specified - by atype. aid is a globally unique analyzer ID that identifies - a particular analyzer instance. - - The values for atype are symbolic names associated with - each of Bro's analyzers, such as ANALYZER_IRC. See the - initialization at the beginning of Analyzer.cc for the - full set of names. - - The function analyzer_name(atype: count): string translates - these symbolic names into text. For example, - - analyzer_name(ANALYZER_IRC) - - yields "IRC". - - event protocol_violation(c: connection, atype: count, aid: count, - reason: string) - Generated when the given connection has been found to - violate the protocol of the given application type, with - "reason" giving details. - - Variables: - - dpd_buffer_size: count (default 1024) - Specifies how much pending data Bro keeps for connections - that have not been classified yet. Once this fills, the - data is deleted, though classification can still continue - (see below). - - dpd_match_only_beginning: bool (default T) - If set, specifies that Bro should stop signature matching - if it has processed dpd_buffer_size bytes. - - dpd_ignore_ports: bool (default F) - If set, then Bro does not take into consideration the port - numbers associated with connections when attempting to - classify them (which can otherwise help the process in - some cases). - - dpd_reassemble_first_packets: bool (default T) - If set, then Bro does TCP stream reassembly before applying - signature-matching to detect protocols. - - likely_server_ports: set[port] - Specifies a list of ports that Bro will consider as likely - used by servers. For example, if Bro sees a connection - that has already been established (so it does not know - which side sent the initial SYN), and one side uses a port - in this set, then it will assume that that side is the - server (connection responder). The set is empty unless - you populate it or @load server-ports.bro, which specifies - a large number of values. - - dpd_config: table[AnalyzerTag] of dpd_protocol_config - Specifies the DPD configuration associated with each tag. - The type dpd_protocol_config is simply: - - type dpd_protocol_config: record { - ports: set[port] &optional; - }; - - i.e., an optional $ports field specifying a set of ports - associatd with the tag. For example, ftp.bro now includes - the equivalent of: - - redef dpd_config += { - [ANALYZER_FTP] = [$ports = 21/tcp] - }; - - Functions: - - The function - - expect_connection(orig: addr, resp: addr, resp_p: port, - analyzer: count, tout: interval) - - is called to alert Bro that a new connection is expected, initiated - by orig to a server running on resp's port resp_p (note: orig's port - is not specified) which will correspond to the specified analyzer - (e.g., "FILE", which is used to analyze files transferred by FTP - - see next item). "tout" is a timeout to associate with the waiting. - - The function - - function disable_analyzer(cid: conn_id, aid: count) - - instructs Bro to disable the analyzer that generated the current - event, assuming the analyzer is associated with the given connection - ID. This is used by the dyn-disable.bro script discussed above. - -- A much more complete BinPAC compiler, along with new HTTP, DNS, and - RPC/Portmap analyzers in binpac (Ruoming Pang). The flag "--use-binpac" - activates the BinPAC-based analyzers (currently for HTTP and DNS). - See www.cs.princeton.edu/~rpang/binpac-paper.pdf for a description of - BinPAC, and let Ruoming know if you are interested in using BinPAC to build - new analyzers. - -- A new type of analyzer, FILE, analyzes the contents of a connection as - though it were a data file (Robin Sommer). Currently, it can generate - two events: - - event file_transferred(c: connection, prefix: string, descr: string, - mime_type: string) - Indicates that the connection transferred a file. "prefix" - is the beginning of the file's data; "descr" and "mime_type" - are indicators of the file's type, as reported by the - "libmagic" library. - - descr/mime_type are only set if Bro is configured on a - system that includes the "libmagic" library. - - event file_virus(c: connection, virname: string) - Indicates the connection transferred an executable - corresponding to a known virus of the given name. - - This functionality is only available if Bro is configured - on a system that includes the "libclamav" library. - - Note, this analyzer is enabled via a call to expect_connection by - the FTP analyzer. - -- New events relating to IRC analysis (Robin Sommer): - - event irc_client(c: connection, prefix: string, data: string) - Generated upon seing a client message sent over the given - IRC connection. "prefix" is the command's prefix as defined - by the IRC protocol. It is used by servers to indicate the - true origin of the message; it may be empty. "data" contains - the message. - - event irc_server(c: connection, prefix: string, data: string) - Same for server messages. - - event irc_user_message(c: connection, user: string, host: string, - server: string, real_name: string) - Generated upon seeing an IRC "USER" command. - - event irc_password_message(c: connection, password: string) - Generated upon seeing an IRC "PASS" command. - - event irc_channel_topic(c: connection, channel: string, topic: string) - Generated upon seeing an IRC server reply that includes - the channel topic. - - event irc_global_users(c: connection, prefix: string, msg: string) - Generated upon seeing an IRC server reply that includes - a count of the number of IRC users. - -- The new experimental script irc-bot.bro tracks IRC-based bots (Robin Sommer). - The accompanying script irc-bot-syslog.bro syslog's the state of the - bot analysis every IrcBot::summary_interval seconds (default 1 minute). - -- The new script proxy.bro looks for open Web proxies by matching incoming - requests to a server with outgoing requests it makes (Robin Sommer). It - generates HTTPProxyFound Notices when it finds one. - -- Changes to notices.bro (Robin Sommer): - - - notice_policy_item's now have a default $result of - NOTICE_FILE and a default $priority of 1. - - - The new notice_action_filter, notice_alarm_per_orig, alarms - on the first NoticeType from a specific source. Subsequent - instances are tallied. - - - notice_action_filters now reside in the new script - notice-action-filter.bro (automatically loaded by notice.bro). - - - The notice actions NOTICE_ALARM_PER_CONN, NOTICE_ALARM_PER_ORIG, - and NOTICE_ALARM_ONCE have been removed, as they were never - actually implemented. - - - If the notice_policy returns IGNORE or FILE, the action_filters - filters are no longer consulted. - -- A new attribute for tables and sets, &mergeable, changes the semantics - of assignments, as follows (Robin Sommer). Given two &mergeable tables/sets - A and B, an assignment "A = B" becomes actually a join "A = A \cup B" - (i.e., union). The envisoned use is to help avoid race conditions - when doing remote state synchronization. - -- The semantics of &synchronized expire_funcs has changed (Robin Sommer). - Now, when a table entry is expired and the operation is propagated to a - a peer, the peer will call its expire_function. - -- TRW analysis now skips UDP traffic because it currently treats - all UDP connections as failures (Robin Sommer). - -- trw.bro has been split into trw-impl.bro (the algorithm) and - trw.bro (which simply activates the analysis), to facilitate writing - scripts that have hooks into TRW analysis but don't presume it's - active (Robin Sommer). - -- The option report_remote_notices in remote.bro has been replaced - by a new script you include, remote-report-notices.bro (Robin Sommer). - -- The new function connect_peer() explicitly connects to a remote host - (Robin Sommer). - -- The new script remote-send-id.bro sends the current value of an ID - to a remote Bro and then terminates processing (Robin Sommer). It's - intended for use from the command-line, as in - - bro -e "redef dst="" id="" remote-send-id - - The other scripts must set up the connection. is an index into - Remote::destinations corresponding to the destination. - -- New built-ins {suspend,resume}_state_updates() can be called to - temporarily avoid propagating updates to &sync'ed values (Robin Sommer). - This can avoid duplicated activity. - -- The new function terminate_communication() instructs Bro to end its - communication with remote peers (Robin Sommer). - -- The new event remote_state_access_performed is raised when remote state - access has been performed (Robin Sommer). This is primarily for debugging. - -- The log() built-in has been renamed to ln() to avoid conflict (Vern Paxson). - -- bifcl now generates event generation wrapper functions from event.bif - (Ruoming Pang). For example, to generate event http_reply, currently - one writes: - - val_list* vl = new val_list; - vl->append(BuildConnVal()); - vl->append(new StringVal(fmt("%.1f", reply_version))); - vl->append(new Val(reply_code, TYPE_COUNT)); - if ( reply_reason_phrase ) - vl->append(reply_reason_phrase); - else - vl->append(new StringVal("")); - ConnectionEvent(http_reply, vl); - - In the future, one will be able to just call bro_event_http_reply(), and - the code generated by bifcl looks like: - - void bro_event_http_reply(Connection* c, StringVal* version, - bro_uint_t code, StringVal* reason) - { - val_list* vl = new val_list; - - vl->append(c->BuildConnVal()); - vl->append(version); - vl->append(new Val(code, TYPE_COUNT)); - vl->append(reason); - - mgr.QueueEvent(http_reply, vl, SOURCE_LOCAL, c); - } - - Accompanying this change is a semantic shift to types "string" and "port" - in .bif files. They used to be translated to C++ types BroString* and - uint32, respectively. Now they are translated to StringVal* and PortVal*. - The functions in bro.bif are changed accordingly, and please be aware - of this change when you write built-in functions in future. - - Also for this change, the parameter 'new' for rsh_request has been renamed - 'new_session', as 'new' is a reserved word for C++. - -- Some ICMP "connections" now have services identified ("icmp-echo", - "icmp-unreach") rather than just listing the service as "other" - (Ruoming Pang). - -- The new option remote_trace_sync_interval specifies an interval after - which each Bro will stop processing its trace and wait for all others - to signal that they have reached the same time (Robin Sommer). The - intent is support for operating Bro in a distributed cluster fashion - (and in particular for debugging such clusters when running off-line - on traces). - - This option only works in pseudo-realtime mode, and requires the new - global remote_trace_sync_peers to give the total number of remote peers - (not including self). Signaling is done via a new communication message - type. - -- Extensions for DNS transformation/anonymization, including introduction - of trace transformation for protocols other than TCP (Jason Lee). - Not yet fully developed/debugged. - -- Extensions for HTTP transformation/anonymization (Martin Casado). - Not yet fully developed/debugged. - -- The $conn field is now included in HTTPProxyFound notices (Robin Sommer). - -- Changed service inference algorithm to favor lower-numbered - likely-servers over higher-numbered ones (Vern Paxson). - -- In pseudo-realtime mode, Bro now uses real-time for deciding which - peer should send state (Robin Sommer). - -- Time synchronization for Bro's running on traces in pseudo-realtime mode - added (Robin Sommer). - -- Avoidance of false content gaps improved when sorting packets with - out-of-order timestamps (Ruoming Pang). - -- Packets from the packet sorter are now more robustly drained upon - termination of input (Ruoming Pang). - -- Documentation for deep-copy updated (Christian Kreibich). - -- Nasty fragment reassembly bug fixed (Vern Paxson). - -- Serious bugs in EDNS0 processing fixed (Vern Paxson). - -- Fixed significant misfeature of interconn.bro that stopped all processing - of a connection once it makes a detection (Vern Paxson). - -- Fixes for &read_expire operation across synchronizes tables (Robin Sommer). - -- Fixes for multiple peers exchanging initial &sync state simultaneously - (Robin Sommer). - -- Improvements to graceful termination of Bro when communicating with - remote peers (Robin Sommer). - -- Fix for ICMP analyzer not always generating icmp_sent events - (Robin Sommer). This appears to still need some work, as now - it generates redundant events. - -- Fix for initial exchange of &sync state which could lead to - referencing unknown IDs (Robin Sommer). - -- Fix to scan detection for differing semantics of connection compressor - vs. non-compressor (Robin Sommer). - -- Bug fix for distinguishing regular expression matches of length 0 from - those of length 1 (Ruoming Pang). - -- Fix for SSH version parsing in the presence of content gaps (Robin Sommer). - -- Bug fix for IRC that could lead to crashes (Robin Sommer). - -- Bug fix to refrain from adding new timers when a connection has - already been removed from the connection table (Robin Sommer). - -- Bug fix for packet_contents not including the transport-layer header - (Robin Sommer). - -- Some memory leaks fixed (Robin Sommer). - -- A bunch of portability and distribution problems fixed (Christian - Kreibich, Robin Sommer, Vern Paxson). - - -1.1 Mon May 15 10:50:33 PDT 2006 - -- Bro now supports a "when" statement for taking action upon something - becoming true asynchronously (Robin Sommer). This provides a powerful - new mechanism with numerous applications. - - Syntax: - - when '(' ')' [timeout '{ '}'] - - where the first can be a single statement or a block enclosed - in {}'s, but the set associated with "timeout" must be enclosed in - {}'s (to reduce ambiguities in Bro's grammar). - - Bro executes the first statement when becomes true. If you give - a timeout and the condition has not been satisfied before it expires, Bro - executes the second statement instead. - - A simple example: - - global t: table[addr] of count; - event connection_established(c: connection) - { - local orig = c$id$orig_h; - if ( orig !in t ) - { - t[orig] = 1; - - when ( t[orig] == 5 ) - print fmt("%s has established 5 connections", orig); - timeout 1 hr - { - print fmt("%s has NOT established 5 connections", orig); - delete t[orig]; - } - } - else - ++t[orig]; - } - - Notes: - - The condition may be evaluated more than once, and at arbitrary - times. - - - When the when-body is executed, the condition is guaranteed to be - still satisfied. - - - Expression reevaluation is primarily triggered by modifications - to globals. However, reevaluations do not take place immediately - but potentially at a later point. This means that if we change a - global to a value which would execute the trigger but then change - it back, the change may go unnoticed. - - - Inside the condition you may introduce new locals. For example, - - when ( (local x = foo()) && x == 42 ) ... - - Such an assignment always yields true as its expression value - (but the assignment might be delayed, for example if foo() is - a delayed function call - see below). - - Delaying function calls - ======================= - - Functions called inside the condition of a when-clause may delay their - results until they're ready. This works for both script-level and built-in - functions. - - For script-level functions, there is a new construct, "return ", - to delay a function's result. When used, the function returns at the - time the when-stmt's condition becomes true, and it yields the value - that the when-stmt's body then returns. Toy example: - - global X: table[string] of count; - - function a() : count - { - # This delays until condition becomes true. - return when ( "a" in X ) - { - return X["a"]; - } - timeout 5 min - { - return 0; - } - } - - event bro_init() - { - # Installs a trigger which fires if a() returns 42. - when ( a() == 42 ) { print "Yippie!"; } - - X["a"] = 42; - } - - There's also a new built-in function which can delay - - lookup_addr(host: addr) - - performs asynchronous DNS address->hostname lookups. Example: - - local h; addr; - [...] - when (local name = lookup_addr(h)) { print h, name; } - - See the function gen_hot_notice_with_hostnames() in conn.bro for - a more worked-out example of using the "when" clause to translate the - local address in SensitiveConnection notices to a hostname (contributed - by Brian Tierney). This functionality is activated by redef'ing - xlate_hot_local_addr to T. - - Here is the full evaluation model of a when's condition: - - - The condition may be evaluated more than once, at arbitrary times. - - - It is always fully evaluated, no matter whether some former - evaluation has been suspended by a delaying function call. - - - All function calls which do not delay are always *fully* executed - each time the condition is evaluated. - - - Function calls which delay are only executed *once*; their result is - cached and re-used in the case the condition is evaluated again. - - - The condition is guaranteed to be true when the body is executed - (potentially using cached function results) - -- By default Bro now uses a configuration similar to what used to be - activated using reduce-memory.bro, along with some additional state - timeouts that are new (Robin Sommer and Vern Paxson). This allows for - better state management out-of-the-box, at the cost of some precision - of analysis and resilience to evasion. In particular, the intent is to - move towards being able to run Bro continuously without inexorably growing - the amount of memory used until exhaustion. - - You can access a configuration similar to the previous default state - management settings by loading heavy-analysis.bro. It turns on a - load-prefix of "heavy", so when you load XXX.bro, a file heavy.XXX.bro - will also be automatically loaded if present. Note that, as was the - case for reduce-memory, you need to load heavy-analysis prior to other - files for it to have effect. - -- The new module clear-passwords.bro monitors login/FTP/IRC/POP traffic - for cleartext passwords (Jason Lee). - -- The new script service-probe.bro looks for remote hosts that repeatedly - connect to the same service on local hosts (for a configurable set of - services and connection sizes) in order to detect brute-forcing attacks - such as password-guessing (Jim Mellander). - -- A new ARP analyzer generates three events: - - event arp_request(mac_src: string, mac_dst: string, - SPA: addr, SHA: string, TPA: addr, THA: string); - - event arp_reply(mac_src: string, mac_dst: string, - SPA: addr, SHA: string, TPA: addr, THA: string); - - event bad_arp(SPA: addr, SHA: string, TPA: addr, THA: string, - explanation: string); - - with a corresponding policy script arp.bro (Chema Gonzalez and Vern Paxson). - It writes logs to arp.$BRO_LOG_SUFFIX. It has not been tested much yet. - -- Bro Lite changes (Jason Lee): - - default user for is now user 'bro' - - now uses the correct sysctl on FreeBSD 6 - - now uses the correct Perl path if site-report.pl not installed - into '/usr/local/bro' - - no longer prompts to encrypt email unless you pick to email reports - -- The default Bro Lite install now only checkpoints Bro once a week - (Brian Tierney). - -- Implicit Bro file extensions (such as .bro for policy scripts and .sig - for signatures) are now searched for first rather than only if the - non-extension-version of the file doesn't exist (Vern Paxson). For - example, running "bro -r trace mt" now first searches $BROPATH for - "mt.bro" before searching for "mt", whereas it used to do these in - the other order. - -- There's now a simpler mechanism for redef'ing variables on the command-line - (Christian Kreibich). Any command line arguments of the form = - are now expanded into policy code of the form "redef var=val;", where - is wrapped in quotation marks if the value appears to be a string - and doesn't have quotation marks already. This works with strings with - whitespace such as foo="Hello World"; however, note that it means you - can't use the mechanism to redef an enum value. - -- The Bro distribution now includes (and builds by default) Christian - Kreibich's Broccoli library (Bro C Client Library), which enables programs - to communicate with running Bro's (Christian Kreibich and Jason Lee). - Configure with --disable-broccoli to turn this off. - -- Built-in functions log(x: double): double and exp(x: double): double - which do natural logarithms and their inverses (Jaeyeon Jung). - -- The new built-in function gethostname() returns the local host's name - (Jason Lee & Robin Sommer). - -- The new built-in function reading_traces() returns true if Bro - is reading trace files (Robin Sommer). - -- The new built-ins suspend_processing() and continue_processing() provide - script-level control for instructing the event engine to stop or resume - processing packets (Robin Sommer). This is useful for coordinating - simultaneous processing by multiple Bro's. - -- Email notices are now by default sent via /bin/mail, with "[Bro Alarm]" - in the subject. - -- redef'ing a function now replaces the existing body rather than - supplementing it (Robin Sommer), which was a bug. - -- You can now configure Bro to process encapsulated IP packets either - by setting, as before, a fixed encap_hdr_size (for VLANs), or setting - parse_udp_tunnels to T (Ruoming Pang). For the latter, you specify a - UDP tunnel port using udp_tunnel_port (the previous variable "tunnel_port" - has gone away); or you can leave it set to its default of 0/udp, in which - case Bro will look for IP encapsulated in UDP packets on any port. - -- Added a simple form of profiling based on sampling the work done - per-packet (Vern Paxson). The event engine generates a - - event load_sample(samples: load_sample_info, CPU: interval, dmem: int) - - event every load_sample_freq packets (roughly; it's randomized), where - load_sample_freq defaults to 20. "samples" is simply a set[string]; it - contains the names of the functions, event handlers, and their source - files that were accessed during the processing of the sampled packet, - along with an estimate of the CPU cost of processing the packet and - (currently broken) memory allocated/freed. - -- Bro now includes experimental support for Endace DAG cards (Gregor Maier - and Robin Sommer). To activate, configure with - - --with-DAG=/path/to/dagtool/installation - - and use "dag0" as the network interface. You may need to configure the - card with the dagtools first. In general, if dagsnap works, Bro should - work as well. - -- Log rotation has changed in a number of ways (Mark Dedlow & Robin Sommer): - - * The new variable log_rotate_base_time: string, if defined, - specifies that logs should be rotated at log_rotate_base_time + - i * rotate_interval intervals. Format is as a string in - 24-hour time, "%H:%M", e.g, "12:00". This format may change - in the future to instead be a Bro time type. - - * RotateLogs::date_format can be redefined to change format of - timestamps in rotated files. - - * RotateLogs::build_name() can be redefined to implement an - arbitrary naming scheme for rotated files. - - Note, this code has not been extensively tested. - -- Bro now by default builds a version of malloc bundled with its - distribution (Vern Paxson & Brian Tierney). - -- The syntax for the clone operator now looks like a function call, - "copy(x)" (Vern Paxson). - -- The new flag DNS::logging (default F), if T, disables generation of - dns.log (which is often uninteresting and very large), though it - still performs analysis leading to NOTICEs (Robin Sommer). - -- A new global, hostile_domain_list, has been added to dns.bro which - lists domains to be flagged if A or MX records are queried (Scott Campbell). - -- Added globals dns_skip_all_{auth,addl} to skip all DNS AUTH/ADDL processing - (Vern Paxson). Skipping these is on (true) by default, because such - processing is quite expensive. - -- backdoor.bro now turns off by default some detectors that from experience - have too many false positives, or (such as for HTTP) too many uninteresting - true positives (Brian Tierney). In addition: - - - the module now generates a BackdoorFound notice for each backdoor - - - the new variable dump_backdoor_packets (default F) if set causes - the packet that triggered the backdoor detection to be written to - backdoor-packets/: