This caused test baseline changes in one of the test: notice now ties in
netcontrol due to ACTION_DROP. Catch and release uses the new_connection
event, which was not before triggered and can cause uids to be generated
for connections that are not usually assigned uids in bare mode.
When inserting, existance of the given subnet is checked using exact
matching instead of longest prefix matching. Before, inserting a subnet
would have updated the subnet item, which is the longest prefix of the
inserted subnet, if present.
Changes:
* change virtual to override where appropriate
* analyzer triggered assert in debug mode because it did not call Done()
on manually instantiated analyzers.
* added a few length checks to methods
* commented unused code and removed a few unused class members
One tweak: I made ts optional and set it to network_time() if not given.
BIT-1578 #merged
* origin/topic/johanna/bit-1578:
Weird: fix potential small issue when ignoring duplicates
Rewrite weird logging.
BIT-1594 #merged
* origin/topic/johanna/rawleak:
Exec: fix reader cleanup when using read_files
Raw Writer: First step - make code more c++11-y, remove raw pointers.
- SMTP protocol headers now do some minimal parsing to clean up
email addresses.
- New function named split_mime_email_addresses to take MIME headers
and get addresses split apart but including the display name.
- Update tests.
Wen using read_files, the Exec framework called Input::remove on the
wrong input stream: it always got called on the input stream of the
execution, not on the input stream of the current file that was being
read.
This lead to threads never being closed and file handles being kept open
until Bro is closed. This means that before this patch, every time
ActiveHTTP is used, a thread stays around and several file handles are
used.
In all versions so far, the identifier string that was used for
comparisons might have been different from the identifier string that
was added (when certain notices are used).
This commit rewrites the way that weirds are logged and fixes a number
of issues on the way. Most prominently, flow weirds now actually log
information about the flow that they occur in (before this change, they
only logged the name of the weird, which is only marginally helpful).
Besides restructuring how weird logging works internally, weirds can now
also be generated by calling Weird::weird with the info record directly,
allowing more fine-granular passing of information. This is e.g. used
for DNS weirds, which do not have the connection record available any
more when they are generated (before data like the connection ID was
just not logged in these instances).
Addresses BIT-1578
File Analysis Framework related code has been moved into a separate
script. Using redefinitions of the corresponding records causes the
file-related columns to appear last.
We now extract email addresses in the fields that one would expect
to contain addresses. This makes further downstream processing of
these fields easier like log analysis or using these fields in the
Intel framework. The primary downside is that any other content
in these fields is no longer available such as full name and any
group information. I believe the simplification of the content in
these fields is worth the change.
Added "cc" to the script that feeds information from SMTP into the
Intel framework.
A new script for email handling utility functions has been created
as a side effect of these changes.
This changes the HTTP log format slightly but shouldn't mess
up anything that anyone was doing because the old "filename"
field was never actually filled out. Tests are updated as well.
