There was a copy of every single packet being made as it entered
the SMB analyzer. This was because the entire analyzer used to be
handwritten (only using binpac for unit parsing, not for parsing
the whole protocol). Now that the entire parser is binpac we don't
need that layer of indirection anymore and we're able to remove
a lot of complexity.
The types used by the core did not match the types expected by the
scripting framework, leading, e.g., to trouble with the logging
framework.
Discovered and fixed by Grant Moyer.
Fixes BIT-1650
Calling Error() in an input reader now automatically will disable the
reader and return a failure in the Update/Heartbeat calls.
Also adds more tests.
Addresses BIT-1181
This change introduces error events for Table and Event readers. Users
can now specify an event that is called when an info, warning, or error
is emitted by their input reader. This can, e.g., be used to raise
notices in case errors occur when reading an important input stream.
Example:
event error_event(desc: Input::TableDescription, msg: string, level: Reporter::Level)
{
...
}
event bro_init()
{
Input::add_table([$source="a", $error_ev=error_event, ...]);
}
For the moment, this converts all errors in the Asciiformatter into
warnings (to show that they are non-fatal) - the Reader itself also has
to throw an Error to show that a fatal error occurred and processing
will be abort.
It might be nicer to change this and require readers to mark fatal
errors as such when throwing them.
Addresses BIT-1181
This allows all threads accessing the same database to share sqlite
objects. This, for example, fixes the issue with several threads
simultaneously writing to the same database file.
See https://www.sqlite.org/sharedcache.html
Addresses BIT-1325
Addig a new random seed for external tests.
I added a wrapper around the siphash() function to make calling it a
little bit safer at least.
BIT-1612 #merged
* origin/topic/johanna/bit-1612:
HLL: Fix missing typecast in test case.
Remove the -K/-J options for setting keys.
Add test checking the quality of HLL by adding a lot of elements.
Fix serializing probabilistic hashers.
Baseline updates after hash function change.
Also switch BloomFilters from H3 to siphash.
Change Hashing from H3 to Siphash.
HLL: Remove unnecessary comparison.
Hyperloglog: change calculation of Rho
VectorType unref'd the Record type we used during construction,
apparently leading to it not being present anymore at some point of
time.
Fix this problem by doing it just like all other analyzers.
The options were never really used and do not seem especially useful;
initialization with a seed file still works.
This also fixes a bug with the initialization of the siphash key.
The test adds 170,000 IP addresses. After the recent hashing changes,
HLL estimates 171,250 entries (completely stable). Before, HLL estimated,
depending on the initial seeds, ~700 to 300,000 entries.
This commit mostly changes the hash function that is used for Internal
hashing of data < 36 bytes from H3 to Siphash. This change is motivated
by the fact that it turns out that H3 apparently does not deliver a very
good source of data uniqueness; running HLL with H3 as a hashing
function results in quite poor results (up to of 75% off in my tests).
In difference, running HLL with Siphash (or HMAC-MD5) changes this
factor to ~2%.
This also fixes a long-standing bug in Hash.h which truncated our hash
values to 32 bit on most machines.
Furthermore, it once again fixes a problem with the Rank function in
HLL.
non-partial connections.
Before, if we saw a responder-side SYN/ACK, but had not seen the
initial orginator-side SYN, Bro would treat the connection as partial,
meaning that most application-layer analyzers would refuse to inspect
the payload. That was unfortunate because all payload data was
actually there (and even passed to the analyzers). This change make
Bro consider these connections as complete, so that analyzers will
just normally process them.
The leads to couple more connections in the test-suite to now being
analyzed.
Addresses #1492. (I used an HTTP trace for debugging instead of the
HTTPS trace from the ticket, as the clear-text makes it easier to
track the data flow).
* origin/topic/dnthayer/ticket1645:
Disable broker by default
I added the --disable-broker back to configure (as a no-op), to not
break the build for everyone that is currently doing that in their build
scripts.
BIT-1645 #merged
