Commit graph

11 commits

Author SHA1 Message Date
Robin Sommer
4d84ee82da Merge remote-tracking branch 'origin/topic/johanna/bit-1612'
Addig a new random seed for external tests.

I added a wrapper around the siphash() function to make calling it a
little bit safer at least.

BIT-1612 #merged

* origin/topic/johanna/bit-1612:
  HLL: Fix missing typecast in test case.
  Remove the -K/-J options for setting keys.
  Add test checking the quality of HLL by adding a lot of elements.
  Fix serializing probabilistic hashers.
  Baseline updates after hash function change.
  Also switch BloomFilters from H3 to siphash.
  Change Hashing from H3 to Siphash.
  HLL: Remove unnecessary comparison.
  Hyperloglog: change calculation of Rho
2016-07-14 16:26:17 -07:00
Johanna Amann
e1218cc7fa Change Hashing from H3 to Siphash.
This commit mostly changes the hash function that is used for Internal
hashing of data < 36 bytes from H3 to Siphash. This change is motivated
by the fact that it turns out that H3 apparently does not deliver a very
good source of data uniqueness; running HLL with H3 as a hashing
function results in quite poor results (up to of 75% off in my tests).
In difference, running HLL with Siphash (or HMAC-MD5) changes this
factor to ~2%.

This also fixes a long-standing bug in Hash.h which truncated our hash
values to 32 bit on most machines.

Furthermore, it once again fixes a problem with the Rank function in
HLL.
2016-07-13 06:44:51 -07:00
Robin Sommer
3957091e1b Renaming config.h to bro-config.h.
A couple times now I had this conflicting with files of the same name
in other projects.
2015-07-28 11:57:04 -07:00
Jon Siwek
d7dafe2fe2 Refactoring various usages of new IPAddr class.
Reducing number of places that internal representation was exposed
via GetBytes/CopyIPv6.

Also fixed a bug in remask_addr bif.
2012-02-22 14:45:44 -06:00
Jon Siwek
0f207c243c Port DNS_Mgr to use new IPAddr class, enable lookups on IPv6 addrs.
Host lookups still need to be changed to also do AAAA queries.
2012-02-13 15:57:59 -06:00
Robin Sommer
bd2e30f521 Merge remote-tracking branch 'origin/topic/dist-cleanup'
* origin/topic/dist-cleanup:
  Updating INSTALL
  Updating README
  Remove $Id$ tags
  Remove policy.old directory, adresses #511
2011-09-18 16:17:42 -07:00
Jon Siwek
d412aa9d63 Fix H3 assumption of an 8-bit byte/char.
The hash function was internally casting the void* data argument into an
unsigned char* and then using values from that to index another internal
array that's dimensioned based on the assumption of 256 values possible
for an unsigned char (8-bit chars/bytes).  This is probably a correct
assumption most of the time, but should be safer to use the limits as
defined in standard headers to get it right for the particular
system/compiler.

There was an unused uint8* casted variable in HashKey::HashBytes that
seemed like it might have been meant to be passed to H3's hash function
as an unfinished attempt to solve the 8-bit byte assumption problem, but
that doesn't seem as good as taking care of that internally in H3 so
users of the API are only concerned with byte sizes as reported by
`sizeof`.  Removing the unused variable addresses #530.

Also a minor tweak to an hmac_md5 call that was casting away const from
one argument (which doesn't match the prototype).
2011-08-17 15:03:18 -05:00
Jon Siwek
495e987938 Remove $Id$ tags 2011-08-04 15:21:18 -05:00
Robin Sommer
03c0d587a4 Removing code for unused hash functions. 2011-04-01 16:09:28 -07:00
Jon Siwek
13569aaab7 Removal of the --enable-int64 config option.
This will now be always on.  As such, uses of the USE_INT64 preprocessor
definition have been cleaned out.
2010-11-17 20:38:33 -06:00
Robin Sommer
61757ac78b Initial import of svn+ssh:://svn.icir.org/bro/trunk/bro as of r7088 2010-09-27 20:42:30 -07:00