* topic/robin/hyperloglog-merge: (35 commits)
Making the confidence configurable.
Renaming HyperLogLog->CardinalityCounter.
Fixing bug introduced during merging.
add clustered leak test for hll. No issues.
make gcc happy
(hopefully) fix refcounting problem in hll/bloom-filter opaque vals. Thanks Robin.
re-use same hash class for all add operations
get hll ready for merging
and forgot a file...
adapt to new structure
fix opaqueval-related memleak.
make it compile on case-sensitive file systems and fix warnings
make error rate configureable
add persistence test not using predetermined random seeds.
update cluster test to also use hll
persistence really works.
well, with this commit synchronizing the data structure should work.. ...if we had consistent hashing.
and also serialize the other things we need
ok, this bug was hard to find.
serialization compiles.
...
* origin/topic/bernhard/hyperloglog: (32 commits)
add clustered leak test for hll. No issues.
make gcc happy
(hopefully) fix refcounting problem in hll/bloom-filter opaque vals. Thanks Robin.
re-use same hash class for all add operations
get hll ready for merging
and forgot a file...
adapt to new structure
fix opaqueval-related memleak.
make it compile on case-sensitive file systems and fix warnings
make error rate configureable
add persistence test not using predetermined random seeds.
update cluster test to also use hll
persistence really works.
well, with this commit synchronizing the data structure should work.. ...if we had consistent hashing.
and also serialize the other things we need
ok, this bug was hard to find.
serialization compiles.
change plugin after feedback of seth
Forgot a file. Again. Like always. Basically.
do away with old file.
...
CompositeHash.
We do this by hashing values added to a BloomFilter another time more
with a stable hash seeded only by either the filter's name or the
global_hash_seed (or Bro's random() seed if neither is defined).
I'm also adding a new bif bloomfilter_internal_state() that returns a
string representation of a Bloom filter's current internal state. This
is solely for writing tests that check that the filters end up
consistent when seeded with the same value.
I'm moving the new files into a subdirectory probabilistic, and into a
corresponding namespace. We can later put code for the other
probabilistic data structures there as well.
* origin/topic/matthias/bloom-filter: (45 commits)
Implement and test Bloom filter merging.
Make hash functions equality comparable.
Make counter vectors mergeable.
Use half adder for bitwise addition and subtraction.
Fix and test counting Bloom filter.
Implement missing CounterVector functions.
Tweak hasher interface.
Add missing include for GCC.
Fixing for unserializion error.
Small fixes and style tweaks.
Only serialize Bloom filter type if available.
Create hash policies through factory.
Remove lingering debug code.
Factor implementation and change interface.
Expose Bro's linear congruence PRNG as utility function.
H3 does not check for zero length input.
Support seeding for hashers.
Add utility function to access first random seed.
Update H3 documentation (and minor style nits.)
Make H3 seed configurable.
...
We also keep track of the Bloom filter's element type inside each value. The
first use of the BiF bloomfilter_add will "typify" the Bloom filter and lock
the Bloom filter's type to the element type.
hyperloglog.h was missing guards and randomly deleting memory at
addresses equal to variable contents.
I am not entirely sure why that did not crash before...
* origin/topic/matthias/opaque:
Add new unit test for opaque serialization.
Migrate entropy testing to opaque.
C++ify RandTest.*
Fix a hard-to-spot bug.
Use more descriptive error message.
Fix the fix :-/.
Fix initialization of hash values.
Be clearer about delegation.
Implement serialization of opaque types.
Update hash BiF documentation.
Migrate free SHA* functions to SHA*Val::digest().
Add missing type name that caused failing tests.
Update base scripts and unit tests.
Simplify hash function BiFs.
Add support for opaque hash values.
Adapt BiF & Bro parser to handle opaque types.
More lexer/parser work.
Implement equivalence relation for opaque types.
Support basic serialization of opaque.
Add opaque type to lexer, parser, and BroType.
Closes#925
Conflicts:
aux/broccoli
