Reimplement serialization infrastructure for OpaqueVals.

We need this to sender through Broker, and we also leverage it for
cloning opaques. The serialization methods now produce Broker data
instances directly, and no longer go through the binary formatter.

Summary of the new API for types derived from OpaqueVal:

    - Add DECLARE_OPAQUE_VALUE(<class>) to the class declaration
    - Add IMPLEMENT_OPAQUE_VALUE(<class>) to the class' implementation file
    - Implement these two methods (which are declated by the 1st macro):
        - broker::data DoSerialize() const
        - bool DoUnserialize(const broker::data& data)

This machinery should work correctly from dynamic plugins as well.

OpaqueVal provides a default implementation of DoClone() as well that
goes through serialization. Derived classes can provide a more
efficient version if they want.

The declaration of the "OpaqueVal" class has moved into the header
file "OpaqueVal.h", along with the new serialization infrastructure.
This is breaking existing code that relies on the location, but
because the API is changing anyways that seems fine.

This adds an internal BiF
"Broker::__opaque_clone_through_serialization" that does what the name
says: deep-copying an opaque by serializing, then-deserializing. That
can be used to tests the new functionality from btests.

Not quite done yet. TODO:
    - Not all tests pass yet:
        [  0%] language.named-set-ctors ... failed
        [ 16%] language.copy-all-opaques ... failed
        [ 33%] language.set-type-checking ... failed
        [ 50%] language.table-init-container-ctors ... failed
        [ 66%] coverage.sphinx-zeekygen-docs ... failed
        [ 83%] scripts.base.frameworks.sumstats.basic-cluster ... failed

      (Some of the serialization may still be buggy.)

    - Clean up the code a bit more.
This commit is contained in:
Robin Sommer 2019-06-15 21:19:21 +00:00
parent 1ce5521ecc
commit 01e662b3e0
28 changed files with 1556 additions and 52 deletions

View file

@ -505,6 +505,47 @@ uint64 BitVector::Hash() const
return digest;
}
broker::expected<broker::data> BitVector::Serialize() const
{
broker::vector v = {static_cast<uint64>(num_bits), static_cast<uint64>(bits.size())};
v.reserve(2 + bits.size());
for ( size_t i = 0; i < bits.size(); ++i )
v.emplace_back(static_cast<uint64>(bits[i]));
return {v};
}
std::unique_ptr<BitVector> BitVector::Unserialize(const broker::data& data)
{
auto v = caf::get_if<broker::vector>(&data);
if ( ! (v && v->size() >= 2) )
return nullptr;
auto num_bits = caf::get_if<uint64>(&(*v)[0]);
auto size = caf::get_if<uint64>(&(*v)[1]);
if ( ! (num_bits && size) )
return nullptr;
if ( v->size() != 2 + *size )
return nullptr;
auto bv = std::unique_ptr<BitVector>(new BitVector());
bv->num_bits = *num_bits;
for ( size_t i = 0; i < *size; ++i )
{
auto x = caf::get_if<uint64>(&(*v)[2 + i]);
if ( ! x )
return nullptr;
bv->bits.push_back(*x);
}
return std::move(bv);
}
BitVector::size_type BitVector::lowest_bit(block_type block)
{
block_type x = block - (block & (block - 1));