We need this to sender through Broker, and we also leverage it for
cloning opaques. The serialization methods now produce Broker data
instances directly, and no longer go through the binary formatter.
Summary of the new API for types derived from OpaqueVal:
- Add DECLARE_OPAQUE_VALUE(<class>) to the class declaration
- Add IMPLEMENT_OPAQUE_VALUE(<class>) to the class' implementation file
- Implement these two methods (which are declated by the 1st macro):
- broker::data DoSerialize() const
- bool DoUnserialize(const broker::data& data)
This machinery should work correctly from dynamic plugins as well.
OpaqueVal provides a default implementation of DoClone() as well that
goes through serialization. Derived classes can provide a more
efficient version if they want.
The declaration of the "OpaqueVal" class has moved into the header
file "OpaqueVal.h", along with the new serialization infrastructure.
This is breaking existing code that relies on the location, but
because the API is changing anyways that seems fine.
This adds an internal BiF
"Broker::__opaque_clone_through_serialization" that does what the name
says: deep-copying an opaque by serializing, then-deserializing. That
can be used to tests the new functionality from btests.
Not quite done yet. TODO:
- Not all tests pass yet:
[ 0%] language.named-set-ctors ... failed
[ 16%] language.copy-all-opaques ... failed
[ 33%] language.set-type-checking ... failed
[ 50%] language.table-init-container-ctors ... failed
[ 66%] coverage.sphinx-zeekygen-docs ... failed
[ 83%] scripts.base.frameworks.sumstats.basic-cluster ... failed
(Some of the serialization may still be buggy.)
- Clean up the code a bit more.
All types (besides EntropyVal) now support a native copy operation,
which uses primitives of the underlying datatypes to perform a quick
copy, without serialization.
EntropyVal is the one exception - since that type is rather complex
(many members) and will probably not be copied a lot, if at all, it
makes sense to just use the serialization function.
This will have to be slightly re-written in the near-term-future to use
the new serialization function for that opaque type.
This change also introduces a new x509_from_der bif, which allows to
parse a der into an opaque of x509.
This change removes the d2i_X509_ wrapper function; this was a remnant
when d2i_X509 took non-const arguments. We directly use d2i_X509 at
several places assuming const-ness, so there does not seem to ba a
reason to keep the wrapper.
This change also exposed a problem in the File cache - cases in which an
object was brought back into the cache, and writing occurred in the
file_open event were never correctly handeled as far as I can tell.
Note - this compiles, but you cannot run Bro anymore - it crashes
immediately with a 0-pointer access. The reason behind it is that the
required clone functionality does not work anymore.
This commit marks (hopefully) ever one-parameter constructor as explicit.
It also uses override in (hopefully) all circumstances where a virtual
method is overridden.
There are a very few other minor changes - most of them were necessary
to get everything to compile (like one additional constructor). In one
case I changed an implicit operation to an explicit string conversion -
I think the automatically chosen conversion was much more convoluted.
This took longer than I want to admit but not as long as I feared :)
BIT-1048 #merged
I'm reverting the serializer version update for now as that breaks
Broccoli. Let's do that later for 2.2.
* topic/robin/topk-merge:
update documentation, rename get* to Get* and make hasher persistent
adapt to new folder structure
fix opaqueval-related memleak
synchronize pruned attribute
potentially found wrong Ref.
add sum function that can be used to get the number of total observed elements.
in cluster settings, the resultvals can apparently been uninitialized in some special cases
fix memory leaks
fix warnings
add topk cluster test
make size of topk-list configureable when using sumstats
implement merging for top-k.
add serialization for topk
make the get function const
topk for sumstats
well, a test that works..
implement topk.
* origin/topic/bernhard/topk:
adapt to new folder structure
fix opaqueval-related memleak
synchronize pruned attribute
potentially found wrong Ref.
add sum function that can be used to get the number of total observed elements.
in cluster settings, the resultvals can apparently been uninitialized in some special cases
fix memory leaks
fix warnings
add topk cluster test
make size of topk-list configureable when using sumstats
implement merging for top-k.
add serialization for topk
make the get function const
topk for sumstats
well, a test that works..
implement topk.
