Currently, creating a StringVal from a std::string did not work with
data that contains \0 characters. This easy fix changes this - and
should also lead to a small speed increase for code using this
constructor.
This obviously means that more data might copied now in some cases that
were previously cut off at the first 0-byte. Our test-suite did not
reveal any such cases.
The default value of an ID is now truly the one used to initialize it,
unaltered by any subsequent redefs.
Redefs are now shown separately, along with the expression that
modifies the ID's value.
* 'paraglob' of https://github.com/ZekeMedley/zeek:
Add leak test to paraglob.
Catch paraglob serialization errors in DoClone.
Update paraglob serialization.
Stop execution on paraglob error.
Update paraglob submodule
Change C++11 detection in paraglob.
Make paraglob serializable and copyable.
Initial paraglob integration.
I made a bunch of small changes:
* paraglob now deals better with \0 characters
* I rolled back the changes to Binary Serialization format,
* there were some small formatting issue
* the error output was slightly unsafe
* build_unique is now in util.h.
and perhaps a few more small things.
* origin/topic/timw/171-deprecation-warnings:
GH-171: support warning messages alongside deprecated attributes
Made a minor tweak to give a better error message when using
&deprecated= with something other than a string literal expression.
* 'rdp_client_cluster_data' of https://github.com/neslog/zeek:
Fixing types.
Correcting types.
Removing misc data from Client Cluster data trying to assign values.
Adding options field to RDP::ClientChannelDef Adding Client Cluster Data
Adjustments:
- Reformatting
- Added comments
- Changed the REDIRECTED_SESSIONID_FIELD_VALID field to a bool
Fixed a few small bugs - Modifiable had an uninitialized member and the
Registry looped over a map while deleting elements from it.
Fixes GH-319
* remotes/origin/topic/robin/gh59-when:
Renaming src/StateAccess.{h,cc} to src/Notifier.{h,cc}.
Clean up new code.
Remove MutableVal class.
Redo API for notifiers.
Remove most of MutableVal (but not the class itelf yet)
Remove enum Opcode.
Remove StateAccess class.
Redo NotfifierRegistry to no longer rely on StateAccess.
Add new test for when-statement watching global variables.
Couple of compile fixes.
* origin/topic/johanna/remove-serializer:
Fix memory leak introduced by removing opaque of ocsp_resp.
Change return value of OpaqueVal::DoSerialize.
Add missing ShallowClone implementation for SetType
Remove opaque of ocsp_resp.
Remove remnants of event serializer.
Fix cardinalitycounter deserialization.
Smaller compile fixes for the new opaque serialization.
Reimplement serialization infrastructure for OpaqueVals.
Couple of compile fixes.
Remove const from ShallowClone.
Remove test-case for removed functionality
Implement a Shallow Clone operation for types.
Remove value serialization.
Various changes I made:
- Fix memory leak in type-checker for opaque vals wrapped in broker::data
- Noticed the two "copy-all" leak tests weren't actually checking for
memory leaks because the heap checker isn't active until after zeek_init()
is evaluated.
- Change OpaqueVal::DoClone to use the clone caching mechanism
- Improve copy elision for broker::expected return types in the various
OpaqueVal serialize methods
- Not all compilers end up properly treating the return of
local/automatic variable as an rvalue that can be moved, and ends up
copying it instead.
- Particularly, until GCC 8, this pattern ends up copying instead of
moving, and we still support platforms whose default compiler
pre-dates that version.
- Generally seems it's something that wasn't addressed until C++14.
See http://www.open-std.org/jtc1/sc22/wg21/docs/cwg_defects.html#1579
- Change OpaqueVal::SerializeType to return broker::expected
- Change probabilistic DoSerialize methods to return broker::expected
* origin/topic/timw/393-vector-slicing:
Fix memory leak in vector slice assignment
Misc. tweaks to vector slicing implementation
Add ability to grow/shrink a vector using slicing, also adds Insert/Remove methods for VectorVal
Allow assignment for vectors using slices
Check for integral slice indexes, add extra test for [:]
Return an empty vector if the indices for slicing don't make sense
GH-393: Add slice notation for vectors
Two parts to this:
* Only allow vector slice assignment in statement contexts, not in
arbitrary assignment expressions. E.g. it's not clear what the
resulting value of `(v[1:2] = vector(1))` is for further expression
chaining. For reference, Python doesn't allow it either.
* Add a subclass of AssignExpr to specialize the behavior for index
slice assignments (because its behavior regarding expression
chaining is different per the previous point) and Unref the RHS
of things like `v[1:2] = vector(1)` after IndexExpr::Assign is
finished inserting it (since no one else takes ownership of it).
Instead of using an Expr subclass, IndexSliceAssignExpr, we could
use a proper Stmt, since that's the only context we currently use it
for, but if we did ever to decide on allowing its use in arbitrary
expression contexts, then I expect we'll need it this way anyway
(just with a different IndexSliceAssignExpr::Eval implementation).
Only used in one event, without any way to use the opaque for anything
else. At this point this just seems like a complication that has no
reason to be there.
This one took me way too long to admit. Values were pushed back on
deserialization - instead of assigned. Meaning they were added to the
end of the already 0-assigned vector.
The mean thing here is that estimation still worked - just merging
resulted in 0. And estimation still was correct because m, V, alpha_m
are enough for this - and those were correctly copied...
With this change, all tests pass.
We need this to sender through Broker, and we also leverage it for
cloning opaques. The serialization methods now produce Broker data
instances directly, and no longer go through the binary formatter.
Summary of the new API for types derived from OpaqueVal:
- Add DECLARE_OPAQUE_VALUE(<class>) to the class declaration
- Add IMPLEMENT_OPAQUE_VALUE(<class>) to the class' implementation file
- Implement these two methods (which are declated by the 1st macro):
- broker::data DoSerialize() const
- bool DoUnserialize(const broker::data& data)
This machinery should work correctly from dynamic plugins as well.
OpaqueVal provides a default implementation of DoClone() as well that
goes through serialization. Derived classes can provide a more
efficient version if they want.
The declaration of the "OpaqueVal" class has moved into the header
file "OpaqueVal.h", along with the new serialization infrastructure.
This is breaking existing code that relies on the location, but
because the API is changing anyways that seems fine.
This adds an internal BiF
"Broker::__opaque_clone_through_serialization" that does what the name
says: deep-copying an opaque by serializing, then-deserializing. That
can be used to tests the new functionality from btests.
Not quite done yet. TODO:
- Not all tests pass yet:
[ 0%] language.named-set-ctors ... failed
[ 16%] language.copy-all-opaques ... failed
[ 33%] language.set-type-checking ... failed
[ 50%] language.table-init-container-ctors ... failed
[ 66%] coverage.sphinx-zeekygen-docs ... failed
[ 83%] scripts.base.frameworks.sumstats.basic-cluster ... failed
(Some of the serialization may still be buggy.)
- Clean up the code a bit more.
* 'ntp-rewrite' of https://github.com/mauropalumbo75/zeek: (25 commits)
update tests baseline
Apply requested changes: - file dpd.sig and TODO comments for signature protocol detection removed - missing doc field filled in events.bif - rename OpCode and ReqCode fields into op_code and req_code respectively - removed unnecessary child method in NTP.h/.cc - main.zeek and ntp-protocol.pac reformatted
minor changes in the documentation
fix some initializations
fix wrong assignment of control key_id/crypto_checksum
code clean up
add extension fields parsing
add extended mac field with 20 byte digest (+4 byte key id)
update tests and add a new one for key_id and mac
fix auth field (key_id and mac) in standard and control msg
remove old NTP record in init-bare.zeek
fix key_id and digest (WIP)
fix wrong Assign with reference_id
add tests for ntp protocol (finished)
add tests for ntp protocol (WIP)
fix problem with time vals
add ntp records to init-bare.zeek
update ntp analyzer to val_mgr
extend and refact script-side of NTP analyzer
extend and refactor several fields
...
SPF response records are identical to TXT records in structure, and
can be parsed and interpreted the same way. However, they have a
different RR type, so they would generate weird events and not be
parsed by Zeek before this change.
Even though they're the same as TXT records from a protocol stance, I
created a new event type (dns_SPF_reply), and call the records out as
SPF in the logs, instead of as TXT records, since the distinction
could be important for detection purposes.
SPF records have been obsoleted, but continue to be seen in the wild.
* origin/topic/johanna/tls13-details:
Update SSL documentation.
support the newer TLS 1.3 key_share extension.
Include all data of the server-hello random
Parse pre-shared-key extension.
Added a small portability fix for the gmt_unix_time byte-swapping.
* 'rdp_client_security' of https://github.com/neslog/zeek:
Adding comments specific to client security data in record definition.
Cleaning up indentations and return true.
Adding record to init-bare
Adding client_security_data to the analyzer.
I added a unit test.
By using a consistent timestamp. That avoids rare chances of sqlite
output from rounding the current time into such a form that happens
to bypass the timestamp canonifier script (whenever it happened to
land on a whole or tenth second).
* "bro_is_terminating" is now "zeek_is_terminating"
* "bro_version" is now "zeek_version"
The old function names still exist for now, but are deprecated.
These are no longer loaded by default due to the performance impact they
cause simply by being loaded (they have event handlers for commonly
generated events) and they aren't generally useful enough to justify it.
