* origin/topic/timw/cleaner-utf8:
GHI-486: Switch over to using LLVM utf8-checking code to better validate characters
I addressed a buffer over-read during the merge and added test-cases for
it.
The field is populated in this order of preference:
(1) Use a client-identifier option sent by client
(2) Use the server's CHADDR field
(3) Use the client's CHADDR field
Case (3) did not exist before this patch.
* origin/topic/zeke/expire-func:
Ignore abs-path in test.
Report argument # type check failed on.
Update test baseline.
Improve func arg type checking.
&expire_func(table, arg1, arg2, ...) + type checking.
This allows one to tune the number of protocol violations to tolerate
from any given analyzer type before just disabling a given instance
of it.
Also removes the "disabled_aids" field from the DPD::Info record
since it serves no purpose: in this case, calling disable_analyzer
multiple times for the same analyzer is a no-op.
Which was used by the new ShallowClone method and causes multiple enum
declarations/definitions to now crash. Such code is not typically
seen in scripts, but being able to define an enum in both .bif and
.bro files has historically been supported.
Unit tests sometimes use current_time(), which occassionally lands on
a perfect tenth of a second and end up fooling the old canonification
regexp and resulting in output which differs from what's in the
test baseline.
* origin/topic/timw/150-to-json:
Update submodules for JSON work
Update unit tests for JSON logger to match new output
Modify JSON log writer to use the external JSON library
Update unit test output to match json.zeek being deprecated and slight format changes to JSON output
Add proper JSON serialization via C++, deprecate json.zeek
Add new method for escaping UTF8 strings for JSON output
Move do_sub method from zeek.bif to StringVal class method
Move record_fields method from zeek.bif to Val class method
Add ToStdString method for StringVal
For broker.log and cluster.log: there was a race condition. A worker's
first IOSource that it processes is potentially Broker if there were
no packets available yet and thread scheduling happens to work out
such that network connections (inside CAF threads) become established
before we enter the main I/O loop. Such peering establishments would
generate logs with timestamp 0 as there was not yet any code path
taken that would update network_time.
For reporter.log: any non-worker (packet-processing) node would just
unnecessarily use a timestamp of 0 for their reporter messages.
- rotate_file
- rotate_file_by_name
- calc_next_rotate
These still have use-cases even though no longer used for our logging
functionality. E.g. rotate_file_by_name may be used to rotate
pcap dump files.
Also the log_rotate_base_time option was marked deprecated, but still
used in the new logging framework.
Currently, creating a StringVal from a std::string did not work with
data that contains \0 characters. This easy fix changes this - and
should also lead to a small speed increase for code using this
constructor.
This obviously means that more data might copied now in some cases that
were previously cut off at the first 0-byte. Our test-suite did not
reveal any such cases.
In the past they were processed on the manager - which requires big
records to be sent around.
This has a potential of incompatibilities if someone relied on global
state for notice processing.
GH-214
The default value of an ID is now truly the one used to initialize it,
unaltered by any subsequent redefs.
Redefs are now shown separately, along with the expression that
modifies the ID's value.
* 'paraglob' of https://github.com/ZekeMedley/zeek:
Add leak test to paraglob.
Catch paraglob serialization errors in DoClone.
Update paraglob serialization.
Stop execution on paraglob error.
Update paraglob submodule
Change C++11 detection in paraglob.
Make paraglob serializable and copyable.
Initial paraglob integration.
I made a bunch of small changes:
* paraglob now deals better with \0 characters
* I rolled back the changes to Binary Serialization format,
* there were some small formatting issue
* the error output was slightly unsafe
* build_unique is now in util.h.
and perhaps a few more small things.
* origin/topic/timw/171-deprecation-warnings:
GH-171: support warning messages alongside deprecated attributes
Made a minor tweak to give a better error message when using
&deprecated= with something other than a string literal expression.
* 'rdp_client_cluster_data' of https://github.com/neslog/zeek:
Fixing types.
Correcting types.
Removing misc data from Client Cluster data trying to assign values.
Adding options field to RDP::ClientChannelDef Adding Client Cluster Data
Adjustments:
- Reformatting
- Added comments
- Changed the REDIRECTED_SESSIONID_FIELD_VALID field to a bool
Fixed a few small bugs - Modifiable had an uninitialized member and the
Registry looped over a map while deleting elements from it.
Fixes GH-319
* remotes/origin/topic/robin/gh59-when:
Renaming src/StateAccess.{h,cc} to src/Notifier.{h,cc}.
Clean up new code.
Remove MutableVal class.
Redo API for notifiers.
Remove most of MutableVal (but not the class itelf yet)
Remove enum Opcode.
Remove StateAccess class.
Redo NotfifierRegistry to no longer rely on StateAccess.
Add new test for when-statement watching global variables.
Couple of compile fixes.
* origin/topic/johanna/remove-serializer:
Fix memory leak introduced by removing opaque of ocsp_resp.
Change return value of OpaqueVal::DoSerialize.
Add missing ShallowClone implementation for SetType
Remove opaque of ocsp_resp.
Remove remnants of event serializer.
Fix cardinalitycounter deserialization.
Smaller compile fixes for the new opaque serialization.
Reimplement serialization infrastructure for OpaqueVals.
Couple of compile fixes.
Remove const from ShallowClone.
Remove test-case for removed functionality
Implement a Shallow Clone operation for types.
Remove value serialization.
Various changes I made:
- Fix memory leak in type-checker for opaque vals wrapped in broker::data
- Noticed the two "copy-all" leak tests weren't actually checking for
memory leaks because the heap checker isn't active until after zeek_init()
is evaluated.
- Change OpaqueVal::DoClone to use the clone caching mechanism
- Improve copy elision for broker::expected return types in the various
OpaqueVal serialize methods
- Not all compilers end up properly treating the return of
local/automatic variable as an rvalue that can be moved, and ends up
copying it instead.
- Particularly, until GCC 8, this pattern ends up copying instead of
moving, and we still support platforms whose default compiler
pre-dates that version.
- Generally seems it's something that wasn't addressed until C++14.
See http://www.open-std.org/jtc1/sc22/wg21/docs/cwg_defects.html#1579
- Change OpaqueVal::SerializeType to return broker::expected
- Change probabilistic DoSerialize methods to return broker::expected
