Adjustments during merge:
- kept the UNKNOWN Log::ID as placeholder value
- changed the coverage.find-bro-logs test to check for arbitrary $path
field values instead of just string literals
- don't force EnumVal to unsigned integer since the relevant union member
is the signed integer and added the relevant enum values/types to
.bif files for easier access
- compare FILE* versus file name to check for stdout equality (don't
think it matters much, just a bit more efficient)
- minor whitespace/style tweaks
* origin/topic/dev/print-to-log:
Added a non boolean configuration and other changes as suggested by Jon
Allow Print Statements to be redirected to a Log# This is a combination of 3 commits.
And switch Zeek's base scripts over to using it in place of
"connection_state_remove". The difference between the two is
that "connection_state_remove" is raised for all events while
"successful_connection_remove" excludes TCP connections that were never
established (just SYN packets). There can be performance benefits
to this change for some use-cases.
There's also a new event called ``connection_successful`` and a new
``connection`` record field named "successful" to help indicate this new
property of connections.
Now, in addition to setting thresholds for bytes and packet, one can set
a threshold for connection duration. Note that the threshold event is
only raised once the next packet in the connection is seen.
This also fixes a small pre-existing bug, in which a bunch of warnings
were raised if someone just used the lower-level functions without going
through the higher-level scripting API.
For broker.log and cluster.log: there was a race condition. A worker's
first IOSource that it processes is potentially Broker if there were
no packets available yet and thread scheduling happens to work out
such that network connections (inside CAF threads) become established
before we enter the main I/O loop. Such peering establishments would
generate logs with timestamp 0 as there was not yet any code path
taken that would update network_time.
For reporter.log: any non-worker (packet-processing) node would just
unnecessarily use a timestamp of 0 for their reporter messages.
* origin/topic/johanna/remove-serializer:
Fix memory leak introduced by removing opaque of ocsp_resp.
Change return value of OpaqueVal::DoSerialize.
Add missing ShallowClone implementation for SetType
Remove opaque of ocsp_resp.
Remove remnants of event serializer.
Fix cardinalitycounter deserialization.
Smaller compile fixes for the new opaque serialization.
Reimplement serialization infrastructure for OpaqueVals.
Couple of compile fixes.
Remove const from ShallowClone.
Remove test-case for removed functionality
Implement a Shallow Clone operation for types.
Remove value serialization.
Various changes I made:
- Fix memory leak in type-checker for opaque vals wrapped in broker::data
- Noticed the two "copy-all" leak tests weren't actually checking for
memory leaks because the heap checker isn't active until after zeek_init()
is evaluated.
- Change OpaqueVal::DoClone to use the clone caching mechanism
- Improve copy elision for broker::expected return types in the various
OpaqueVal serialize methods
- Not all compilers end up properly treating the return of
local/automatic variable as an rvalue that can be moved, and ends up
copying it instead.
- Particularly, until GCC 8, this pattern ends up copying instead of
moving, and we still support platforms whose default compiler
pre-dates that version.
- Generally seems it's something that wasn't addressed until C++14.
See http://www.open-std.org/jtc1/sc22/wg21/docs/cwg_defects.html#1579
- Change OpaqueVal::SerializeType to return broker::expected
- Change probabilistic DoSerialize methods to return broker::expected
* origin/topic/timw/393-vector-slicing:
Fix memory leak in vector slice assignment
Misc. tweaks to vector slicing implementation
Add ability to grow/shrink a vector using slicing, also adds Insert/Remove methods for VectorVal
Allow assignment for vectors using slices
Check for integral slice indexes, add extra test for [:]
Return an empty vector if the indices for slicing don't make sense
GH-393: Add slice notation for vectors
Two parts to this:
* Only allow vector slice assignment in statement contexts, not in
arbitrary assignment expressions. E.g. it's not clear what the
resulting value of `(v[1:2] = vector(1))` is for further expression
chaining. For reference, Python doesn't allow it either.
* Add a subclass of AssignExpr to specialize the behavior for index
slice assignments (because its behavior regarding expression
chaining is different per the previous point) and Unref the RHS
of things like `v[1:2] = vector(1)` after IndexExpr::Assign is
finished inserting it (since no one else takes ownership of it).
Instead of using an Expr subclass, IndexSliceAssignExpr, we could
use a proper Stmt, since that's the only context we currently use it
for, but if we did ever to decide on allowing its use in arbitrary
expression contexts, then I expect we'll need it this way anyway
(just with a different IndexSliceAssignExpr::Eval implementation).
* 'ntp-rewrite' of https://github.com/mauropalumbo75/zeek: (25 commits)
update tests baseline
Apply requested changes: - file dpd.sig and TODO comments for signature protocol detection removed - missing doc field filled in events.bif - rename OpCode and ReqCode fields into op_code and req_code respectively - removed unnecessary child method in NTP.h/.cc - main.zeek and ntp-protocol.pac reformatted
minor changes in the documentation
fix some initializations
fix wrong assignment of control key_id/crypto_checksum
code clean up
add extension fields parsing
add extended mac field with 20 byte digest (+4 byte key id)
update tests and add a new one for key_id and mac
fix auth field (key_id and mac) in standard and control msg
remove old NTP record in init-bare.zeek
fix key_id and digest (WIP)
fix wrong Assign with reference_id
add tests for ntp protocol (finished)
add tests for ntp protocol (WIP)
fix problem with time vals
add ntp records to init-bare.zeek
update ntp analyzer to val_mgr
extend and refact script-side of NTP analyzer
extend and refactor several fields
...
* 'master' of https://github.com/ZekeMedley/zeek:
Use the right delete and improve the leak test. Increases the size of the table being loaded in the pattern leak test and uses the right delete method.
Fix formatting.
Fix memory leak and add test.
Add pattern support to input framework.
All types (besides EntropyVal) now support a native copy operation,
which uses primitives of the underlying datatypes to perform a quick
copy, without serialization.
EntropyVal is the one exception - since that type is rather complex
(many members) and will probably not be copied a lot, if at all, it
makes sense to just use the serialization function.
This will have to be slightly re-written in the near-term-future to use
the new serialization function for that opaque type.
This change also introduces a new x509_from_der bif, which allows to
parse a der into an opaque of x509.
This change removes the d2i_X509_ wrapper function; this was a remnant
when d2i_X509 took non-const arguments. We directly use d2i_X509 at
several places assuming const-ness, so there does not seem to ba a
reason to keep the wrapper.
This change also exposed a problem in the File cache - cases in which an
object was brought back into the cache, and writing occurred in the
file_open event were never correctly handeled as far as I can tell.
For backward compatibility when reading values, we first check
the ZEEK-prefixed value, and if not set, then check the corresponding
BRO-prefixed value.
* origin/topic/robin/gh-239:
Undo a change to btest.cfg from a recent commit
Updating submodule.
Fix zeek-wrapper
Update for renaming BroControl to ZeekControl.
Updating submodule.
GH-239: Rename bro to zeek, bro-config to zeek-config, and bro-path-dev to zeek-path-dev.
This commit removed functions/events that have been deprecated in Bro
2.6. It also removes the detection code that checks if the old
communication framework is used (since all the functions that are
checked were removed).
Addresses parts of GH-243
This also installs symlinks from "zeek" and "bro-config" to a wrapper
script that prints a deprecation warning.
The btests pass, but this is still WIP. broctl renaming is still
missing.
#239
The load-duplicates.bro test would never fail because loading
the provided script code twice wouldn't trigger an error.
Fixed this by changing the sample script content. Also added a
test case to verify that an error occurs as expected
when two scripts with the same content are loaded.
* 'master' of https://github.com/rdenniston/zeek:
Add linux netfilter NFLOG capture functionality initial commit
I made modifications:
- Formatting / code style
- More error handling and validity checks
- The Type and Length value of TLVs is technically host order
- Changed / fixed the Length value padding check: it's generally
32-bit alignment, not just aligning any TLV less than 8 bytes.
* Better parsing/error-checking of VXLAN and encapsulated packet headers
* Add/implement the "vxlan_packet" event
* Add "Tunnel::vxlan_ports" option to tune the set of VXLAN ports to
analyze/decapsulate
* Add "Tunnel::validate_vxlan_checksums" option to allow for tuning of how
checksums associated with the outer UDP header of a possible VXLAN
tunnel are handled
Fixes GH-250
