- Add constructors for ConnIDKey, remove BuildConnIDKey()
- Rename protocol stats classes and move to implementation file
- Rename "num" field of protocol stats to "active"
- Explicitly delete copy operations for SessionKey
- Change argument for ProtocolStats methods to const-reference
- Make key validity methods in Session not be virtual
- Rename Session::ClearKey and Session::IsKeyValid
This also includes:
- Deprecating the NetSessions name.
- Renaming the zeek::sessions global to zeek::session_mgr and deprecating the old name.
- Renaming Sessions.{h,cc} to SessionManager.{h,cc}.
This commit also includes:
- Storing the transport protocol in ConnID and ConnIDKey to allow tcp and
udp connections from the same IP/Port combinations. This happens in the
core.cisco-fabric-path test, for example.
- Lots of test updates. The reasons for these are two fold. First, with
the change to only store a single map means that TCP, UDP, and ICMP
connections are now mixed. When Zeek drains the map at shutdown, it drains
each of those protocols together instead of separately. The second is
because of how Sessions are stored in the map. We're now storing them
keyed by the hash of the key stored by the Session objects, which causes
them to again be in the map in a different order.
* origin/topic/jsiwek/misc-table-stuff:
Add a fatal error condition for invalid Dictionary insertion distances
Fix using clear_table() within an &expire_func
Remove saving/restoring of value pointer after calling expire_func
Avoid allocating a HashKey for no-op table expiry iterations
Fixes to `decode_netbios_name`:
* Improve validation that input string is a NetBIOS encoding
(32 bytes, with characters ranging from 'A' to 'P'). This helps
prevent Undefined Behavior of left-shifting negative values.
Invalid encodings now cause a return-value of an empty string.
* More liberal in what decoded characters are allowed. Namely,
spaces are now allowed (but any trailing null-bytes and spaces
are trimmed, similar to before).
Fixes to `decode_netbios_name_type`:
* Improve validation that input string is a NetBIOS encoding
(32 bytes, with characters ranging from 'A' to 'P'). This helps
prevent Undefined Behavior of left-shifting negative values and
a heap-buffer-overread when the input string is too small.
Invalid encodings now cause a return-value of 256.
This patch adds the ability to decap Geneve packets to process the inner
payload. The structure of the analyzer borrows heavily from the VXLAN
analyzer.
When choosing poor/aggressive values for `table_expire_interval`,
`table_expire_delay`, and/or `table_incremental_step` that tend to
leave tables in state of constant table-expiry-iteration, the underlying
Dictionary is never allowed the chance to complete remapping operations
which re-position entries to more ideal locations (e.g. after
reallocating the table to be able to store more entries).
That situation not only leads to the Dictionary generally having a less
efficient structure, but eventually, the lack of re-positioning may
cause an insertion to calculate the new entry's
distance-from-ideal-position to be a value requiring a full 16-bits or
more (>=65535), but an entry only allows storing 16-bit distance values,
with 65535 being a sentinel value that is supposed to indicate an empty
entry. Dictionary operations may start misbehaving if that's allowed to
happen.
This previously crashed since clear_table()/TableVal::RemoveAll() left
behind a stale iterator to the old table causing a heap-use-after-free
when resuming table expiry iteration in TableVal::DoExpire().
It's no longer used for anything. Previously, it was used to detect
whether the expiry batch finished iterating the entire table or not, but
that's now determined by directly checking if the iterator itself
signifies the end of the table.
For example, trying to assign a vector, table, set, or record
constructor expression to a global variable of a different type now
provides a more explanatory error message than the previous
"Val::CONVERTER" fatal-error and coredump.
* origin/topic/vern/vector-holes:
Remove NEWS entry regarding changed vector-holes functionality
Fix potential segfaults in VectorVal Insert/Remove methods
Fix copy() to work with a vector that has trailing holes
update test suite for vector holes now being supported for numeric types
add vector tests for creating holes, "in" operator, "?" operator, copying vectors with holes
restore support for vectors with holes remove vestigial comment
fix using ++/-- to vectors that contain holes
Several limitations still apply:
- TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 is the only supported cipher suite
- Some tests are broken due to a failing assertion regarding bytestring
- No newly written tests for decryption (the patch was tested extensively for our paper)
- Several small open technical questions marked with FIXME
- Architecture in the ssl module might not be optimal
- Extended a btest to cover tables/sets with index types
(this was originally broken until fixed by GH-1514)
* origin/topic/jsiwek/gh-1506-fix-broker-func-indices:
GH-1506: Fix Broker unserialization of set/table function indices
* origin/topic/timw/1487-not-valid-enum:
Move an assert() in input/Manager.cc to account for ValueToVal errors
Add test for config framework
Fix similar issues with ValueTo* methods in the input framework
GH-1487: Handle error from ValueToVal instead of ignoring it
Zeek function types are serialized as a broker::vector, but the
unserialization logic for Zeek set/table types that use a function for
as an index incorrectly identified it as a composite-index, which also
use broker::vector, rather than a singleton-index, and makes such
unserialization fail.
A general example where this failure can happen in practice is when
trying to unserialize a connection record for which there's a
Conn::RemovalHook, since that's a set[function], and a specific case of
that is use of the Intel Framework in a Zeek cluster.
This allows for data that won't match a SIP request method to precede an
actual request and generates a new 'sip_junk_before_request' weird when
encountering such a situation.
Previously, incompatible &expire_funcs could mistakenly be used, such as
when using that attribute on the unspecified table()/set()
initializations/assignments, resulting in invalid function calls that
eventually crash Zeek.
And clarified the error message that it's more about finding an
unterminated pattern than knowing for sure there's remaining pattern
text spanning multiple lines.
Some libpcaps (observed in Myricom's) may claim to have read a packet,
but either did not really read a packet or at least provide no way
to access its contents, so this adds a check for null-data to
handle those cases.
The modp_dtoa/modp_dtoa2 functions aren't capable of handling double
values larger than INT_MAX and fallback on using sprintf() in that
situation. Previously, the format string to that sprintf() was "%e",
defaulting to a precision of 6, which is already too few digits to
represent a number known to be larger than INT_MAX. Now, an sprintf()
is still performed for values larger than INT_MAX and still uses a
scientific notation format, but in a way that uses as many decimal
digits as needed to preserve information.
- Minor adjustments to header includes, whitespace, and a
compiler warning fix during merge
* origin/topic/neverlord/gh-1408:
Add new Timer class to the telemetry API
Fix build with latest VectorVal API
Update baselines
Prefix telemetry BIFs with __
Expose telemetry API to scripting land
Add handle types for histogram metrics
Move duplicated code to detail header
Adhere to Zeek coding style
Apply suggestions from code review
Add telemetry Manager to Zeek setup / teardown
Add missing whitespaces for Zeek coding style
Add gauge metric types
Add scaffold for new metrics API with counters
