This also includes:
- Deprecating the NetSessions name.
- Renaming the zeek::sessions global to zeek::session_mgr and deprecating the old name.
- Renaming Sessions.{h,cc} to SessionManager.{h,cc}.
This commit also includes:
- Storing the transport protocol in ConnID and ConnIDKey to allow tcp and
udp connections from the same IP/Port combinations. This happens in the
core.cisco-fabric-path test, for example.
- Lots of test updates. The reasons for these are two fold. First, with
the change to only store a single map means that TCP, UDP, and ICMP
connections are now mixed. When Zeek drains the map at shutdown, it drains
each of those protocols together instead of separately. The second is
because of how Sessions are stored in the map. We're now storing them
keyed by the hash of the key stored by the Session objects, which causes
them to again be in the map in a different order.
Fixes to `decode_netbios_name`:
* Improve validation that input string is a NetBIOS encoding
(32 bytes, with characters ranging from 'A' to 'P'). This helps
prevent Undefined Behavior of left-shifting negative values.
Invalid encodings now cause a return-value of an empty string.
* More liberal in what decoded characters are allowed. Namely,
spaces are now allowed (but any trailing null-bytes and spaces
are trimmed, similar to before).
Fixes to `decode_netbios_name_type`:
* Improve validation that input string is a NetBIOS encoding
(32 bytes, with characters ranging from 'A' to 'P'). This helps
prevent Undefined Behavior of left-shifting negative values and
a heap-buffer-overread when the input string is too small.
Invalid encodings now cause a return-value of 256.
This patch adds the ability to decap Geneve packets to process the inner
payload. The structure of the analyzer borrows heavily from the VXLAN
analyzer.
This previously crashed since clear_table()/TableVal::RemoveAll() left
behind a stale iterator to the old table causing a heap-use-after-free
when resuming table expiry iteration in TableVal::DoExpire().
For example, trying to assign a vector, table, set, or record
constructor expression to a global variable of a different type now
provides a more explanatory error message than the previous
"Val::CONVERTER" fatal-error and coredump.
* origin/topic/vern/vector-holes:
Remove NEWS entry regarding changed vector-holes functionality
Fix potential segfaults in VectorVal Insert/Remove methods
Fix copy() to work with a vector that has trailing holes
update test suite for vector holes now being supported for numeric types
add vector tests for creating holes, "in" operator, "?" operator, copying vectors with holes
restore support for vectors with holes remove vestigial comment
fix using ++/-- to vectors that contain holes
Several limitations still apply:
- TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 is the only supported cipher suite
- Some tests are broken due to a failing assertion regarding bytestring
- No newly written tests for decryption (the patch was tested extensively for our paper)
- Several small open technical questions marked with FIXME
- Architecture in the ssl module might not be optimal
- Extended a btest to cover tables/sets with index types
(this was originally broken until fixed by GH-1514)
* origin/topic/jsiwek/gh-1506-fix-broker-func-indices:
GH-1506: Fix Broker unserialization of set/table function indices
* origin/topic/timw/1487-not-valid-enum:
Move an assert() in input/Manager.cc to account for ValueToVal errors
Add test for config framework
Fix similar issues with ValueTo* methods in the input framework
GH-1487: Handle error from ValueToVal instead of ignoring it
Zeek function types are serialized as a broker::vector, but the
unserialization logic for Zeek set/table types that use a function for
as an index incorrectly identified it as a composite-index, which also
use broker::vector, rather than a singleton-index, and makes such
unserialization fail.
A general example where this failure can happen in practice is when
trying to unserialize a connection record for which there's a
Conn::RemovalHook, since that's a set[function], and a specific case of
that is use of the Intel Framework in a Zeek cluster.
This allows for data that won't match a SIP request method to precede an
actual request and generates a new 'sip_junk_before_request' weird when
encountering such a situation.
Previously, incompatible &expire_funcs could mistakenly be used, such as
when using that attribute on the unspecified table()/set()
initializations/assignments, resulting in invalid function calls that
eventually crash Zeek.
And clarified the error message that it's more about finding an
unterminated pattern than knowing for sure there's remaining pattern
text spanning multiple lines.
The modp_dtoa/modp_dtoa2 functions aren't capable of handling double
values larger than INT_MAX and fallback on using sprintf() in that
situation. Previously, the format string to that sprintf() was "%e",
defaulting to a precision of 6, which is already too few digits to
represent a number known to be larger than INT_MAX. Now, an sprintf()
is still performed for values larger than INT_MAX and still uses a
scientific notation format, but in a way that uses as many decimal
digits as needed to preserve information.
- Minor adjustments to header includes, whitespace, and a
compiler warning fix during merge
* origin/topic/neverlord/gh-1408:
Add new Timer class to the telemetry API
Fix build with latest VectorVal API
Update baselines
Prefix telemetry BIFs with __
Expose telemetry API to scripting land
Add handle types for histogram metrics
Move duplicated code to detail header
Adhere to Zeek coding style
Apply suggestions from code review
Add telemetry Manager to Zeek setup / teardown
Add missing whitespaces for Zeek coding style
Add gauge metric types
Add scaffold for new metrics API with counters
* origin/topic/vern/global-stmts:
whitespace adjustment
"balance" tests with multiple Zeek scripts to load the same elements
put global statements into a quasi-function to support script optimization
The change in 44f558df7b that made analyzer_list
a std::vector instead of a std::list doesn't take into account that in some
cases an analyzer may chain back into itself, such as with UDP-in-UDP tunnels.
In these cases, the second call to ForwardPacket may cause iterator
invalidation, leading to a crash.
* origin/topic/vern/zval: (42 commits)
whitespace tweaks
resolved some TODO comments
remove unnecessary casts, and change necessary ones to use static_cast<>
explain cmp_func default
change functions for ZVal type management to static members
fix some unsigned/signed integer warnings
address lint concern about uninitialized variable
Remove use of obsolete forward-declaration macros
fix #include's that lack zeek/ prefixes
explicitly populate holes created in vectors
fixes for now-incorrect assumption that GetField always returns an existing ValPtr
memory management for assignment to vector elements
memory management for assignment to record fields
destructor cleanup from ZAM_vector/ZAM_record
fix #include's that lack zeek/ prefixes
overlooked another way in which vector holes can be created
initialize vector holes to the correct corresponding type
explicitly populate holes created in vectors
fix other instances of GetField().get() assuming long-lived ValPtr's
fix for now-incorrect assumption that GetField always returns an existing ValPtr
...
- Removed a couple of dead statements during merge
* origin/topic/vern/cpp-new-func:
option for deterministic descriptions of sets & tables
determinism for concurrent Zeek test suite invocations; split out deprecations
disambiguate descriptions of enums; include attributes when describing records
more liberal view of attribute equality; allow suppressing attr type-checking
support for operations on sets that return new values
low-level addition of enum values
sundry accessors/cast-ers; RE_Matcher's track their construction values
convenience functions for comparing IP addresses
