* origin/topic/jsiwek/gh-554-file-signature-optimizations:
GH-554: don't init PIA endpoint matchers if there's only file-magic
GH-554: remove use of file magic in protocol-based signature logic
The logic for initializing PIA endpoint matchers was previously
skipped if "there's no global rule matcher", and that's only true
when no signature files get loaded.
But when using `zeek -b`, some file-magic signatures still get loaded
by default, so the PIA endpoint matchers still get initialized even
though they don't need to be -- file-magic patterns play no part
in PIA.
For typical use-cases (not using the `-b` flag), this change won't
help any, but we do at least use `-b` often within the test suite.
This can be a significant performance/memory improvement since
otherwise the protocol-based rule matching logic ends up superfluously
creating file-matching state per file-matcher per connection/endpoint.
- Fixed the context list padding to only be used for dialect 0x0311.
The new test case includes an example where parsing the optional
padding would fail for another dialect.
* '555-smb3-negotiate-context-fix' of https://github.com/mad/zeek:
Fix for smb3 negotiate context
- Minor whitespace and comment adjustments
* origin/topic/timw/mapping:
Fix unit tests for new ordering from NetSessions::Drain
Change FragReassembler to use a tuple as a key and use std::map for fragments in Sessions
Rework Session/Connection tracking to use a std::map instead of PDict
I copied the same style that caf uses ("zk" with single dot and no space).
This gives some consistency with caf and avoids us wasting more
space beyond "bro: ". OSs only give 16 characters for thread names
so anything we can gain here is nice.
If dpd_match_only_beginning is disabled, matches of protocol signatures
can be handeld using protocol_late_match. To prevent further matching in
this case, dpd_late_match_stop may be activated.
Includes fix for potential iterator invalidation during iteration.
* origin/topic/jsiwek/gh-532-improve-disable-analyzer:
GH-532: improve disable_analyzer BIF
* origin/topic/timw/deprecate-int-types:
Deprecate the internal int/uint types in favor of the cstdint types they were based on
Merge adjustments:
* A bpf type mistakenly got replaced (inside an unlikely #ifdef)
* Did a few substitutions that got missed (likely due to
pre-processing out of DEBUG macros)
* origin/topic/timw/easy-pdict-replacements:
Cleanups related to PDict -> std::map replacements
Remove other simple uses of PDict
Protocols: Remove uses of PDict
g_dbgfilemaps: Remove uses of PDict
Scope: remove uses of PDict
DFA: remove uses of PDict
EventRegistry: remove uses of PDict
* origin/topic/timw/clang-tidy:
Change over to whitelisting clang-tidy options instead of blacklisting
Use FindClangTidy in the cmake submodule to make things easier
* origin/topic/jazoff/memory-reduction-contentline:
Remove redundant buffering in contentline
Adjusted NVT analyzer during merge for cases where it may have been
assuming the buffer was already resized before DoDeliver().
* origin/topic/jazoff/memory-reduction-mime:
Avoid buffering all http headers
Adjusted during merge:
* whitespace
* logic for HTTP needs to consider whether either mime_all_headers
or http_all_headers events are being used
Using exit() here may generally not work well since:
* That will result in calling global destructors
* We have global state that we potentially modify at run-time and
are in the middle of modiying at the time the FatalError occurs.
E.g. out-of-memory is one situation where it's likely we could
call the dtor of an object in which operation on it's internal
state is no longer consistent/safe.
- Add an extra "prevent" parameter (default value of false), which
helps prevent the same analyzer type from being attached in the
future. It's useful in situations where you want to disable early
on, but a DPD signature may still trigger later and re-attach
the same analyzer. E.g. when not using this flag, but calling
disable_analyzer() inside an http_request event, will remove the
HTTP analyzer that was attached due to well-known-port, but a later
DPD signature match from upon seeing the HTTP reply will end up
attaching another HTTP analyzer. More surprising is that upon
re-attaching that analyzer, you'll get the same http_request as
before since the DPD buffer will get replayed into the new analyzer.
- Fixes disable_analyzer() to work when called even earlier, like
within the protocol_confirmation event. At that time, the
Analyzer tree may have not properly added the new analyzer into
Analyzer::children yet, but rather the temporary waiting list,
Analyzer::new_children. Analyzer::RemoveChildAnalyzer previously
did not inspect the later list.
- Fixes disable_analyzer() when called on an analyzer added to the
tree via TCP_Analyzer::AddChildPacketAnalyzer. TCP_Analyzer
keeps track of such children in its own list,
TCP_Analyzer::packet_children, which the previous
Analyzer::RemoveChildAnalyzer implementation didn't inspect.
Packet length is encoded in up to four bytes, with MSB (0x80)
indicating if there's more bytes in the representation still to follow.
The comparison/bitwise-mask wasn't correctly testing the MSB.
Coverity CID 1403964
