* origin/topic/seth/624:
Support whitespace at end of line for config reader.
This merge fixes a failing test; it also sprinkles a few more spaces
into another test file.
The main change is that this now also works with configuration lines
that don't have a value.
* origin/topic/timw/595-json-perf:
Update COPYING.3rdparty
Use json::emplace to avoid some extra calls to operator[]
Use tessil/unordered-map instead of nlohmann/fifo-map to mitigate performance issues when logging JSON
When encrypted data was bundled within the same segment as the NewKeys
message, it wasn't not reported via a ssh_encrypted_package event as
it should have been.
- Fixed the context list padding to only be used for dialect 0x0311.
The new test case includes an example where parsing the optional
padding would fail for another dialect.
* '555-smb3-negotiate-context-fix' of https://github.com/mad/zeek:
Fix for smb3 negotiate context
* 'known_services_multiprotocols' of https://github.com/mauropalumbo75/zeek:
improve logging with broker store
drop services starting with -
remove service from key for Cluster::publish_hrw
remove check for empty services
update tests
order list of services in store key
remove repeated services in logs if already seen
add multiprotocol known_services when Known::use_service_store = T
remove hyphen in front of some services (for example -HTTP, -SSL) In some cases, there is an hyphen before the protocol name in the field connection$service. This can cause problems in known_services and is removed here. It originates probably in some analyzer where it would be better removed in the future.
add multiprotocol known_services when Known::use_service_store = F
Changes during merge:
* whitespace
* add unit test
This caps size of payload strings within mqtt_publish events and
mqtt_publish.log files. A new "payload_len" field in the log file
shows the real payload size in cases where it may have been truncated.
This also required updating a test that required a root-certificate that
was removed from the Mozilla store - the test now directly includes that
specific root-cert.
Now, in addition to setting thresholds for bytes and packet, one can set
a threshold for connection duration. Note that the threshold event is
only raised once the next packet in the connection is seen.
This also fixes a small pre-existing bug, in which a bunch of warnings
were raised if someone just used the lower-level functions without going
through the higher-level scripting API.
* origin/topic/seth/mqtt:
Bug fixes and test baseline updates
Fix an issue with bro_init -> zeek_init
MQTT Analyzer heavily updated and ported from the analyzer originally by Supriya Kumar
Adjustments during merge:
* Minor whitespace cleanups
* Some bro to zeek renaming
* Fixed the parsing of unsubscribe messages to generate an event for each topic
* origin/topic/dev/non-ascii-logging:
Removed Policy Script for UTF-8 Logs
Commented out UTF-8 Script in Test All Policy
Minor Style Tweak
Use getNumBytesForUTF8 method to determine number of bytes
Added Jon's test cases as unit tests
Prioritizes escaping predefined Escape Sequences over Unescaping UTF-8 Sequences
Added additional check to confirm anything unescaping is a multibyte UTF-8 sequence, addressing the test case Jon brought up
Added optional script and redef bool to enable utf-8 in ASCII logs
Initial Commit, removed std::isprint check to escape
Made minor code format and logic adjustments during merge.
This analyzer generates three logs to fully display what is happening over the MQTT connection.
- mqtt_connect.log
- mqtt_subscribe.log
- mqtt_publish.log
At this time it only supports MQTT 3.1 and 3.1.1
* origin/topic/timw/cleaner-utf8:
GHI-486: Switch over to using LLVM utf8-checking code to better validate characters
I addressed a buffer over-read during the merge and added test-cases for
it.
This allows one to tune the number of protocol violations to tolerate
from any given analyzer type before just disabling a given instance
of it.
Also removes the "disabled_aids" field from the DPD::Info record
since it serves no purpose: in this case, calling disable_analyzer
multiple times for the same analyzer is a no-op.
* origin/topic/timw/150-to-json:
Update submodules for JSON work
Update unit tests for JSON logger to match new output
Modify JSON log writer to use the external JSON library
Update unit test output to match json.zeek being deprecated and slight format changes to JSON output
Add proper JSON serialization via C++, deprecate json.zeek
Add new method for escaping UTF8 strings for JSON output
Move do_sub method from zeek.bif to StringVal class method
Move record_fields method from zeek.bif to Val class method
Add ToStdString method for StringVal
