Mostly rewrote the parsing logic to support incremental parsing and
to support parsing of client messages. Though I did not add events
for client messages, that's easy to add later.
Parsing now stops for both client and server if either encounters
any parsing error or invalid state.
After a complete handshake, server messages are no longer parsed.
Support for that is incomplete and not sure it's that useful anyway
since it mostly contains pixel data.
This changes many weird names to move non-static content from the
weird name into the "addl" field to help ensure the total number of
weird names is reasonably bounded. Note the net_weird and flow_weird
events do not have an "addl" parameter, so information may no longer
be available in those cases -- to make it available again we'd need
to either (1) define new events that contain such a parameter, or
(2) change net_weird/flow_weird event signature (which is a breaking
change for user-code at the moment).
Also, the generic handling of binpac exceptions for analyzers which
to not otherwise catch and handle them has been changed from a Weird
to a ProtocolViolation.
Finally, a new "file_weird" event has been added for reporting
weirdness found during file analysis.
* 'smb3-negotiate-response' of https://github.com/mauropalumbo75/zeek:
added test and pcap files for smb 3.1.1 negotiate-response
smb3.1.1 additions to negotiate-response command
I made several modifications:
- Code format, style, naming changes
- For completeness/correctness, I added parsing support for the remaining
context type structures.
- Moved the optional padding before the NegotiateContextList field to
also require the 0x0311 dialect version (some failures in
pre-existing unit tests pointed this out as an issue)
* 'smb3-transform-header' of https://github.com/mauropalumbo75/zeek:
clean up, test and pcap for transform_header added
added smb2-com-transform-header for smb3.x
Essentially, it will now process/parse priority values if they are
there, or else just accept whatever remaining data/text is there as the
syslog message. Reasoning is that there's syslog producers out there
that may have simply forgotten/neglected to send the priority value
and technically won't conform to what the standard says, though we can
infer the intent (some syslog consumers already may do similarly, but
I didn't verify).
The "orig_fuids", "orig_filenames", "orig_mime_types" http.log fields as
well as their "resp" counterparts are now limited to having
"HTTP::max_files_orig" or "HTTP::max_files_resp" entries, which are 15
by default. The limit can also be ignored case-by-case via the
"HTTP::max_files_policy" hook.
Fixes GH-289
* origin/master: (23 commits)
Improve Travis script to show multiple core dump stacks
Improve a weird stats unit test
Updating submodule(s).
Fix Travis script typo
Add more debug output to Travis script
Add core file search and stack trace output for Travis builds
Updating submodule(s).
Update license year for 2018
GH-199: change `bro --help` exit status from 1 to 0
Updating submodule(s).
Add a test with an encrypted MySQL connection
Fix parsing of MySQL NUL Strings, where we now require it to have a NUL value at the end.
Updating submodule(s).
Add missing record field comment
Add missing record field comments
Fix minor documentation mistakes
Updating submodule(s).
Updating submodule(s).
Update btest baselines for fix in 46f727a6fa
Generate ssh_auth_attempted for the 'none' authentication method.
...
* 'fatemabw/bro' of https://github.com/fatemabw/bro:
DNSSEC support in Bro
I made several changes:
- renamed event/record types
- reformatted the info added to dns.log
- removed the "addl" scripts that added extended dnssec info to dns.log
- simplifications/improvements to the internal parsing logic
This adds previously-missing support for "Alter Context"
request/response PDUs (initial patch contributed by Mark Fernandez).
Also, context ID arguments were added to dce_rpc_bind, dce_rpc_request,
and dce_rpc_response in order to properly track what endpoint/operation
a given opnum maps to.
This field isn't publicly available via the OpenSSL 1.1 API, not used
in the base scripts, and has no example in the test suit, so removing
it is simpler than trying to support manually parsing it out of the
raw data.
When encountering an extension unknown to OpenSSL, we would read from
the wrong memory location. Also added a testcase to prevent this specific
case from happening again.
It was possibly never updated for newer Analyzer API changes, as simply
attaching the NCP analyzer to a connection would result in null pointer
derefernces and also support analyzers were not attached.
* origin/topic/jsiwek/binpac-fixes:
Update test baseline for binpac changes
Update test baseline for optimized binpac static-size array parsing
Fixes for MySQL and SMB protocol parsers
BIT-1829: add unit test for modbus parser issue
* MySQL: the parser for this was generally broken (not following
the specification well) and needed many changes. One addition is a
new "mysql_result_row" event that provides access to the results of
queries.
* SMB: the spec seems to explitly call out the omission of the
PrimaryDomain field on SMB_COM_SESSION_SETUP_ANDX responses (and I
don't see that field in pcaps either), so this may have just been a
typo that used to work fine in the past only due to faulty array
parsing behavior in binpac.
* origin/topic/seth/dhcp-update:
Rework to the DHCP analyzer.
First step of DHCP analyzer rearchitecture.
Add .btest scripts for dhck_ack and dhcp_discover messages verifying that new options are correctly reported in dhcp.log records.
Extend DHCP protocol analyzer with new options.
BIT-1924 #merged
Additional changes:
* Removed known-hosts.bro as the only thing populating its table was
the already-removed known-hosts-and-devices.bro. So a
known_devices.log will no longer be generated.
* In dhcp-options.pac, the process_relay_agent_inf_option had a memleak
and also process_auto_proxy_config_option looked like it accessed one
byte past the end of the available bytestring, so fixed those.
* The altered Modbus checks seemed overly strict -- the pcap used
for the unit test at least had quantities/byte_count fields of
zero, to which the server responds with an error (expected).
* Most of the altered DNP3 checks seemed overly strict and caused
the unit tests to fail. The one that was just wrong was the
'start' field in header blocks.
* Removed the "start" parameter of the dnp3_header_block event
since it's always the same value.
* The SMB check failed to compile and I don't know what it intended
to do, so removed.
The way in which TLS 1.3 is negotiated was changed slightly in later
revisions of the standard. The final version is only sent in an
extension - while the version field in the server hello still shows TLS
1.2.
This patch makes ssl.log show the correct version again.
Highlights:
- Reduced all DHCP events into a single dhcp_message event. (removed legacy events since they weren't widely used anyway)
- Support many more DHCP options.
- DHCP log is completely reworked and now represents DHCP sessions
based on the transaction ID (and works on clusters).
- Removed the known-devices-and-hostnames script since it's generally
less relevant now with the updated log.
Closes BIT-1897.
* origin/topic/johanna/ssl_signature_details:
Make parsing of ServerKeyExchange work for D(TLS) < 1.2.
Add more details to ssl_server_signature.
* 'smb-transaction-messages' of https://github.com/jbencteux/bro:
add test for smb1_com_transaction_response event changes
add test for smb1_com_transaction2_secondary_request event changes
add test for smb1_com_transaction2_request event changes
add test for smb1_com_transaction_secondary_request event changes
add test for smb1_com_transaction_request event changes
fix setup field handling in smb1_com_transaction_request messages
fix smb1_com_transaction* messages
add smb1_transaction2_secondary_request event
add smb1_transaction_secondary_request event
add parameters and data to smb1_transaction_request/response messages
add SMB_Parameters.Words to smb1_transaction2_request event
