Changes during merge
- Changed the policy script to use an event handler that behaves
for like the base script: &priority=5, msg$opcode != early-out,
no record field existence checks
- Also extended dns_query_reply event with original_query param
- Removed ExtractName overload, and just use default param
* 'dns-original-query-case' of https://github.com/rvictory/zeek:
Fixed some places where tabs became spaces
Stricter checking if we have a dns field on the connection being processed
Modified the DNS protocol analyzer to add a new parameter to the dns_request event which includes the DNS query in its original case. Added a policy script that will add the original_case to the dns.log file as well. Created new btests to test both.
- Updated the logic significantly: still filters out ICMP from being
considered an active service (like before) and adds a new
"Known::service_udp_requires_response" option (defaults to true) for
whether to require UDP server response before being considered an
active service.
* 'topic/dopheide/known-services' of https://github.com/dopheide-esnet/zeek:
Log services with unknown protocols
* 'master' of https://github.com/mmguero-dev/zeek:
check for the existance of f?$conns in file_sniff event in policy/protocols/ssl/log-hostcerts-only.zeek
In using the corelight/bro-xor-exe-plugin (https://github.com/corelight/bro-xor-exe-plugin) I noticed this error when running the PCAP trace file in its tests directory:
1428602842.525435 expression error in /opt/zeek/share/zeek/policy/protocols/ssl/log-hostcerts-only.zeek, line 44: field value missing (X509::f$conns)
Examining log-hostcerts-only.zeek, I saw that although f$conns is being checked for length, it's not being checked to see if it exists first.
This commit changes "if ( |f$conns| != 1 )" to "if (( ! f?$conns ) || ( |f$conns| != 1 ))" so that the script returns if there is no f$conns field.
In my local testing, this seems to fix the error. My testing was being done with v3.0.5, but I think this patch can be applied to both the 3.0.x and 3.1.x branches.
- All timers are now handled by a single global timer manager, which simplifies how they handled by the IOSource manager.
- This change flows down a number of changes to other parts of the code. The timer manager tag field is removed, which means that matching connections to a timer manager is also removed. This removes the ability to tag a connection as internal or external, since that's how the connections where differentiated. This in turn removes the `current_conns_extern` field from the `ConnStats` record type in the script layer.
Typically in base scripts, Log::create_stream() is called in zeek_init()
handler with &priority=5 such that it will have already been created
in the default zeek_init() &priority=0.
And switch Zeek's base scripts over to using it in place of
"connection_state_remove". The difference between the two is
that "connection_state_remove" is raised for all events while
"successful_connection_remove" excludes TCP connections that were never
established (just SYN packets). There can be performance benefits
to this change for some use-cases.
There's also a new event called ``connection_successful`` and a new
``connection`` record field named "successful" to help indicate this new
property of connections.
* 'known_services_multiprotocols' of https://github.com/mauropalumbo75/zeek:
improve logging with broker store
drop services starting with -
remove service from key for Cluster::publish_hrw
remove check for empty services
update tests
order list of services in store key
remove repeated services in logs if already seen
add multiprotocol known_services when Known::use_service_store = T
remove hyphen in front of some services (for example -HTTP, -SSL) In some cases, there is an hyphen before the protocol name in the field connection$service. This can cause problems in known_services and is removed here. It originates probably in some analyzer where it would be better removed in the future.
add multiprotocol known_services when Known::use_service_store = F
Changes during merge:
* whitespace
* add unit test
* 'empty_services' of https://github.com/mauropalumbo75/zeek:
remove empty services and include udp active connections when logging in connection_state_remove
I.e. a type was not in the export section, but a field was added
to connection record via a redef that uses the "hidden" type.
That generally doesn't help to hide it that way since a user comes
to rely on it indirectly anyway, and it also causes problems with
the Zeekygen documentation not being able to find it.
In some cases, there is an hyphen before the protocol name in the field
connection$service. This can cause problems in known_services and
is removed here. It originates probably in some analyzer where it
would be better removed in the future.
This commit removed functions/events that have been deprecated in Bro
2.6. It also removes the detection code that checks if the old
communication framework is used (since all the functions that are
checked were removed).
Addresses parts of GH-243
* All "Broxygen" usages have been replaced in
code, documentation, filenames, etc.
* Sphinx roles/directives like ":bro:see" are now ":zeek:see"
* The "--broxygen" command-line option is now "--zeexygen"
