The main part of this commit are changes in tests. A lot of the tests
that previously relied on analyzer.log or dpd.log now use the new
analyzer-failed.log.
I verified all the changes and, as far as I can tell, everything
behaves as it should. This includes the external test baselines.
This change also enables logging of file and packet analyzer to
analyzer_failed.log and fixes some small behavior issues.
The analyzer_failed event is no longer raised when the removal of an
analyzer is vetoed.
If an analyzer is no longer active when an analyzer violation is raised,
currently the analyzer_failed event is raised. This can, e.g., happen
when an analyzer error happens at the very end of the connection. This
makes the behavior more similar to what happened in the past, and also
intuitively seems to make sense.
A bug introduced in the failed service logging was fixed.
In AWS GLB environments, the max_depth of 2 is easily reached due to packets
being encapsulated with GENEVE and VXLAN [1]. Any additional encapsulation
layer causes Zeek raise a weird and ignore the inner traffic. Bump the default
maximum depth to 4, while not common it's not unusual either to observe
this in the wild.
[1] https://docs.aws.amazon.com/vpc/latest/mirroring/traffic-mirroring-packet-formats.htmlCloses#3439
There's a logic error in the packet analyzer's AnalyzerConfirmation()
method that causes analyzer_confirmation() events to be raised for every
packet rather than stopping after the first confirmation which appears to
have been the intention. This affects, for example, VXLAN and Geneve tunnels.
The optional arg_tag parameter was used for short-circuit'ing, but the return
value of GetAnalyzerTag() used for setting the session state causing the
disconnect.
In scenarios where Zeek receives purely tunneled monitoring traffic, this may
result in a non-negligible performance impact.
Somewhat related, ensure the session state is set to violated before
short-circuiting if no analyzer_violations are installed.
Suggesting this as a 5.0.3 candidate.
Now that it's loaded in bare mode, no need to load it explicitly.
The main thing that tests were relying on seems to be tracking of
c$service for conn.log baselines. Very few were actually checking
for dpd.log
The change in 44f558df7b that made analyzer_list
a std::vector instead of a std::list doesn't take into account that in some
cases an analyzer may chain back into itself, such as with UDP-in-UDP tunnels.
In these cases, the second call to ForwardPacket may cause iterator
invalidation, leading to a crash.
- Use `-b` most everywhere, it will save time.
- Start some intel tests upon the input file being fully read instead of
at an arbitrary time.
- Improve termination condition for some sumstats/cluster tests.
- Filter uninteresting output from some supervisor tests.
- Test for `notice_policy.log` is no longer needed.
Previously, a single `icmp_conn` record was built per ICMP "connection"
and re-used for all events generated from it. This may have been a
historical attempt at performance optimization, but:
* By default, Zeek does not load any scripts that handle ICMP events.
* The one script Zeek ships with that does handle ICMP events,
"detect-traceroute", is already noted as being disabled due to
potential performance problems of doing that kind of analysis.
* Re-use of the original `icmp_conn` record tends to misreport
TTL and length values since they come from original packet instead
of the current one.
* Even if we chose to still re-use `icmp_conn` records and just fill
in a new TTL and length value each packet, a user script could have
stored a reference to the record and not be expecting those values
to be changed out from underneath them.
Now, a new `icmp_info` record is created/populated in all ICMP events
and should be used instead of `icmp_conn`. It also removes the
orig_h/resp_h fields as those are redundant with what's already
available in the connection record.
- Updated the logic significantly: still filters out ICMP from being
considered an active service (like before) and adds a new
"Known::service_udp_requires_response" option (defaults to true) for
whether to require UDP server response before being considered an
active service.
* 'topic/dopheide/known-services' of https://github.com/dopheide-esnet/zeek:
Log services with unknown protocols
This changes the decapsulation logic for GRE/ERSPAN payloads to re-use
existing Layer 2 parsing logic that already handles things like 802.1Q
tags correctly before going on to process the inner IPv4/IPv6 payload.
This also installs symlinks from "zeek" and "bro-config" to a wrapper
script that prints a deprecation warning.
The btests pass, but this is still WIP. broctl renaming is still
missing.
#239
* Better parsing/error-checking of VXLAN and encapsulated packet headers
* Add/implement the "vxlan_packet" event
* Add "Tunnel::vxlan_ports" option to tune the set of VXLAN ports to
analyze/decapsulate
* Add "Tunnel::validate_vxlan_checksums" option to allow for tuning of how
checksums associated with the outer UDP header of a possible VXLAN
tunnel are handled
Fixes GH-250
IP packets that have a header length that is greater than the total
length of the packet cause a integer overflow, which cause range-checks
to fail, which causes OOB reads.
Furthermore Bro does not currently check the version field of IP packets
that are read from tunnels. I added this check - otherwhise Bro reports
bogus IP information in its error messages, just converting the data
from the place where the IP information is supposed to be to IPs.
This behavior brings us closer to what other software (e.g. Wireshark)
displays in these cases.
This includes enhanced GRE headers. GRE tunnels are treated just like
IP-in-IP tunnels by parsing past the GRE header in between the delivery
and payload IP packets.
All tests pass with one exception: some Broxygen tests are broken
because dpd_config doesn't exist anymore. Need to update the mechanism
for auto-documenting well-known ports.
Added a generic gtpv1_message event generated for any GTP message type.
Added specific events for the create/update/delete PDP context
request/response messages.
Addresses #934.
This currently supports automatic decapsulation of GTP-U packets on
UDP port 2152.
The GTPv1 headers for such tunnels can be inspected by handling the
"gtpv1_g_pdu_packet" event, which has a parameter of type "gtpv1_hdr".
Analyzer and test cases are derived from submissions by Carsten Langer.
Addresses #690.
This option indicates that the Teredo analyzer should wait until
it sees both sides of a connection using a valid Teredo encapsulation
before issuing a protocol_confirmation. Previous behavior confirmed
on the first instance of a valid encapsulation, which could result
in more false positives (and e.g. bogus entries in known-services.log).
Addresses #890.
Teredo bubble packets (IPv6 w/ No Next Header and zero Payload Length)
with data extending past the inner IPv6 header (the outer IPv4 header's
Total Length and UDP header's Length indicate this) now only raises
a "Teredo_payload_len_mismatch" weird instead of causing a
ProtocolViolation().
This also fixes a crash in NetSessions::ParseIPPacket() that occurred
when the packet length didn't match the payload length field.
