* topic/robin/http-connect:
HTTP fix for output handlers.
Expanding the HTTP methods used in the signature to detect HTTP traffic.
Updating submodule(s).
Fixing removal of support analyzers, plus some tweaking and cleanup of CONNECT code.
HTTP CONNECT proxy support.
BIT-1132 #merged
chain. This means that it is now possible to get information about the
root-certificates that were used to secure a connection.
Intermediate commit before changing the script interface again.
addresses BIT-953, BIT-760
CONNECT code.
Removal of support analyzers was broken. The code now actually doesn't
delete them immediately anymore but instead just flags them as
disabled. They'll be destroyed with the parent analyzer later.
Also includes a new leak tests exercising the CONNECT code.
Lines starting # with '#' will be ignored, and an empty message aborts
the commit. # On branch topic/robin/http-connect # Changes to be
committed: # modified: scripts/base/protocols/http/main.bro #
modified: scripts/base/protocols/ssl/consts.bro # modified:
src/analyzer/Analyzer.cc # modified: src/analyzer/Analyzer.h #
modified: src/analyzer/protocol/http/HTTP.cc # new file:
testing/btest/core/leaks/http-connect.bro # modified:
testing/btest/scripts/base/protocols/http/http-connect.bro # #
Untracked files: # .tags # changes.txt # conn.log # debug.log # diff #
mpls-in-vlan.patch # newfile.pcap # packet_filter.log # reporter.log #
src/PktSrc.cc.orig # weird.log #
This changes the internal type that is used to signal that a vector
is unspecified from any to void.
I tried to verify that the behavior of Bro is still the same. After
a lot of playing around, I think everything still should worl as before.
However, it might be good for someone to take a look at this.
addresses BIT-1144
This supports parsing of SNMPv1 (RFC 1157), SNMPv2 (RFC 1901/3416), and
SNMPv2 (RFC 3412). An event is raised for each SNMP PDU type, though
there's not currently any event handlers for them and not a default
snmp.log either. However, simple presence of SNMP is currently visible
now in conn.log service field and known_services.log.
The amount of timers involved in DNS::PendingMessage tables'
expiration attributes have a significant performance hit. Instead the
script now relies solely on maximum thresholds for pending message
quantities to limit amount of accumulated state. There's a new option,
"DNS::max_pending_query_ids", to limit the number outstanding messages
across all DNS query IDs ("DNS::max_pending_msgs" still limits number
of outstanding messages for a *given* query ID).
* origin/topic/jsiwek/dns-improvements:
Rewrite DNS state tracking which matches queries and replies.
Change dns.log to include only standard DNS queries.
Improve DNS analysis.
* origin/topic/jsiwek/tcp-improvements:
Add script to detect filtered TCP traces, addresses BIT-1119.
BIT-1119 #merged
If we could only disable an event handler dynamically ...
If reading a trace file w/ only TCP control packets, a warning is
emitted to suggest the 'detect_filtered_traces' option if the user
doesn't desire Bro to report missing TCP segments for such a trace file.
The previous method of matching queries with replies was still
unreliable in cases where the reply contains no answers. The new code
also takes extra measures to avoid pending state growing too large in
cases where the condition to match a query with a corresponding reply is
never met, but yet DNS messages continue to be exchanged over the same
connection 5-tuple (preventing cleanup of the pending state).
* origin/topic/jsiwek/tcp-improvements:
Fix file_over_new_connection event to trigger when entire file is missed.
Improve TCP connection size reporting for half-open connections.
Improve gap reporting in TCP connections that never see data.
Improve TCP FIN retransmission handling.
BIT-1119
The scope of dns.log is now only standard queries (OPCODE == 0). Other
kinds of queries (e.g. inverse query) were not handled correctly and
could interfere with the state tracking of the default DNS scripts.
- Fix parsing of empty question sections (when QDCOUNT == 0). In this
case, the DNS parser would extract two 2-byte fields for use in either
"dns_query_reply" or "dns_rejected" events (dependent on value of
RCODE) as qclass and qtype parameters. This is not correct, because
such fields don't actually exist in the DNS message format when
QDCOUNT is 0. As a result, these events are no longer raised when
there's an empty question section. Scripts that depends on checking
for an empty question section can do that in the "dns_message" event.
- Add a new "dns_unknown_reply" event, for when Bro does not know how
to fully parse a particular resource record type. This helps fix a
problem in the default DNS scripts where the logic to complete
request-reply pair matching doesn't work because it's waiting on more
RR events to complete the reply. i.e. it expects ANCOUNT number of
dns_*_reply events and will wait until it gets that many before
completing a request-reply pair and logging it to dns.log. This could
cause bogus replies to match a previous request if they happen to
share a DNS transaction ID.
The event now really returns the extension. If openssl supports printing
it, it is converted into the openssl ascii output.
The output does not always look pretty because it can contain newlines.
New event syntax:
event x509_extension(c: connection, is_orig: bool, cert:X509, extension: X509_extension_info)
Example output for extension:
[name=X509v3 Extended Key Usage,
short_name=extendedKeyUsage,
oid=2.5.29.37,
critical=F,
value=TLS Web Server Authentication, TLS Web Client Authentication]
[name=X509v3 Certificate Policies,
short_name=certificatePolicies,
oid=2.5.29.32,
critical=F,
value=Policy: 1.3.6.1.4.1.6449.1.2.1.3.4^J CPS: https://secure.comodo.com/CPS^J]
The previous behavior was to accomodate SYN/FIN/RST-filtered traces by
not reporting missing data (via the content_gap event) for such
connections. The new behavior always reports gaps for connections that
are established and terminate normally, but sequence numbers indicate
that all data packets of the connection were missed. The behavior can
be reverted by redef'ing "detect_filtered_trace".
* origin/topic/jazoff/notice_file_info:
Include file information in notices
BIT-1113 #merged
I'm wondering if there's a way to test creating these emails ... ?
This includes enhanced GRE headers. GRE tunnels are treated just like
IP-in-IP tunnels by parsing past the GRE header in between the delivery
and payload IP packets.
- The reassembly behavior can be modified per-file by enabling or
disabling the reassembler and/or modifying the size of the reassembly
buffer.
- Changed the file extraction analyzer to use the stream to avoid
issues with the chunk based approach not immediately triggering
the file_new event due to mime-type detection delay. Early chunks
frequently ended up lost before.
- Generally things are working now and I'd consider this in testing.
Instead of storing the entire notice in Notice::suppressing,
just store the time the notice should be suppressed until.
This has the same functionality, except that end_suppression can no
longer be generated.
