And switch Zeek's base scripts over to using it in place of
"connection_state_remove". The difference between the two is
that "connection_state_remove" is raised for all events while
"successful_connection_remove" excludes TCP connections that were never
established (just SYN packets). There can be performance benefits
to this change for some use-cases.
There's also a new event called ``connection_successful`` and a new
``connection`` record field named "successful" to help indicate this new
property of connections.
* origin/topic/robin/gh-239:
Undo a change to btest.cfg from a recent commit
Updating submodule.
Fix zeek-wrapper
Update for renaming BroControl to ZeekControl.
Updating submodule.
GH-239: Rename bro to zeek, bro-config to zeek-config, and bro-path-dev to zeek-path-dev.
This also installs symlinks from "zeek" and "bro-config" to a wrapper
script that prints a deprecation warning.
The btests pass, but this is still WIP. broctl renaming is still
missing.
#239
Added ConnectionEventFast() and QueueEventFast() methods to avoid
redundant event handler existence checks.
It's common practice for caller to already check for event handler
existence before doing all the work of constructing the arguments, so
it's desirable to not have to check for existence again.
E.g. going through ConnectionEvent() means 3 existence checks:
one you do yourself before calling it, one in ConnectionEvent(), and then
another in QueueEvent().
The existence check itself can be more than a few operations sometimes
as it needs to check a few flags that determine if it's enabled, has
a local body, or has any remote receivers in the old comm. system or
has been flagged as something to publish in the new comm. system.
Majority of PLists are now created as automatic/stack objects,
rather than on heap and initialized either with the known-capacity
reserved upfront or directly from an initializer_list (so there's no
wasted slack in the memory that gets allocated for lists containing
a fixed/known number of elements).
Added versions of the ConnectionEvent/QueueEvent methods that take
a val_list by value.
Added a move ctor/assign-operator to Plists to allow passing them
around without having to copy the underlying array of pointers.
This changes many weird names to move non-static content from the
weird name into the "addl" field to help ensure the total number of
weird names is reasonably bounded. Note the net_weird and flow_weird
events do not have an "addl" parameter, so information may no longer
be available in those cases -- to make it available again we'd need
to either (1) define new events that contain such a parameter, or
(2) change net_weird/flow_weird event signature (which is a breaking
change for user-code at the moment).
Also, the generic handling of binpac exceptions for analyzers which
to not otherwise catch and handle them has been changed from a Weird
to a ProtocolViolation.
Finally, a new "file_weird" event has been added for reporting
weirdness found during file analysis.
* origin/topic/jsiwek/val_mgr:
Pre-allocate and re-use Vals for bool, int, count, enum and empty string
Preallocate booleans and small counts
I added a tiny change to CompHash to make sure that nothing messes this
up in the future.
Added check for the optional sub-header in ERSPAN Type III as well
as additional truncation checks to the GRE parsing logic in general.
Also added a unit test for ERSPAN Type II.
/home/johanna/bro/master/src/Sessions.cc: In member function 'void NetSessions::DoNextPacket(double, const Packet*, const IP_Hdr*, const EncapsulationStack*)':
/home/johanna/bro/master/src/Sessions.cc:343:18: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
if ( ip_hdr_len > len )
~~~~~~~~~~~^~~~~
/home/johanna/bro/master/src/Sessions.cc:349:18: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
if ( ip_hdr_len > caplen )
~~~~~~~~~~~^~~~~~~~
/home/johanna/bro/master/src/Sessions.cc:399:20: warning: comparison between signed and unsigned integer expressions [-Wsign-compare]
if ( ip_hdr_len > len )
~~~~~~~~~~~^~~~~
In file included from /usr/include/machine/endian.h:6:0,
from /usr/include/sys/types.h:44,
from /usr/include/unistd.h:37,
from /home/johanna/bro/master/src/Anon.cc:2:
/home/johanna/bro/master/src/Anon.cc: In member function 'virtual ipaddr32_t AnonymizeIPAddr_Seq::anonymize(ipaddr32_t)':
/home/johanna/bro/master/src/Anon.cc:85:18: warning: operation on '((AnonymizeIPAddr_Seq*)this)->AnonymizeIPAddr_Seq::seq' may be undefined [-Wsequence-point]
return htonl(seq++);
^
IP packets that have a header length that is greater than the total
length of the packet cause a integer overflow, which cause range-checks
to fail, which causes OOB reads.
Furthermore Bro does not currently check the version field of IP packets
that are read from tunnels. I added this check - otherwhise Bro reports
bogus IP information in its error messages, just converting the data
from the place where the IP information is supposed to be to IPs.
This behavior brings us closer to what other software (e.g. Wireshark)
displays in these cases.
This is a small caveat to this implementation. The ethernet
header that is carried over the tunnel is ignored. If a user
tries to do MAC address logging, it will only show the MAC
addresses for the outer tunnel and the inner MAC addresses
will be stripped and not available anywhere.
If connection flipping occured in Sessions.cc code (invoked e.g. when
the original SYN is missing), layer 2 flipping was not performed. This
change switches to always use the connection flipping code in Conn.cc
which performs the switch correctly.
The definition of a "struct pcap_pkthdr" on OpenBSD contains a member
of type "struct bpf_timeval" instead of "struct timeval" used on other
systems.
Also, on OpenBSD the header netinet/if_ether.h does not #include
net/if_arp.h as it does on other systems.
Broke out the stats collection into a bunch of new Bifs
in stats.bif. Scripts that use stats collection functions
have also been updated. More work to do.
- Removed the gap_report event. It wasn't used anymore
and functionally no more capable that scheduling events
and using the get_gap_summary bif.
- Added functionality to Dictionaries to count cumulative
numbers of inserts performed. This is further used to
measure the total number of connections of various types.
Previously only the number of active connections was
available.
- The Reassembler base class now tracks active reassembly
size for all subclasses (File/TCP/Frag & unknown).
- Improvements to the stats.log. Mostly, more information.
That way it can be reused more easily. This also avoid having to
change the serialization structure for packets, which is a problem as
external sources of packets (via Broccoli) wouldn't have the new
attributes available to send.
Also moving Packet.{h,cc} and Layer2.{h,cc} into iosource/, and
removing header size from properties that packet sources have to
provide, as we can now compute that easily from the link type.
Plus some more cleanup.
* 'topic/rework-packets' of https://github.com/jsbarber/bro:
One more tinker to Packet -- ensure no uninitialized values
Packet::IP()-created IP_Hdr should not free
Make enums work for non-C++11 config
Refactor to make bro use a common Packet object. Do a better job of parsing layer 2 and keeping track of layer 3 proto. Add support for raw packet event, including Layer2 headers.
Conflicts:
aux/plugins
