This largely copies over Spicy's `.clang-format` configuration file. The
one place where we deviate is header include order since Zeek depends on
headers being included in a certain order.
This also includes:
- Deprecating the NetSessions name.
- Renaming the zeek::sessions global to zeek::session_mgr and deprecating the old name.
- Renaming Sessions.{h,cc} to SessionManager.{h,cc}.
The Zeek code base has very inconsistent #includes. Many sources
included a few headers, and those headers included other headers, and
in the end, nearly everything is included everywhere, so missing
#includes were never noticed. Another side effect was a lot of header
bloat which slows down the build.
First step to fix it: in each source file, its own header should be
included first to verify that each header's includes are correct, and
none is missing.
After adding the missing #includes, I replaced lots of #includes
inside headers with class forward declarations. In most headers,
object pointers are never referenced, so declaring the function
prototypes with forward-declared classes is just fine.
This patch speeds up the build by 19%, because each compilation unit
gets smaller. Here are the "time" numbers for a fresh build (with a
warm page cache but without ccache):
Before this patch:
3144.94user 161.63system 3:02.87elapsed 1808%CPU (0avgtext+0avgdata 2168608maxresident)k
760inputs+12008400outputs (1511major+57747204minor)pagefaults 0swaps
After this patch:
2565.17user 141.83system 2:25.46elapsed 1860%CPU (0avgtext+0avgdata 1489076maxresident)k
72576inputs+9130920outputs (1667major+49400430minor)pagefaults 0swaps
Started by factoring some details into a new DataBlockList class to at
least make it more clear where modifications occur. More abstractions
likely to happen later as I experiment with alternate data structures
aimed at improving worse-case scenarios.
This also installs symlinks from "zeek" and "bro-config" to a wrapper
script that prints a deprecation warning.
The btests pass, but this is still WIP. broctl renaming is still
missing.
#239
- Removed the gap_report event. It wasn't used anymore
and functionally no more capable that scheduling events
and using the get_gap_summary bif.
- Added functionality to Dictionaries to count cumulative
numbers of inserts performed. This is further used to
measure the total number of connections of various types.
Previously only the number of active connections was
available.
- The Reassembler base class now tracks active reassembly
size for all subclasses (File/TCP/Frag & unknown).
- Improvements to the stats.log. Mostly, more information.
The main change is that reassembly code (e.g. for TCP) now uses
int64/uint64 (signedness is situational) data types in place of int
types in order to support delivering data to analyzers that pass 2GB
thresholds. There's also changes in logic that accompany the change in
data types, e.g. to fix TCP sequence space arithmetic inconsistencies.
Another significant change is in the Analyzer API: the *Packet and
*Undelivered methods now use a uint64 in place of an int for the
relative sequence space offset parameter.
Mem leaks and a null pointer deref (not actually expected to occur
unless already in one of the the odd states that warrants an internal
warning/error).
Also dead code.
Replaced some with InternalWarning or InternalAnalyzerError, the later
being a new method which signals the analyzer to not process further
input. Some usages I just removed if they didn't make sense or clearly
couldn't happen. Also did some minor refactors of related code while
reviewing/exploring ways to get rid of InternalError usages.
Also, for TCP content file write failures there's a new event:
"contents_file_write_failure".
- Add more guards against trying to analyze captured packets with a
truncated IPv6 static header or extension header chain.
- Add back in the ICMP payload tracking for ICMP "connections".
- Fix 'icmp_context' record construction. Some field assignments
were mismatched for ICMP and ICMP6. Source and destination
addresses were set incorrectly for context packets that don't
contain a full IP header. Some fields for ICMP6 weren't filled out.
- Changed ICMP Time Exceeded packets to raise the 'icmp_time_exceeded'
event instead of 'icmp_error_message'.
- Add unit tests for truncation and the main types of ICMP/ICMP6
that have specific events.
- Documentation clarifications.
* origin/topic/jsiwek/mobile-ipv6:
Add support for mobile IPv6 Mobility Header (RFC 6275).
Refactor IP_Hdr routing header handling, add MobileIPv6 Home Address handling.
Revert TCP checksumming to cache common data, like it did before.
Revert "Improve handling of IPv6 Routing Type 0 headers."
Improve handling of IPv6 routing type 0 extension headers.
The FragReassembler expire_timer was left uninitialized until after
the first fragment is added, but since the atomic fragment is also
the last, the reassembler thought expire_timer needed to be deleted.
This fix just initializes expire_timer before the first fragment is
added.
- flow_weird event with name argument value of "routing0_hdr" is raised
for packets containing an IPv6 routing type 0 header because this
type of header is now deprecated according to RFC 5095.
- packets with a routing type 0 header and non-zero segments left
now use the last address in that header in order to associate
with a connection/flow and for calculating TCP/UDP checksums.
- added a set of IPv4/IPv6 TCP/UDP checksum unit tests
In response to feedback from Robin:
- rename "ip_hdr" to "ip4_hdr"
- pkt_hdr$ip6 is now of type "ip6_hdr" instead of "ip6_hdr_chain"
- "ip6_hdr_chain" no longer contains an "ip6_hdr" field, instead
it's the other way around, "ip6_hdr" contains an "ip6_hdr_chain"
- other internal refactoring
* origin/topic/jsiwek/ipv6-ext-headers:
Update PacketFilter/Discarder code for IP version independence.
Add a few comments to IP.h
Fix some IPv6 header related bugs.
Add IPv6 fragment reassembly.
Add handling for IPv6 extension header chains (addresses #531)
