This largely copies over Spicy's `.clang-format` configuration file. The
one place where we deviate is header include order since Zeek depends on
headers being included in a certain order.
On Linux with a default ext4 or tmpfs filesystem, the default buffer size for
reading a pcap is chosen as 4k (strace/gdb validated). When reading large pcaps
containing raw data transfers, the syscall overhead for read becomes visible
in profiles. Support configurability of the buffer size and default to 128kb.
When processing a ~830M PCAP (16 UDP connections, each transferring ~50MB) in
bare mode, this change improves runtime from 1.39 sec to 1.29 sec. Increasing
the buffer further didn't provide a noticeable boost.
The PktSrc::Stats object works with 64bit unsigned integers. Unfortunately,
libpcap's struct pcap_stat is using 32bit values and users have reported
the wrapping of these values being visible in their stats.log roughly every
7.5 hours (~160kpps).
This change moves tracking of link and drop counters into the PktSrc::Stats
object (like is done for received and bytes_received) and updates them
on a call to PcapSource::Statistics() with the difference to the
previous stats values to prevent the wrap from becoming visible to
script land.
This doesn't cover the case of the stats counters wrapping around multiple
times between two invocations of PktSrc::Statistics(). With the default
interval of 5 minutes for the stats script, this seems acceptable.
Closes#2791.
Switches from pcap_next() to pcap_next_ex() to better handle all error
conditions. This allows, for example, to have a non-zero exit code for
a Zeek process that fails to fully process all packets in a pcap file.
The Zeek code base has very inconsistent #includes. Many sources
included a few headers, and those headers included other headers, and
in the end, nearly everything is included everywhere, so missing
#includes were never noticed. Another side effect was a lot of header
bloat which slows down the build.
First step to fix it: in each source file, its own header should be
included first to verify that each header's includes are correct, and
none is missing.
After adding the missing #includes, I replaced lots of #includes
inside headers with class forward declarations. In most headers,
object pointers are never referenced, so declaring the function
prototypes with forward-declared classes is just fine.
This patch speeds up the build by 19%, because each compilation unit
gets smaller. Here are the "time" numbers for a fresh build (with a
warm page cache but without ccache):
Before this patch:
3144.94user 161.63system 3:02.87elapsed 1808%CPU (0avgtext+0avgdata 2168608maxresident)k
760inputs+12008400outputs (1511major+57747204minor)pagefaults 0swaps
After this patch:
2565.17user 141.83system 2:25.46elapsed 1860%CPU (0avgtext+0avgdata 1489076maxresident)k
72576inputs+9130920outputs (1667major+49400430minor)pagefaults 0swaps
This commit marks (hopefully) ever one-parameter constructor as explicit.
It also uses override in (hopefully) all circumstances where a virtual
method is overridden.
There are a very few other minor changes - most of them were necessary
to get everything to compile (like one additional constructor). In one
case I changed an implicit operation to an explicit string conversion -
I think the automatically chosen conversion was much more convoluted.
This took longer than I want to admit but not as long as I feared :)
I reworked this a bit:
- Moved the globals into a new Pcap::* namespace, and renamed them
slightly.
- Moved the definitions of the globals into pcap/const.bif.
- Also moved the existing 'snaplen' into Pcap::* and removed
SnapLen() from the PktSrc API (it's really a pcap thing).
- Likewise moved the existing functions precompile_pcap_filter,
install_pcap_filter, and pcap_error, into Pcap::*.
- Did some more refactoring for the pcap code.
* 'master' of https://github.com/knielander/bro:
Refactored patch (removed options, less ambiguous name)
Allow Bro to run in fanout mode.
Allow libpcap buffer size to be set manually.
Allow Bro to run in fanout mode.
Allowed libpcap buffer size to be set via configuration.
