The fmt_mac() function returning a std::string means the resulting mac
is copied at least once upon returning. Then, the Assign() in GetVal()
taking a std::string internally allocates a new zeek::String which
hits a malloc (no short-string optimization for zeek::String) and then
also copies the content from the std::string into the malloced memory.
Save a few cycles by directly using the allocated memory with the
String instance. This change improves runtime for a SYN-only pcap
with just base/protocols/conn loaded by some 1-2%.
This largely copies over Spicy's `.clang-format` configuration file. The
one place where we deviate is header include order since Zeek depends on
headers being included in a certain order.
Borrows the `in_cksum` code from tcpdump, which borrowed from FreeBSD.
It handles unaligned data better and also unrolls the inner loop to
process 16 two-byte values at a time versus 2 one-byte values at a time
in the previous version. Generally measured as ~1.5x faster in a
release build. The new API should generally be more amenable to any
future optimization explorations since all relevant data blocks are
available within a single call rather than spread across multiple.
This unfortunately cuases a ton of flow-down changes because a lot of other
code was depending on that definition existing. This has a fairly large chance
to break builds of external plugins, considering how many internal ones it broke.
safe_snprintf and safe_vsnprintf just exist to ensure that the resulting strings are always null-terminated. The documentation for snprintf/vsnprintf states that the output of those methods are always null-terminated, thus making the safe versions obsolete.
This also installs symlinks from "zeek" and "bro-config" to a wrapper
script that prints a deprecation warning.
The btests pass, but this is still WIP. broctl renaming is still
missing.
#239
The ones_complement_checksum function assumes that the bytes passed into
it are aligned on 16 bit boundaries.
When using gcc (GCC) 6.2.1 20160916 (Red Hat 6.2.1-2) with -O2, this
does not seem to hold true anymore; assuming 16 bit alignment will lead
to accesses to uninitialized memory and wrong checksums.
This commit adds a minimally invasive change that does not assume
alignment anymore. This might have a small performance impact for every
single packet we process.
This error occured reproducibly when called from icmp6_checksum.
The link-layer addresses are now part of the connection endpoints
following the originator-responder-pattern. The addresses are printed
with leading zeros. Additionally link-layer addresses are also extracted
for 802.11 plus RadioTap.
So that src/dst addresses used in pseudo-header are correct when
there's certain extension headers (routing/destination).
Add ICMP/ICMPv6 checksum unit tests.
- Accessible at script-layer through 'mobile_ipv6_message' event.
- All Mobile IPv6 analysis now enabled through --enable-mobile-ipv6
configure-time option, otherwise the mobility header, routing type 2,
and Home Address Destination option are ignored.
- flow_weird event with name argument value of "routing0_hdr" is raised
for packets containing an IPv6 routing type 0 header because this
type of header is now deprecated according to RFC 5095.
- packets with a routing type 0 header and non-zero segments left
now use the last address in that header in order to associate
with a connection/flow and for calculating TCP/UDP checksums.
- added a set of IPv4/IPv6 TCP/UDP checksum unit tests
As we can't use the IPAddr class (because it's not thread-safe), this
involved a bit manual address manipulation and also shuffling some
things around a bit.
Not fully working yet, the tests for remote logging still fail.
Internally, all BROv6 preprocessor switches were removed and
addr/subnet representations wrapped in the new IPAddr/IPPrefix classes.
Some script-layer changes of note:
- dns_AAAA_reply event signature changed: the string representation
of an IPv6 addr is easily derived from the addr value, it doesn't
need to be another parameter. This event also now generated directly
by the DNS analyzer instead of being "faked" into a dns_A_reply event.
- removed addr_to_count BIF. It used to return the host-order
count representation of IPv4 addresses only. To make it more
generic, we might later add a BIF to return a vector of counts
in order to support IPv6.
- changed the result of enclosing addr variables in vertical pipes
(e.g. |my_addr|) to return the bit-width of the address type which
is 128 for IPv6 and 32 for IPv4. It used to function the same
way as addr_to_count mentioned above.
- remove bro_has_ipv6 BIF
The Logger class is now in charge of reporting all errors, warnings,
informational messages, weirds, and syslogs. All other components
route their messages through the global bro_logger singleton.
The Logger class comes with these reporting methods:
void Message(const char* fmt, ...);
void Warning(const char* fmt, ...);
void Error(const char* fmt, ...);
void FatalError(const char* fmt, ...); // Terminate Bro.
void Weird(const char* name);
[ .. some more Weird() variants ... ]
void Syslog(const char* fmt, ...);
void InternalWarning(const char* fmt, ...);
void InternalError(const char* fmt, ...); // Terminates Bro.
See Logger.h for more information on these.
Generally, the reporting now works as follows:
- All non-fatal message are reported in one of two ways:
(1) At startup (i.e., before we start processing packets),
they are logged to stderr.
(2) During processing, they turn into events:
event log_message%(msg: string, location: string%);
event log_warning%(msg: string, location: string%);
event log_error%(msg: string, location: string%);
The script level can then handle them as desired.
If we don't have an event handler, we fall back to
reporting on stderr.
- All fatal errors are logged to stderr and Bro terminates
immediately.
- Syslog(msg) directly syslogs, but doesn't do anything else.
The three main types of messages can also be generated on the
scripting layer via new Log::* bifs:
Log::error(msg: string);
Log::warning(msg: string);
Log::message(msg: string);
These pass through the bro_logger as well and thus are handled in the
same way. Their output includes location information.
More changes:
- Removed the alarm statement and the alarm_hook event.
- Adapted lots of locations to use the bro_logger, including some
of the messages that were previously either just written to
stdout, or even funneled through the alarm mechanism.
- No distinction anymore between Error() and RunTime(). There's
now only one class of errors; the line was quite blurred already
anyway.
- util.h: all the error()/warn()/message()/run_time()/pinpoint()
functions are gone. Use the bro_logger instead now.
- Script errors are formatted a bit differently due to the
changes. What I've seen so far looks ok to me, but let me know
if there's something odd.
Notes:
- The default handlers for the new log_* events are just dummy
implementations for now since we need to integrate all this into
the new scripts anyway.
- I'm not too happy with the names of the Logger class and its
instance bro_logger. We now have a LogMgr as well, which makes
this all a bit confusing. But I didn't have a good idea for
better names so I stuck with them for now.
Perhaps we should merge Logger and LogMgr?
Removing everything related to trace rewriting.
(I wasn't too careful in ensuring that I catch everything in the
scripts; Seth is working on those anyway.)
(Merging by cherry-picking the corresponding commit, as the branch was
accidentally made off of the logging stuff).
