util::safe_write() calls abort() in case of EAGAIN errors. This is
easily observed when starting clusters with 32 workers or more.
Add a custom write_message() function handling EAGAIN by retrying
after a small sleep. It's not clear a more complicated poll() would be
much better: The pipe might be ready for writing, but then our message
might not actually fit in, resulting in another EAGAIN error. And even
poll() would introduce blocking/sleeping code.
Take some precautions against the stem and the supervisor dead-locking
when both pipes are full by draining the other end on EAGAIN errors.
Closes#3043
This change allows to invoke a reproducer with different script options
and extra scripts to ease debugging and investigation. For example, enabling
the DPD debug stream and adding misc/dump-events can be done as follows after
this change:
ZEEK_DEBUG_LOG_STDERR=1 ../../build/src/fuzzers/zeek-pop3-fuzzer <test-case> -- -B dpd misc/dump-events
The ssl_history field may grow unbounded (e.g., ssl_alert event). Prevent this
by capping using a configurable limit (default 100) and raise a weird once reached.
Limit the number of events raised from an SSL record with content_type
alert (21) to a configurable maximum number (default 10). For TLS 1.3,
the limit is set to 1 as specified in the RFC. Add a new weird cases
where the limit is exceeded.
OSS-Fuzz managed to generate a reproducer that raised ~660k ssl_plaintext
and ssl_alert events given ~810kb of input data. This change prevents this
with hopefully no negative side-effect in the real-world.
Previously, seq was computed as the result of |pending_commands|+1. This
opened the possibility to override queued commands, as well as logging
the same pending ftp reply multiple times.
For example, when commands 1, 2, 3 are pending, command 1 may be dequeued,
but the incoming command then receives seq 3 and overrides the already
pending command 3. The second scenario happens when ftp_reply() selected
command 3 as pending for logging, but is then followed by many ftp_request()
events. This resulted in command 3's response being logged for every
following ftp_request() over and over again.
Avoid both scenarios by tracking the command sequence as an absolute counter.
Unsure what it's used for today and also results in the situation that on
some platforms we generate a reporter.log in bare mode, while on others
where spicy is disabled, we do not.
If we want base/frameworks/version loaded by default, should put it into
init-bare.zeek and possibly remove the loading of the reporter framework
from it - Reporter::error() would still work and be visible on stderr,
just not create a reporter.log.
Makes testing easier and aligns better with log rotation and timer
expiration. Should not have an effect in practice. Also, log detail
about whether inode or modification time changed, too.
The mmdb references needs to use reset() instead of release()
so that the destructor gets called and the stale DB is actually
cleaned up. Otherwise, the pointer is leaked and the stale DB
remains in memory.
Because EventMgr is defined globally as an object (rather than a global
pointer to an EventMgr object), its pipe is created even before main()
is entered. This further means that in the fork-based supervisor setup,
all Zeek processes created from the top-level supervisor process share
the same pipe object for the EventMgr. In turn, whenever any of the
processes enqueued an event, the flare was fired and ready for reading
on all other processes in the cluster, causing much contention and
unneeded overhead.
Closes#3190
Corpus was generated from a few btest pcaps, fuzzing for 60 seconds
locally and minimizing afterwards as in the documentation.
List of pcaps:
certificate-request-failed.pcap
certificate-with-sct.pcap
cert-no-cn.pcap
chrome-34-google.trace
chrome-63.0.3211.0-canary-tls_experiment.pcap
heartbleed-encrypted.pcap
heartbleed-encrypted-short.pcap
heartbleed-encrypted-success.pcap
heartbleed.pcap
heartbleed-success.pcap
ssl-v2.trace
ssl.v3.trace
tls1_1.pcap
tls-1.2-handshake-failure.trace
tls1.2-protocol-violation.pcap
tls1.2.trace
tls13_wolfssl.pcap
tls-early-alert.trace
For records and enums, the type registration calls could get pretty
large, and then slow to compile. We now build up their vectors
entry by entry instead of all at once.
The fuzzer generated input where data length left was 14 and the amsdu_len
field set to 14. That caused buffer overread due not taking into account
the amsdu header length of 14.
The first pcap only contained packets from the originator, not the responder.
What stands out here is that the Linux kernel doesn't seem to use a symmetric
flow hash for the tunneled connection, resulting in a total of four tunnel
connections for the two inner connections. Sigh.
