* origin/topic/dnthayer/ticket1836:
Add test to verify that log rotation works with gzipped logs
Fix ascii writer to not discard a ".gz" file extension
BIT-1836 #close
It turns out that the serial number field in all events was never
populated correctly. Instead, the previous field (issuer key hash) was
re-read and repeated in all events.
The pcap file format has a global header and a header per packet. The
global header of the pcap in question had a snaplen of 1, but with
packet headers indicating the full number of bytes saved within the
file. It seems like the pcap file must of been artifically edited in
order for it to be this way.
When reporting the captured length of a packet, Apple's version of
libpcap now seems to report the full number of bytes saved within the
pcap's per-packet headers, but other versions seem to report the snaplen
from the global pcap header. This caused the core.truncation test to
behave differently on macOS from other platforms.
I've manually hexedit'd the pcap so that the snaplen is still 1, but
contains just a single packet with a pcap header indicating a length of
8, which is less than the size of the link layer header and so should
still test the original code path that the unit test intended to
exercise.
The expire-redef.bro test was sometimes failing due to the second "Run"
message being printed after (should happen before) the "Expired"
message. Fixed by increasing the time interval between events.
Also reduced the number of events raised to make the test finish more
quickly.
The catch-and-release.bro test was failing whenever three conditions
were all true: sorting the netcontrol.log before comparing to
the baseline, the presence of LC_ALL=C in btest.cfg changes the sort
order, and sometimes the timestamp increases slightly beginning
with one of the rule_id == 5 lines.
As a result of these three conditions, the sorted order of the lines
with rule_id of 5 were different than the baseline.
Fixed by not sorting netcontrol.log, as this doesn't seem necessary.
Closes#1830.
* origin/topic/johanna/ocsp-sct-validate: (82 commits)
Tiny script changes for SSL.
Update CT Log list
SSL: Update OCSP/SCT scripts and documentation.
Revert "add parameter 'status_type' to event ssl_stapled_ocsp"
Revert "parse multiple OCSP stapling responses"
SCT: Fix script error when mime type of file unknown.
SCT: another memory leak in SCT parsing.
SCT validation: fix small memory leak (public keys were not freed)
Change end-of-connection handling for validation
OCSP/TLS/SCT: Fix a number of test failures.
SCT Validate: make caching a bit less aggressive.
SSL: Fix type of ssl validation result
TLS-SCT: compile on old versions of OpenSSL (1.0.1...)
SCT: Add caching support for validation
SCT: Add signed certificate timestamp validation script.
SCT: Allow verification of SCTs in Certs.
SCT: only compare correct OID/NID for Cert/OCSP.
SCT: add validation of proofs for extensions and OCSP.
SCT: pass timestamp as uint64 instead of time
Add CT log information to Bro
...
I added another small change - since we are inlining Configure(), we can
just set bro_plugin directly to BRO_PLUGIN_BRO_VERSION in
src/plugin/Plugin.h, instead of depending on the plugin to do it. This
also means we do not need to change init-plugin in bro-aux at this
moment.
BIT-1828 #closed
* origin/topic/robin/plugin-version-check:
Adding plugin API number into versioned function name, and removing old runtime API version check.
Extend plugin infrastructure to catch Bro version mismatches at link time.
- Addresses Philip Romero's question from the Bro mailing list.
- Adds Microsoft Edge as a detected browser.
- We are now unescaping encoded characters in software names.
* origin/topic/dnthayer/ticket1821:
Remove loading of listen.bro in tests that do not need it
Serialize tests that load listen.bro
Fix race condition causing some tests to fail
Fix a race condition in some failing tests
The broccoli-v6addrs "-r" option was renamed to "-R"
Fix a race condition in some failing tests
Tests that load "frameworks/communication/listen" must be serialized
to prevent other tests failing due to multiple Bro instances trying
to listen on the same port.
Removed loading of the "frameworks/communication/listen" script for
a couple of tests that don't need this functionality. This was causing
failures of some broccoli-related tests in the "istate" test directory
due to two instances of Bro trying to listen on the same port.
Use the new "-R" option for broccoli-vectors and broping so that
they will retry connecting to Bro until the connection is established.
This avoids a race condition and eliminates the need for a "sleep"
after starting Bro.
* 'nfs_changes' of https://github.com/jwallior/bro:
Add nfs unittest. Includes an example for the new nfs_proc_rename.
Added rename event to rpc/nfs protocol analyzer. This event identifies and reports information about nfs/rpc calls and replies of the type rename.
Expand parsing of RPC Call packets to add Uid, Gid, Stamp, MachineName and AuxGIDs
Fix NFS protocol parser.
The tests that were using the broccoli-v6addrs test program would
sometimes fail because broccoli-v6addrs would try to connect to Bro
and fail (presumably because Bro hadn't yet fully initialized).
Fixed by using the new broccoli-v6addrs "-r" option which will retry
upon failure to connect to Bro.
These were caused by earlier code updates.
This commit also removes the ocsp stapling logging script; it was
defunctional and the information provided by it wnever really has been
especially interesting.
The two hooks being added are:
void HookLogInit(const std::string& writer, const std::string& instantiating_filter, bool local, bool remote, const logging::WriterBackend::WriterInfo& info, int num_fields, const threading::Field* const* fields);
which is called when a writer is being instantiated and contains
information about the fields being logged, as well as
bool HookLogWrite(const std::string& writer, const std::string& filter, const logging::WriterBackend::WriterInfo& info, int num_fields, const threading::Field* const* fields, threading::Value** vals);
which is called for each log line being written by each writer. It
contains all the data being written. The data can be changed in the
function call and lines can be prevented from being written.
This commit also fixes a few small problems with plugin hooks itself,
and extends the tests that were already there, besides introducing tests
for the added functionality.
This feature can be enabled globally for all logs by setting
LogAscii::gzip_level to a value greater than 0.
This feature can be enabled on a per-log basis by setting gzip-level in
$confic to a value greater than 0.
The dpd signature missed a few cases that are used for TLS 1.3,
especially when draft versions (which are all that we are seeing at the
moment) are being negotiated.
This fix mostly allows draft versions in the server hello (identified by
7F[version]; since we do not know how many drafts there will be, we are
currently allowing a rather safe upper limit.
This also rewrites the certificate validation script (which we need for
this) slightly.
This could need a bit of caching, but should generally work very
reliably.
This is much more complex than the TLS Extension/OCSP cases. We need to
first alter the certificate and remove the extension from it, before
extracting the tbscert. Furthermore, we need the key hash of the issuing
certificate to be able to validate the proof - which means that we need
a valid certificate chain.
Missing: documentation, nice integration so that we can just add a
script and use this in Bro.
This does not yet work for certificates, because this requires some
changing the ASN.1 structure before validation (we need to extract the
tbscert and remove the SCT extension before).
API will change in the future.
