* 'nfs_changes' of https://github.com/jwallior/bro:
Add nfs unittest. Includes an example for the new nfs_proc_rename.
Added rename event to rpc/nfs protocol analyzer. This event identifies and reports information about nfs/rpc calls and replies of the type rename.
Expand parsing of RPC Call packets to add Uid, Gid, Stamp, MachineName and AuxGIDs
Fix NFS protocol parser.
The tests that were using the broccoli-v6addrs test program would
sometimes fail because broccoli-v6addrs would try to connect to Bro
and fail (presumably because Bro hadn't yet fully initialized).
Fixed by using the new broccoli-v6addrs "-r" option which will retry
upon failure to connect to Bro.
These were caused by earlier code updates.
This commit also removes the ocsp stapling logging script; it was
defunctional and the information provided by it wnever really has been
especially interesting.
The two hooks being added are:
void HookLogInit(const std::string& writer, const std::string& instantiating_filter, bool local, bool remote, const logging::WriterBackend::WriterInfo& info, int num_fields, const threading::Field* const* fields);
which is called when a writer is being instantiated and contains
information about the fields being logged, as well as
bool HookLogWrite(const std::string& writer, const std::string& filter, const logging::WriterBackend::WriterInfo& info, int num_fields, const threading::Field* const* fields, threading::Value** vals);
which is called for each log line being written by each writer. It
contains all the data being written. The data can be changed in the
function call and lines can be prevented from being written.
This commit also fixes a few small problems with plugin hooks itself,
and extends the tests that were already there, besides introducing tests
for the added functionality.
This feature can be enabled globally for all logs by setting
LogAscii::gzip_level to a value greater than 0.
This feature can be enabled on a per-log basis by setting gzip-level in
$confic to a value greater than 0.
The dpd signature missed a few cases that are used for TLS 1.3,
especially when draft versions (which are all that we are seeing at the
moment) are being negotiated.
This fix mostly allows draft versions in the server hello (identified by
7F[version]; since we do not know how many drafts there will be, we are
currently allowing a rather safe upper limit.
This also rewrites the certificate validation script (which we need for
this) slightly.
This could need a bit of caching, but should generally work very
reliably.
This is much more complex than the TLS Extension/OCSP cases. We need to
first alter the certificate and remove the extension from it, before
extracting the tbscert. Furthermore, we need the key hash of the issuing
certificate to be able to validate the proof - which means that we need
a valid certificate chain.
Missing: documentation, nice integration so that we can just add a
script and use this in Bro.
This does not yet work for certificates, because this requires some
changing the ASN.1 structure before validation (we need to extract the
tbscert and remove the SCT extension before).
API will change in the future.
This commit add the table SSL::ct_logs to Bro. This table is populated
with information about the currently active certificate transparency
logs (data from Google). The data can, e.g., be used to identify which
Logs are being used in SCTs.
The changes are now a bit more succinct with less code changes required.
Behavior is tested a little bit more thoroughly and a memory problem
when reading incomplete lines was fixed. ReadHeader also always directly
returns if header reading failed.
Error messages now are back to what they were before the change, if the
new behavior is not used.
I also tweaked the documentation text a bit.
The linker was complaining about linking files that didn't
have any symbols. These were actually empty files so I just
got rid of them and removed references to them.
* origin/topic/robin/broker-logging:
Another fix for the new Broker-based remote logging.
Fix some minor issues.
Adding Broker ifdefs for new remote logging code.
Changing semantics of Broker's remote logging to match old communication framework.
By default, the ASCII reader does not fail on errors anymore.
If there is a problem parsing a line, a reporter warning is
written and parsing continues. If the file is missing or can't
be read, the input thread just tries again on the next heartbeat.
Options have been added to recreate the previous behavior...
const InputAscii::fail_on_invalid_lines: bool;
and
const InputAscii::fail_on_file_problem: bool;
They are both set to `F` by default which makes the input readers
resilient to failure.
- This fixes BIT-1769 by logging all requests even in the absence of a
reply. The way that request and replying matching were being handled
was restructured to mostly ignore the transaction ids because they
aren't that helpful for network monitoring and it makes the script
structure more complicated.
- Add `framed_addr` field to the radius log to indicate if the radius
server is hinting at an address for the client.
- Add `ttl` field to indicate how quickly the radius server is replying
to the network access server.
- Fix a bunch of indentation inconsistencies.
Broker had changed the semantics of remote logging: it sent over the
original Bro record containing the values to be logged, which on the
receiving side would then pass through the logging framework normally,
including triggering filters and events. The old communication system
however special-cases logs: it sends already processed log entries,
just as they go into the log files, and without any receiver-side
filtering etc. This more efficient as it short-cuts the processing
path, and also avoids the more expensive Val serialization. It also
lets the sender determine the specifics of what gets logged (and how).
This commit changes Broker over to now use the same semantics as the
old communication system.
TODOs:
- The new Broker code doesn't have consistent #ifdefs yet.
- Right now, when a new log receiver connects, all existing logs
are broadcasted out again to all current clients. That doesn't so
any harm, but is unncessary. Need to add a way to send the
existing logs to just the new client.
Re-enable logging, now in policy because it probably is interesting to
no-one. We also only log ocsp replies.
Fix all tests.
Fix an issue where ocsp replies were added to the x.509 certificate
list.
With this change, we also parse signed certificate timestamps from OCSP
replies. This introduces a common base class between the OCSP and X509
analyzer, which now share a bit of common code. The event for signed
certificate timestamps is raised by both and thus renamed do:
x509_ocsp_ext_signed_certificate_timestamp
