This adds a "policy" hook into the logging framework's streams and
filters to replace the existing log filter predicates. The hook
signature is as follows:
hook(rec: any, id: Log::ID, filter: Log::Filter);
The logging manager invokes hooks on each log record. Hooks can veto
log records via a break, and modify them if necessary. Log filters
inherit the stream-level hook, but can override or remove the hook as
needed.
The distribution's existing log streams now come with pre-defined
hooks that users can add handlers to. Their name is standardized as
"log_policy" by convention, with additional suffixes when a module
provides multiple streams. The following adds a handler to the Conn
module's default log policy hook:
hook Conn::log_policy(rec: Conn::Info, id: Log::ID, filter: Log::Filter)
{
if ( some_veto_reason(rec) )
break;
}
By default, this handler will get invoked for any log filter
associated with the Conn::LOG stream.
The existing predicates are deprecated for removal in 4.1 but continue
to work.
Changed some configuration defaults to potentially more same values.
The callback function is now a hook to allow costomization of the events
that are raised.
Tests now exist. Test baselines are updated.
I moved the replay function to a callback - which now means that the replayed
functions are called before file_state remove. Which makes this virtually
identical with the events raised by the core.
Currently this is mostly missing tests, leak-testing and performance-tests.
This approach mostly relies on script-level changes. In scriptland, we track
which certificates should be cached - and also perform the caching and the
replaying of events there.
This actually is probably nearly functional - the problem is the fact that now
the event ordering is wrong - and file_state_remove is called before the x509
events are thrown.
The fix probably is to move to a callback from the core - which can execute
things in the right order again. (Or just write the whole event-raising inside
the core - but that is both less flexible and a lot more cumbersome).
These are no longer loaded by default due to the performance impact they
cause simply by being loaded (they have event handlers for commonly
generated events) and they aren't generally useful enough to justify it.
* All "Broxygen" usages have been replaced in
code, documentation, filenames, etc.
* Sphinx roles/directives like ":bro:see" are now ":zeek:see"
* The "--broxygen" command-line option is now "--zeexygen"
Closes#1830.
* origin/topic/johanna/ocsp-sct-validate: (82 commits)
Tiny script changes for SSL.
Update CT Log list
SSL: Update OCSP/SCT scripts and documentation.
Revert "add parameter 'status_type' to event ssl_stapled_ocsp"
Revert "parse multiple OCSP stapling responses"
SCT: Fix script error when mime type of file unknown.
SCT: another memory leak in SCT parsing.
SCT validation: fix small memory leak (public keys were not freed)
Change end-of-connection handling for validation
OCSP/TLS/SCT: Fix a number of test failures.
SCT Validate: make caching a bit less aggressive.
SSL: Fix type of ssl validation result
TLS-SCT: compile on old versions of OpenSSL (1.0.1...)
SCT: Add caching support for validation
SCT: Add signed certificate timestamp validation script.
SCT: Allow verification of SCTs in Certs.
SCT: only compare correct OID/NID for Cert/OCSP.
SCT: add validation of proofs for extensions and OCSP.
SCT: pass timestamp as uint64 instead of time
Add CT log information to Bro
...
Re-enable logging, now in policy because it probably is interesting to
no-one. We also only log ocsp replies.
Fix all tests.
Fix an issue where ocsp replies were added to the x.509 certificate
list.
This makes it much easier for protocols where the mime type is known in
advance like, for example, TLS. We now do no longer have to perform deep
script-level magic.
Instead of having an additional string argument specifying if we are
sending a request or a reply, we now have an ANALYZER_OCSP_REQUEST and
an ANALYZER_OCSP_REPLY
Instead of having a big event, that tries to parse all the data into a
huge datastructure, we do the more common thing and use a series of
smaller events to parse requests and responses.
The new events are:
ocsp_request -> raised for an ocsp request, giving version and requestor
ocsp_request_certificate -> raised n times per request, once per cert
ocsp_response_status -> raised for each ocsp response, giving status
ocsp_response_bytes -> raised for each ocsp response with information
ocsp_response_certificate -> raised for each cert in an ocsp response
- New fields: extracted_cutoff and extracted_size.
These fields will be null if the file isn't extracted.
- Extended the extraction test to test the files log too.
