While we support initializing records via coercion from an expression
list, e.g.,
local x: X = [$x1=1, $x2=2];
this can sometimes obscure the code to readers, e.g., when assigning to
value declared and typed elsewhere. The language runtime has a similar
overhead since instead of just constructing a known type it needs to
check at runtime that the coercion from the expression list is valid;
this can be slower than just writing the readible code in the first
place, see #4559.
With this patch we use explicit construction, e.g.,
local x = X($x1=1, $x2=2);
This adds a "policy" hook into the logging framework's streams and
filters to replace the existing log filter predicates. The hook
signature is as follows:
hook(rec: any, id: Log::ID, filter: Log::Filter);
The logging manager invokes hooks on each log record. Hooks can veto
log records via a break, and modify them if necessary. Log filters
inherit the stream-level hook, but can override or remove the hook as
needed.
The distribution's existing log streams now come with pre-defined
hooks that users can add handlers to. Their name is standardized as
"log_policy" by convention, with additional suffixes when a module
provides multiple streams. The following adds a handler to the Conn
module's default log policy hook:
hook Conn::log_policy(rec: Conn::Info, id: Log::ID, filter: Log::Filter)
{
if ( some_veto_reason(rec) )
break;
}
By default, this handler will get invoked for any log filter
associated with the Conn::LOG stream.
The existing predicates are deprecated for removal in 4.1 but continue
to work.
This adds two new functions: `Conn::register_removal_hook()` and
`Conn::unregister_removal_hook()` for registering a hook function to be
called back during `connection_state_remove`. The benefit of using hook
callback approach is better scalability: the overhead of unrelated
protocols having to dispatch no-op `connection_state_remove` handlers is
avoided.
And switch Zeek's base scripts over to using it in place of
"connection_state_remove". The difference between the two is
that "connection_state_remove" is raised for all events while
"successful_connection_remove" excludes TCP connections that were never
established (just SYN packets). There can be performance benefits
to this change for some use-cases.
There's also a new event called ``connection_successful`` and a new
``connection`` record field named "successful" to help indicate this new
property of connections.
- This separates NTLM handling away from SMB.
- It logs more accurately when logins are succeed or fail
or even if the resulting status of an authentication is
unknown.
- Adds some new fields where the server is indicating information
about itself (server_nb_computer_name, server_dns_computer_name,
and server_tree_name)
- NTLM Authentication failures over SMB2 are now marked as such in
the ntlm.log.
- Slightly updated filtering mechanism for DCE/RPC operations.
- Uncommented the atsvc file so it compiles now.
If only one side of a connection was seen, the ntlm.log
would indicate that the authentication failed. This has been
modified so that the success is listed as null since it's not
known whether or not the authentication was successful.
It can be inferred from continued SMB analysis though because
activity will continue taking place. I changed it though
because the log shouldn't assume more than what it sees.
- Looser coupling between these analyzers.
- New ntlm.log (still pretty early)
- Improved string handling for NTLM (convert UTF16 to UTF8)
- SMB2 analyzer now supports GSSAPI.
- Improved abstraction of DCE_RPC operations (still not finished)
- Lots of whitespace cleanup.
