This adds a "policy" hook into the logging framework's streams and
filters to replace the existing log filter predicates. The hook
signature is as follows:
hook(rec: any, id: Log::ID, filter: Log::Filter);
The logging manager invokes hooks on each log record. Hooks can veto
log records via a break, and modify them if necessary. Log filters
inherit the stream-level hook, but can override or remove the hook as
needed.
The distribution's existing log streams now come with pre-defined
hooks that users can add handlers to. Their name is standardized as
"log_policy" by convention, with additional suffixes when a module
provides multiple streams. The following adds a handler to the Conn
module's default log policy hook:
hook Conn::log_policy(rec: Conn::Info, id: Log::ID, filter: Log::Filter)
{
if ( some_veto_reason(rec) )
break;
}
By default, this handler will get invoked for any log filter
associated with the Conn::LOG stream.
The existing predicates are deprecated for removal in 4.1 but continue
to work.
This adds two new functions: `Conn::register_removal_hook()` and
`Conn::unregister_removal_hook()` for registering a hook function to be
called back during `connection_state_remove`. The benefit of using hook
callback approach is better scalability: the overhead of unrelated
protocols having to dispatch no-op `connection_state_remove` handlers is
avoided.
And switch Zeek's base scripts over to using it in place of
"connection_state_remove". The difference between the two is
that "connection_state_remove" is raised for all events while
"successful_connection_remove" excludes TCP connections that were never
established (just SYN packets). There can be performance benefits
to this change for some use-cases.
There's also a new event called ``connection_successful`` and a new
``connection`` record field named "successful" to help indicate this new
property of connections.
* 'master' of https://github.com/spitfire55/bro:
Revert DNS query 255 from ANY to *
Missing commas...
Fix typo in dce-rpc consts
Refactor to use consistent numeric type in dce_rpc. Add missing DNS query type codes
I added back in DNS constants for PTR, EDNS, and ANY to avoid breaking
code for any people that use them.
Also omitted the DNP3 function code 0x83 name change from
"AUTHENTICATE_RESP" to "AUTHENTICATE_RESPONSE", again to avoid
potentially breaking code unnecessarily: "RESP" vs. "RESPONSE" is not
wrong in any sense, just maybe a matter of clarify.
This adds previously-missing support for "Alter Context"
request/response PDUs (initial patch contributed by Mark Fernandez).
Also, context ID arguments were added to dce_rpc_bind, dce_rpc_request,
and dce_rpc_response in order to properly track what endpoint/operation
a given opnum maps to.
Removed duplicated entries (`000001a0-0000-0000-c000-000000000046`, with `i=3, i=4`) and related comments line
This change should make this file more readable and it will be easier to convert it to other formats (personally I found it out while I was trying to convert this to a JavaScript structure).
Moved the definitions of DCE_RPC::BackingState and DCE_RPC::State types
into the export block. These types are used in the redef of the
"connection" record.
* origin/topic/seth/smb-auth-fixes:
Disable SMB2 error data parsing.
Removed some files that aren't being used.
Fix SMB tree connect handling.
Fix a small issue where DCE_RPC commands were improperly being logged.
SMB fixes and cleanup.
Including a test for raw NTLM in SMB
Updates for SMB auth handling from Martin van Hensbergen.
BIT-1721 #merged
SMB error handling improved. The analyzer isn't destroyed when a problem
is encoutered anymore. The flowbuffer in the parser is now flushed and
the analyzer is set to resync against an SMB command. This was needed
because there is some state about open files that is kept within the
parser itself which was being destroyed and that was causing analysis
after content gaps or parse errors to be faulty. The new mechanism
doesn't detroy the parser so parsing after gaps is improved.
DCE_RPC handling in SMB is improved in the edge case where a drive
mapping isn't seen. There is a new const named SMB::pipe_filenames
which is used as a heuristic for identifying "files" opened on named
pipe shares. If the share mapping type isn't known and a filename
in this set is found, the share type will change to "PIPE" by
generating an event named "smb_pipe_connect_heuristic". Reads and
writes to that file will be sent to the DCE_RPC analyzer instead of
to the files framework.
The concept of "unknown" share types has been removed due to the new
heuristic detection of share types.
Some general clean up of how the SMB cmd log is written and when.
- The logic for fragment handling has been rewritten and should be correct now.
- There are now tunables for fragment handling overflow situations.
- DCE_RPC::max_cmd_reassembly and DCE_RPC::max_frag_data
- They result in weirds and analyzer removal.
- Memory leak fixed by unique_ptr auto cleanup.
- DCE_RPC is now intolerate of content gaps and will stop
analyzing traffic if content gaps happen (like most other analyzers currently).
- DCE_RPC fragmentation handling returns!
- Fixed some general parsing issues
- Fixed an issue with the DCE_RPC signature not working for IPv6 connections.
- NTLM Authentication failures over SMB2 are now marked as such in
the ntlm.log.
- Slightly updated filtering mechanism for DCE/RPC operations.
- Uncommented the atsvc file so it compiles now.
- Fix an issue with svcctl uuid -> operation mapping.
- Add a heuristic to fill out the endpoint name in
case the original dce/rpc binding wasn't seen.
- Improve naming and code structure in the dce/rpc scripts.
- Previously there was an (incorrect) assumption that a TCP conneciton
would only ever have one DCE_RPC binding. That assumption was
incorrect and with named pipes over SMB there can be multiple
concurrent DCE_RPC bindings. This commit fixes that assumption by
dynamically creating a new DCE_RPC analyzer whenever a new, unknown
binding is created.
- There is a crash fix in how string handling in the bind_ack message
was done.
- Named pipe handling over SMB1 is still not working quite right
and problems will show up with multiplexed DCE_RPC bindings.
- Binding context versions are provided correctly now.
- RPC stubs are no longer provided and instead only the length is given.
- Some parsing problems related to handling the optional
auth fields in RPC messages are fixed. This removes some
dpd log messages I was seeing.
