By default, each certificate is now output only once per hour. This also
should work in cluster mode, where we use the net broker-table-syncing
feature to distribute the information about already seen certificates
across the entire cluster.
Log caching is also pretty configureable and can be changed using a
range of confiuration options and hooks.
Note that this is currently completely separate from X509 events
caching, which prevents duplicate parsing of X509 certificates.
This commit changes the SSL and X.509 logging formats to something that,
hopefully, slowly approaches what they will look like in the future.
X.509 log is not yet deduplicated; this will come in the future.
This commit introduces two new options, which determine if certificate
issuers and subjects are still logged in ssl.log. The default is to have
the host subject/issuer logged, but to remove client-certificate
information. Client-certificates are not a typically used feature
nowadays.
In the past I thought that this is not super interesting. However, it
turns out that this can actually contain a slew of interresting
information - like operating systems querying for the revocation of
software signing certificates, e.g.
So - let's just enable this as a default log for the future.
In 3769ed6c66 we added handling for SSH version 1.99 which unsed a SSH
version of 0 to indicate weird cases where no version could be
determined.
This patch is a fixup for that patch. Instead of using a magic version
of 0 we now use an `&optional` version value. If no SSH version can be
extracted the version will be unset; additionally a `conn_weird` event
will be raised.
Closes#1590.
This allows for data that won't match a SIP request method to precede an
actual request and generates a new 'sip_junk_before_request' weird when
encountering such a situation.
Merge adjustments:
- Rewrote the check for error response as a switch statement to
fix compiler warning about signed/unsigned comparison and also
to just simplify/clarify the logic.
- Changed the btest to use `zeek -b`.
* origin/topic/vlad/gh-1286:
Add tests for new SMB3 multichannel support
Fix SMB2 response status parsing. Fixes#1286
By default all baslines are run through diff-remove-timestamp. On a BSD
sed implementation, this means that a newline is added to the end of the
file, if no newline was there originally. This behavior differs from GNU
sed, which does not add a newline.
In this commit we unify this behavior by always adding a newline, even
when using GNU sed. This commit also disables the canonifier for a bunch
of binary baselines, so we do not have to change them.
* origin/topic/jsiwek/gh-1264-ssh-host-key-fingerprints:
Simply ssh/main.zeek by using "ssh_server_host_key" for fingerprinting
Deprecate "ssh1_server_host_key" parameters *e* and *p*
GH-1264: Implement "ssh_server_host_key" event
SSH can set in its identification a version 1.99 (SSH-1.99-xxx).
That means the client/server is compatible with SSHv1 and SSHv2.
So the version choice depends of the both side.
1.99 : 1.99 => 2.0
1.99 : 1.x => 1.x
1.99 : 2.0 => 2.O
(see "Compatibility With Old SSH Versions" in RFC 4253)
A short-term measure so that the Zeek source tree can simply exist on a
Windows filesystem. For true support, the logic that decides/generates
the filename format will need to change.
- Changed the new stub events to correctly check for existence of
their associated handler before generating an event
- Added a test case for the new stub event
* 'add-dce-rpc-payloads' of https://github.com/ynadji/zeek:
Add stub payload to dce_rpc_request and dce_rpc_response
The RSTOS0 `conn_state` label is documented as "Originator sent a SYN
followed by a RST, never saw SYN-ACK from responder", but was previously
applied to cases where no originator SYN exists, like a single RST-only
packet.
This adds two new functions: `Conn::register_removal_hook()` and
`Conn::unregister_removal_hook()` for registering a hook function to be
called back during `connection_state_remove`. The benefit of using hook
callback approach is better scalability: the overhead of unrelated
protocols having to dispatch no-op `connection_state_remove` handlers is
avoided.
This is to avoid missing large sessions where a single side exceeds
the DPD buffer size. It comes with the trade-off that now the analyzer
can be triggered by anybody controlling one of the endpoints (instead
of both).
Test suite changes are minor, and nothing in "external".
Closes#343.
* origin/topic/timw/open-dict: (40 commits)
Move Dict constants to detail namespace
Add a few missing deprecation fixes
Adjust Dict whitespace/style
Adjust more btest timings
Improve termination reliability/speed for brokerstore btests
General btest cleanup
Update NEWS about change in Dictionary implementation
Improve Intel expire-item btest to be less time-sensitive
Improve btests with unstable table/set output ordering
Update doc submodule
Adjust a few btests that were unstable due to time-sensitivity
Fix DNS script deleting a table element while iterating
Improve a brokerstore btest to filter out Broker connection messages
Sort output of a few SumStats cluster tests
Fix extract_first_email_addr() to really return the first email
Add find_all_ordered() BIF
Extend external test suite canonifier with set-sorting logic
Update btests/baselines for OpenDict compat
Fix new/malloc/delete/free mismatches in Dictionary code
Add explanation for a Dict TODO item
...
- Use `-b` most everywhere, it will save time.
- Start some intel tests upon the input file being fully read instead of
at an arbitrary time.
- Improve termination condition for some sumstats/cluster tests.
- Filter uninteresting output from some supervisor tests.
- Test for `notice_policy.log` is no longer needed.
The body-lengths of sub-entities, like multipart messages, got counted
twice by mistake: once upon the end of the sub-entity and then again
upon the end of the top-level entity that contains all sub-entities.
The size of just the top-level entity is the correct one to use.
* origin/topic/vladg/gh-1084:
Add btest for GH-1084
Update baselines
MySQL: Fix parsing logic bug. We were correctly NOT expecting an EOF, but because we were parsing the header and then not parsing the rest, we would get out of sync
For `DHCP::ClientID$hwtype` fields equal to 0, the `hwaddr` field is
no longer misformatted as a MAC and instead just contains the raw bytes
seen in the DHCP Client ID Option.
This also updates all usages of the deprecated Val ctor to use
either IntervalVal, TimeVal, or DoubleVal ctors. The reason for
doing away with the old constructor is that using it with TYPE_INTERVAL
isn't strictly correct since there exists a more specific subclass,
IntervalVal, with overriden ValDescribe() method that ought to be used
to print such values in a more descriptive way.
- Squashed the original commit set
- Cleaned up formatting
- Fixed register_for_ports() for right RDPEUDP analyzer
* topic/ak/rdpeudp:
Add RDP over UDP analyzer
* The compression capability was incorrectly set to 0x0004 instead of 0x0003
* The padding was 4-byte instead of 8-byte aligned and also the spec.
does not strictly require the padding for the last item in the list.
* Add a default case to handle parsing of unknown context types.
Or otherwise convert into a regular btest if it didn't already seem to
be covered.
There's no need for a separate memory leak test group since compiling
with LeakSanitizer now covers leak checking for the full btest suite.
