* origin/topic/johanna/gh-859:
Add X509/SSL changes to NEWS
X509: add check if function succeeds
GH-1634: Address feedback
Small indentation fixes in ssl-log-ext.zeek
Fix memory leak in x509_check_cert_hostname bif
Small bugfix and updates for external test hashes (SSL/X509)
Baseline updates for recent SSL changes.
Add ability to check if hostname is valid for a specific cert
Add ssl_history field to ssl.log
Add policy script suppressing certificate events
Add new ssl-log-ext policy script
Deprecate extract-certs-pem.zeek and add log-certs-base64.zeek
Implement X509 certificate log caching
Deprecate ICSI SSL notary script.
Change SSL and X.509 logging format
Enable OCSP logging by default.
Split the code that handles X509 event hashing into its own file
Closes GH-859
Changes \x00-\x37 ranges to \x00-\x1f with assumption that the former
was attempting to match ASCII control characters, but mistook an octal
range for hex. This change reduces some false positives.
- Tweaked the Too_Little_Traffic notice message to avoid
cluster-specific terminology.
* origin/topic/vlad/caploss_no_traffic:
Fix scheduling due to network_time being 0 in zeek_init
Add test for CaptureLoss::Too_Little_Traffic
Add CaptureLoss::Too_Little_Traffic
Add CaptureLoss::initial_watch_interval for a quick read on cluster health after startup.
Documentation update, reference the threshold variable. [nomail] [skip ci]
Whitespace fixes only [nomail] [skip ci]
The RSTOS0 `conn_state` label is documented as "Originator sent a SYN
followed by a RST, never saw SYN-ACK from responder", but was previously
applied to cases where no originator SYN exists, like a single RST-only
packet.
* origin/topic/timw/open-dict: (40 commits)
Move Dict constants to detail namespace
Add a few missing deprecation fixes
Adjust Dict whitespace/style
Adjust more btest timings
Improve termination reliability/speed for brokerstore btests
General btest cleanup
Update NEWS about change in Dictionary implementation
Improve Intel expire-item btest to be less time-sensitive
Improve btests with unstable table/set output ordering
Update doc submodule
Adjust a few btests that were unstable due to time-sensitivity
Fix DNS script deleting a table element while iterating
Improve a brokerstore btest to filter out Broker connection messages
Sort output of a few SumStats cluster tests
Fix extract_first_email_addr() to really return the first email
Add find_all_ordered() BIF
Extend external test suite canonifier with set-sorting logic
Update btests/baselines for OpenDict compat
Fix new/malloc/delete/free mismatches in Dictionary code
Add explanation for a Dict TODO item
...
The body-lengths of sub-entities, like multipart messages, got counted
twice by mistake: once upon the end of the sub-entity and then again
upon the end of the top-level entity that contains all sub-entities.
The size of just the top-level entity is the correct one to use.
Changes during merge
- Changed the policy script to use an event handler that behaves
for like the base script: &priority=5, msg$opcode != early-out,
no record field existence checks
- Also extended dns_query_reply event with original_query param
- Removed ExtractName overload, and just use default param
* 'dns-original-query-case' of https://github.com/rvictory/zeek:
Fixed some places where tabs became spaces
Stricter checking if we have a dns field on the connection being processed
Modified the DNS protocol analyzer to add a new parameter to the dns_request event which includes the DNS query in its original case. Added a policy script that will add the original_case to the dns.log file as well. Created new btests to test both.
- Updated the logic significantly: still filters out ICMP from being
considered an active service (like before) and adds a new
"Known::service_udp_requires_response" option (defaults to true) for
whether to require UDP server response before being considered an
active service.
* 'topic/dopheide/known-services' of https://github.com/dopheide-esnet/zeek:
Log services with unknown protocols
* topic/johanna/gh-744-cache-the-heck-out-of-these-certs:
X509 caching API change: callback gets entry directly
Address feedback
Small default updates & external baselines.
X509 caching: small API changes, tests & test updates
Certificate caching - now working in principle.
First (sadly not completely working) approach at caching X509 certificates
Fixes in merge:
- Memory leak in HashKey ctor
- Minor whitespace/style changes
* origin/topic/dev/patterns-in-sets:
Enable Patterns as Table index in non singleton cases
Added support to create a Hashkey for PatternVals using their Pattern Texts
* origin/topic/timw/travis-leaks:
fixup! Add new distro to Travis CI configuration for running leak tests
Add new distro to Travis CI configuration for running leak tests
* 'known_services_multiprotocols' of https://github.com/mauropalumbo75/zeek:
improve logging with broker store
drop services starting with -
remove service from key for Cluster::publish_hrw
remove check for empty services
update tests
order list of services in store key
remove repeated services in logs if already seen
add multiprotocol known_services when Known::use_service_store = T
remove hyphen in front of some services (for example -HTTP, -SSL) In some cases, there is an hyphen before the protocol name in the field connection$service. This can cause problems in known_services and is removed here. It originates probably in some analyzer where it would be better removed in the future.
add multiprotocol known_services when Known::use_service_store = F
Changes during merge:
* whitespace
* add unit test
* 'empty_services' of https://github.com/mauropalumbo75/zeek:
remove empty services and include udp active connections when logging in connection_state_remove
* 'ntp-rewrite' of https://github.com/mauropalumbo75/zeek: (25 commits)
update tests baseline
Apply requested changes: - file dpd.sig and TODO comments for signature protocol detection removed - missing doc field filled in events.bif - rename OpCode and ReqCode fields into op_code and req_code respectively - removed unnecessary child method in NTP.h/.cc - main.zeek and ntp-protocol.pac reformatted
minor changes in the documentation
fix some initializations
fix wrong assignment of control key_id/crypto_checksum
code clean up
add extension fields parsing
add extended mac field with 20 byte digest (+4 byte key id)
update tests and add a new one for key_id and mac
fix auth field (key_id and mac) in standard and control msg
remove old NTP record in init-bare.zeek
fix key_id and digest (WIP)
fix wrong Assign with reference_id
add tests for ntp protocol (finished)
add tests for ntp protocol (WIP)
fix problem with time vals
add ntp records to init-bare.zeek
update ntp analyzer to val_mgr
extend and refact script-side of NTP analyzer
extend and refactor several fields
...
