Mirror/zeek
1
0
Fork 0
mirror of https://github.com/zeek/zeek.git synced 2025-10-02 14:48:21 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 2

Merge remote-tracking branch 'origin/topic/johanna/tls12-decryption'

Browse source 
Documentation is missing and will be added in the next couple of hours.

* origin/topic/johanna/tls12-decryption: (24 commits)
  TLS decryption: add test, fix small issues
  Address PR feedback
  TLS decryption: refactoring, more comments, less bare pointers
  Small code fix and test baseline update.
  SSL decryption: refactor TLS12_PRF
  SSL decryption: small style changes, a bit of documentation
  Deprecation and warning fixes
  Clang-format updates
  add missing call to EVP_KDF_CTX_set_params
  TLS decryption: remove payload from ssl_encrypted_data again.
  TLS 1.2 decryption: adapt OpenSSL 3.0 changes for 1.1
  ssl: adapt TLS-PRF to openSSL 3.0
  ssl/analyzer: potentially fix memory leaks caused by bytestrings
  analyzer/ssl: several improvements
  analyzer/ssl: defensive key length check + more debug logging
  testing: feature gate ssl/decryption test
  testing: add ssl/decryption test
  analyzer/ssl: handle missing <openssl/kdf.h>
  analyzer/ssl: silence warning in DTLS analyzer
  analyzer/ssl: move proc-{client,server}-hello into the respective analyzers
  ...
This commit is contained in:
Johanna Amann 2022-03-02 08:09:28 +00:00
commit d38923cfcf
25 changed files with 947 additions and 109 deletions
Download patch file Download diff file Expand all files Collapse all files

11
CHANGES
View file

@ -1,3 +1,14 @@
5.0.0-dev.151 | 2022-03-02 08:09:28 +0000
* SSL: rudimentary decryption for TLS 1.2 (Florian Wilkens, Johanna Amann)
With this version, we support rudimentary decryption of TLS 1.2 connections, if the key material
of the connection (in our case the pre-master secret) is available. Note that this functionality
only works for TLS 1.2 connections using the TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 cipher suite.
No other combinations are currently supported.
For more information, see the NEWS entry and the TLS Decryption documentation.
5.0.0-dev.121 | 2022-02-24 09:11:03 -0700 5.0.0-dev.121 | 2022-02-24 09:11:03 -0700
* GH-1980: Deprecate and return warning for zeek-config's caf-root option (Tim Wojtulewicz, Corelight) * GH-1980: Deprecate and return warning for zeek-config's caf-root option (Tim Wojtulewicz, Corelight)

8
NEWS
View file

@ -140,6 +140,14 @@ New Functionality
- The GRE analyzer now supports the Aruba WLAN protocol type. - The GRE analyzer now supports the Aruba WLAN protocol type.
- Zeek now features limited TLS decryption capabilities. This feature is experimental
and only works for TLS 1.2 connections that use the TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
ciphersuite. Furthermore Zeek requires access to the pre-master secret of each TLS
connection. Typically this functionality will be most useful when analyzing trace-files
where the TLS client recorded the key material. For more details and examples how to
use this functionality, see the TLS Decryption documentation at
https://docs.zeek.org/en/master/frameworks/tls-decryption.html
Changed Functionality Changed Functionality
--------------------- ---------------------

2
VERSION
View file

@ -1 +1 @@
5.0.0-dev.121 5.0.0-dev.151

111
scripts/policy/protocols/ssl/decryption.zeek Normal file
View file

@ -0,0 +1,111 @@
##! This script allows for the decryption of certain TLS 1.2 connections, if the user is in possession
##! of the private key material for the session. Key material can either be provided via a file (useful
##! for processing trace files) or via sending events via Broker (for live decoding).
##!
##! Please note that this feature is experimental and can change without guarantees to our typical
##! deprecation timeline. Please also note that currently only TLS 1.2 connections that use the
##! TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 cipher suite are supported.
@load base/frameworks/input
@load base/frameworks/notice
@load base/protocols/conn
@load base/protocols/ssl
module SSL;
# Do not disable analyzers after detection - otherwise we will not receive
# encrypted packets.
redef SSL::disable_analyzer_after_detection = F;
export {
## This can be set to a file that contains the session secrets for decryption, when parsing a pcap file.
## Please note that, when using this feature, you probably want to pause processing of data till this
## file has been read.
const keylog_file = getenv("ZEEK_TLS_KEYLOG_FILE") &redef;
## Secrets expire after this time of not being used.
const secret_expiration = 5 mins &redef;
## This event can be triggered, e.g., via Broker to add known keys to the TLS key database.
##
## client_random: client random for which the key is set
##
## keys: key material
global add_keys: event(client_random: string, keys: string);
## This event can be triggered, e.g., via Broker to add known secrets to the TLS secret datbase.
##
## client_random: client random for which the secret is set
##
## secrets: derived TLS secrets material
global add_secret: event(client_random: string, secrets: string);
}
@if ( keylog_file == "" )
# If a keylog file was given via an environment variable, let's disable secret expiration - that does not
# make sense for pcaps.
global secrets: table[string] of string = {} &redef;
global keys: table[string] of string = {} &redef;
@else
global secrets: table[string] of string = {} &read_expire=secret_expiration &redef;
global keys: table[string] of string = {} &read_expire=secret_expiration &redef;
@endif
redef record SSL::Info += {
# Decryption uses client_random as identifier
client_random: string &optional;
};
type SecretsIdx: record {
client_random: string;
};
type SecretsVal: record {
secret: string;
};
const tls_decrypt_stream_name = "tls-keylog-file";
event zeek_init()
{
# listen for secrets
Broker::subscribe("/zeek/tls/decryption");
if ( keylog_file != "" )
{
Input::add_table([$name=tls_decrypt_stream_name, $source=keylog_file, $destination=secrets, $idx=SecretsIdx, $val=SecretsVal, $want_record=F]);
Input::remove(tls_decrypt_stream_name);
}
}
event SSL::add_keys(client_random: string, val: string)
{
SSL::keys[client_random] = val;
}
event SSL::add_secret(client_random: string, val: string)
{
SSL::secrets[client_random] = val;
}
event ssl_client_hello(c: connection, version: count, record_version: count, possible_ts: time, client_random: string, session_id: string, ciphers: index_vec, comp_methods: index_vec)
{
c$ssl$client_random = client_random;
if ( client_random in keys )
set_keys(c, keys[client_random]);
else if ( client_random in secrets )
set_secret(c, secrets[client_random]);
}
event ssl_change_cipher_spec(c: connection, is_orig: bool)
{
if ( c$ssl?$client_random )
{
if ( c$ssl$client_random in keys )
set_keys(c, keys[c$ssl$client_random]);
else if ( c$ssl$client_random in secrets )
set_secret(c, secrets[c$ssl$client_random]);
}
}

1
scripts/test-all-policy.zeek
View file

@ -118,6 +118,7 @@
@load protocols/ssh/geo-data.zeek @load protocols/ssh/geo-data.zeek
@load protocols/ssh/interesting-hostnames.zeek @load protocols/ssh/interesting-hostnames.zeek
@load protocols/ssh/software.zeek @load protocols/ssh/software.zeek
@load protocols/ssl/decryption.zeek
@load protocols/ssl/expiring-certs.zeek @load protocols/ssl/expiring-certs.zeek
# @load protocols/ssl/extract-certs-pem.zeek # @load protocols/ssl/extract-certs-pem.zeek
@load protocols/ssl/heartbleed.zeek @load protocols/ssl/heartbleed.zeek

1
scripts/zeekygen/__load__.zeek
View file

@ -1,6 +1,7 @@
@load test-all-policy.zeek @load test-all-policy.zeek
# Scripts which are commented out in test-all-policy.zeek. # Scripts which are commented out in test-all-policy.zeek.
@load protocols/ssl/decryption.zeek
@load protocols/ssl/notary.zeek @load protocols/ssl/notary.zeek
@load frameworks/control/controllee.zeek @load frameworks/control/controllee.zeek
@load frameworks/control/controller.zeek @load frameworks/control/controller.zeek

4
src/analyzer/protocol/ssl/CMakeLists.txt
View file

@ -10,14 +10,10 @@ zeek_plugin_bif(events.bif)
zeek_plugin_bif(functions.bif) zeek_plugin_bif(functions.bif)
zeek_plugin_bif(consts.bif) zeek_plugin_bif(consts.bif)
zeek_plugin_pac(tls-handshake.pac tls-handshake-protocol.pac tls-handshake-analyzer.pac ssl-defs.pac zeek_plugin_pac(tls-handshake.pac tls-handshake-protocol.pac tls-handshake-analyzer.pac ssl-defs.pac
proc-client-hello.pac
proc-server-hello.pac
proc-certificate.pac proc-certificate.pac
tls-handshake-signed_certificate_timestamp.pac tls-handshake-signed_certificate_timestamp.pac
) )
zeek_plugin_pac(ssl.pac ssl-dtls-analyzer.pac ssl-analyzer.pac ssl-dtls-protocol.pac ssl-protocol.pac ssl-defs.pac zeek_plugin_pac(ssl.pac ssl-dtls-analyzer.pac ssl-analyzer.pac ssl-dtls-protocol.pac ssl-protocol.pac ssl-defs.pac
proc-client-hello.pac
proc-server-hello.pac
proc-certificate.pac proc-certificate.pac
) )
zeek_plugin_pac(dtls.pac ssl-dtls-analyzer.pac dtls-analyzer.pac ssl-dtls-protocol.pac dtls-protocol.pac ssl-defs.pac) zeek_plugin_pac(dtls.pac ssl-dtls-analyzer.pac dtls-analyzer.pac ssl-dtls-protocol.pac dtls-protocol.pac ssl-defs.pac)

7
src/analyzer/protocol/ssl/DTLS.cc
View file

@ -76,4 +76,11 @@ void DTLS_Analyzer::SendHandshake(uint16_t raw_tls_version, uint8_t msg_type, ui
} }
} }
bool DTLS_Analyzer::TryDecryptApplicationData(int len, const u_char* data, bool is_orig,
uint8_t content_type, uint16_t raw_tls_version)
{
// noop for now as DTLS decryption is currently not supported
return false;
}
} // namespace zeek::analyzer::dtls } // namespace zeek::analyzer::dtls

10
src/analyzer/protocol/ssl/DTLS.h
View file

@ -39,6 +39,16 @@ public:
static analyzer::Analyzer* Instantiate(Connection* conn) { return new DTLS_Analyzer(conn); } static analyzer::Analyzer* Instantiate(Connection* conn) { return new DTLS_Analyzer(conn); }
/**
* Try to decrypt TLS application data from a packet.
*
* For DTLS, this operation is not currently implemented and this function will
* always return false.
*
**/
bool TryDecryptApplicationData(int len, const u_char* data, bool is_orig, uint8_t content_type,
uint16_t raw_tls_version);
protected: protected:
binpac::DTLS::SSL_Conn* interp; binpac::DTLS::SSL_Conn* interp;
binpac::TLSHandshake::Handshake_Conn* handshake_interp; binpac::TLSHandshake::Handshake_Conn* handshake_interp;

315
src/analyzer/protocol/ssl/SSL.cc
View file

@ -1,20 +1,56 @@
#include "zeek/analyzer/protocol/ssl/SSL.h" #include "zeek/analyzer/protocol/ssl/SSL.h"
#include <arpa/inet.h>
#include <openssl/evp.h>
#include <openssl/opensslv.h>
#include "zeek/Reporter.h" #include "zeek/Reporter.h"
#include "zeek/analyzer/Manager.h"
#include "zeek/analyzer/protocol/ssl/events.bif.h" #include "zeek/analyzer/protocol/ssl/events.bif.h"
#include "zeek/analyzer/protocol/ssl/ssl_pac.h" #include "zeek/analyzer/protocol/ssl/ssl_pac.h"
#include "zeek/analyzer/protocol/ssl/tls-handshake_pac.h" #include "zeek/analyzer/protocol/ssl/tls-handshake_pac.h"
#include "zeek/analyzer/protocol/tcp/TCP_Reassembler.h" #include "zeek/analyzer/protocol/tcp/TCP_Reassembler.h"
#include "zeek/util.h" #include "zeek/util.h"
#ifdef OPENSSL_HAVE_KDF_H
#include <openssl/kdf.h>
#endif
#if defined(OPENSSL_VERSION_MAJOR) && (OPENSSL_VERSION_MAJOR >= 3)
#include <openssl/core_names.h>
#endif
namespace zeek::analyzer::ssl namespace zeek::analyzer::ssl
{ {
template <typename T> static inline T MSB(const T a)
{
return ((a >> 8) & 0xff);
}
template <typename T> static inline T LSB(const T a)
{
return (a & 0xff);
}
static std::basic_string<unsigned char> fmt_seq(uint32_t num)
{
std::basic_string<unsigned char> out(4, '\0');
out.reserve(13);
uint32_t netnum = htonl(num);
out.append(reinterpret_cast<u_char*>(&netnum), 4);
out.append(5, '\0');
return out;
}
SSL_Analyzer::SSL_Analyzer(Connection* c) : analyzer::tcp::TCP_ApplicationAnalyzer("SSL", c) SSL_Analyzer::SSL_Analyzer(Connection* c) : analyzer::tcp::TCP_ApplicationAnalyzer("SSL", c)
{ {
interp = new binpac::SSL::SSL_Conn(this); interp = new binpac::SSL::SSL_Conn(this);
handshake_interp = new binpac::TLSHandshake::Handshake_Conn(this); handshake_interp = new binpac::TLSHandshake::Handshake_Conn(this);
had_gap = false; had_gap = false;
c_seq = 0;
s_seq = 0;
pia = nullptr;
} }
SSL_Analyzer::~SSL_Analyzer() SSL_Analyzer::~SSL_Analyzer()
@ -98,4 +134,283 @@ void SSL_Analyzer::Undelivered(uint64_t seq, int len, bool orig)
interp->NewGap(orig, len); interp->NewGap(orig, len);
} }
void SSL_Analyzer::SetSecret(const zeek::StringVal& secret)
{
SetSecret(secret.Len(), secret.Bytes());
}
void SSL_Analyzer::SetSecret(size_t len, const u_char* data)
{
secret.clear();
secret.append((const char*)data, len);
}
void SSL_Analyzer::SetKeys(const zeek::StringVal& nkeys)
{
keys.clear();
keys.reserve(nkeys.Len());
std::copy(nkeys.Bytes(), nkeys.Bytes() + nkeys.Len(), std::back_inserter(keys));
}
void SSL_Analyzer::SetKeys(const std::vector<u_char> newkeys)
{
keys = std::move(newkeys);
}
std::optional<std::vector<u_char>>
SSL_Analyzer::TLS12_PRF(const std::string& secret, const std::string& label,
const std::string& rnd1, const std::string& rnd2, size_t requested_len)
{
#ifdef OPENSSL_HAVE_KDF_H
#if defined(OPENSSL_VERSION_MAJOR) && (OPENSSL_VERSION_MAJOR >= 3)
// alloc context + params
EVP_KDF* kdf = EVP_KDF_fetch(NULL, "TLS1-PRF", NULL);
EVP_KDF_CTX* kctx = EVP_KDF_CTX_new(kdf);
OSSL_PARAM params[4], *p = params;
EVP_KDF_free(kdf);
#else /* OSSL 3 */
// alloc buffers
EVP_PKEY_CTX* pctx = EVP_PKEY_CTX_new_id(EVP_PKEY_TLS1_PRF, NULL);
#endif /* OSSL 3 */
// prepare seed: seed = label + rnd1 + rnd2
std::string seed{};
seed.reserve(label.size() + rnd1.size() + rnd2.size());
seed.append(label);
seed.append(rnd1);
seed.append(rnd2);
#if defined(OPENSSL_VERSION_MAJOR) && (OPENSSL_VERSION_MAJOR >= 3)
// setup OSSL_PARAM array: digest, secret, seed
// FIXME: sha384 should not be hardcoded
// The const-cast is a bit ugly - but otherwise we have to copy the static string.
*p++ = OSSL_PARAM_construct_utf8_string(OSSL_KDF_PARAM_DIGEST, const_cast<char*>(SN_sha384), 0);
*p++ = OSSL_PARAM_construct_octet_string(OSSL_KDF_PARAM_SECRET, (void*)secret.data(),
secret.size());
*p++ = OSSL_PARAM_construct_octet_string(OSSL_KDF_PARAM_SEED, (void*)seed.data(), seed.size());
*p = OSSL_PARAM_construct_end();
auto keybuf = std::vector<u_char>(requested_len);
// set OSSL params
if ( EVP_KDF_CTX_set_params(kctx, params) <= 0 )
goto abort;
// derive key material
if ( EVP_KDF_derive(kctx, keybuf.data(), requested_len, nullptr) <= 0 )
goto abort;
EVP_KDF_CTX_free(kctx);
return keybuf;
abort:
EVP_KDF_CTX_free(kctx);
return {};
#else /* OSSL 3 */
auto keybuf = std::vector<u_char>(requested_len);
if ( EVP_PKEY_derive_init(pctx) <= 0 )
goto abort; /* Error */
// setup PKEY params: digest, secret, seed
// FIXME: sha384 should not be hardcoded
if ( EVP_PKEY_CTX_set_tls1_prf_md(pctx, EVP_sha384()) <= 0 )
goto abort; /* Error */
if ( EVP_PKEY_CTX_set1_tls1_prf_secret(pctx, secret.data(), secret.size()) <= 0 )
goto abort; /* Error */
if ( EVP_PKEY_CTX_add1_tls1_prf_seed(pctx, seed.data(), seed.size()) <= 0 )
goto abort; /* Error */
if ( EVP_PKEY_derive(pctx, keybuf.data(), &requested_len) <= 0 )
goto abort; /* Error */
EVP_PKEY_CTX_free(pctx);
return keybuf;
abort:
EVP_PKEY_CTX_free(pctx);
#endif /* OSSL 3 */
#endif /* HAVE_KDF */
return {};
}
bool SSL_Analyzer::TryDecryptApplicationData(int len, const u_char* data, bool is_orig,
uint8_t content_type, uint16_t raw_tls_version)
{
// Unsupported cipher suite. Currently supported:
// - TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 == 0xC030
auto cipher = handshake_interp->chosen_cipher();
if ( cipher != 0xC030 )
{
DBG_LOG(DBG_ANALYZER, "Unsupported cipher suite for decryption: %d\n", cipher);
return false;
}
// Neither secret or key present: abort
if ( secret.size() == 0 && keys.size() == 0 )
{
DBG_LOG(
DBG_ANALYZER,
"Could not decrypt packet due to missing keys/secret. Client_random: %s\n",
util::fmt_bytes(reinterpret_cast<const char*>(handshake_interp->client_random().data()),
handshake_interp->client_random().length()));
// FIXME: change util function to return a printably std::string for DBG_LOG
// print_hex("->client_random:", handshake_interp->client_random().data(),
// handshake_interp->client_random().size());
return false;
}
// Secret present, but no keys derived yet: derive keys
if ( secret.size() != 0 && keys.size() == 0 )
{
#ifdef OPENSSL_HAVE_KDF_H
DBG_LOG(DBG_ANALYZER, "Deriving TLS keys for connection");
uint32_t ts = htonl((uint32_t)handshake_interp->gmt_unix_time());
auto c_rnd = handshake_interp->client_random();
auto s_rnd = handshake_interp->server_random();
std::string crand;
crand.append(reinterpret_cast<char*>(&(ts)), 4);
crand.append(reinterpret_cast<char*>(c_rnd.data()), c_rnd.length());
std::string srand(reinterpret_cast<char*>(s_rnd.data()), s_rnd.length());
// fixme - 72 should not be hardcoded
auto res = TLS12_PRF(secret, "key expansion", srand, crand, 72);
if ( ! res )
{
DBG_LOG(DBG_ANALYZER, "TLS PRF failed. Aborting.\n");
return false;
}
// save derived keys
SetKeys(res.value());
#else
DBG_LOG(DBG_ANALYZER,
"Cannot derive TLS keys as Zeek was compiled without <openssl/kdf.h>");
return false;
#endif
}
// Keys present: decrypt TLS application data
if ( keys.size() == 72 )
{
// FIXME: could also print keys or conn id here
DBG_LOG(DBG_ANALYZER, "Decrypting application data");
// NOTE: you must not call functions that invalidate keys.data() on keys during the
// remainder of this function. (Given that we do not manipulate the key material in this
// function that should not be hard)
// client write_key
const u_char* c_wk = keys.data();
// server write_key
const u_char* s_wk = keys.data() + 32;
// client IV
const u_char* c_iv = keys.data() + 64;
// server IV
const u_char* s_iv = keys.data() + 68;
// FIXME: should we change types here?
u_char* encrypted = (u_char*)data;
size_t encrypted_len = len;
if ( is_orig )
c_seq++;
else
s_seq++;
// AEAD nonce, length 12
std::basic_string<unsigned char> s_aead_nonce;
if ( is_orig )
s_aead_nonce.assign(c_iv, 4);
else
s_aead_nonce.assign(s_iv, 4);
// this should be the explicit counter
s_aead_nonce.append(encrypted, 8);
assert(s_aead_nonce.size() == 12);
EVP_CIPHER_CTX* ctx = EVP_CIPHER_CTX_new();
EVP_CIPHER_CTX_init(ctx);
EVP_CipherInit(ctx, EVP_aes_256_gcm(), NULL, NULL, 0);
encrypted += 8;
// FIXME: is this because of nonce and aead tag?
if ( encrypted_len <= (16 + 8) )
{
DBG_LOG(DBG_ANALYZER, "Invalid encrypted length encountered during TLS decryption");
EVP_CIPHER_CTX_free(ctx);
return false;
}
encrypted_len -= 8;
encrypted_len -= 16;
// FIXME: aes_256_gcm should not be hardcoded here ;)
if ( is_orig )
EVP_DecryptInit(ctx, EVP_aes_256_gcm(), c_wk, s_aead_nonce.data());
else
EVP_DecryptInit(ctx, EVP_aes_256_gcm(), s_wk, s_aead_nonce.data());
EVP_CIPHER_CTX_ctrl(ctx, EVP_CTRL_GCM_SET_TAG, 16, encrypted + encrypted_len);
// AEAD tag
std::basic_string<unsigned char> s_aead_tag;
if ( is_orig )
s_aead_tag = fmt_seq(c_seq);
else
s_aead_tag = fmt_seq(s_seq);
s_aead_tag[8] = content_type;
s_aead_tag[9] = MSB(raw_tls_version);
s_aead_tag[10] = LSB(raw_tls_version);
s_aead_tag[11] = MSB(encrypted_len);
s_aead_tag[12] = LSB(encrypted_len);
assert(s_aead_tag.size() == 13);
auto decrypted = std::vector<u_char>(
encrypted_len +
16); // see OpenSSL manpage - 16 is the block size for the supported cipher
int decrypted_len = 0;
EVP_DecryptUpdate(ctx, NULL, &decrypted_len, s_aead_tag.data(), s_aead_tag.size());
EVP_DecryptUpdate(ctx, decrypted.data(), &decrypted_len, (const u_char*)encrypted,
encrypted_len);
assert(decrypted_len <= decrypted.size());
decrypted.resize(decrypted_len);
int res = 0;
if ( ! (res = EVP_DecryptFinal(ctx, NULL, &res)) )
{
DBG_LOG(DBG_ANALYZER, "Decryption failed with return code: %d. Invalid key?\n", res);
EVP_CIPHER_CTX_free(ctx);
return false;
}
DBG_LOG(DBG_ANALYZER, "Successfully decrypted %d bytes.", decrypted_len);
EVP_CIPHER_CTX_free(ctx);
ForwardDecryptedData(decrypted, is_orig);
return true;
}
// This is only reached if key derivation fails or is unsupported
return false;
}
void SSL_Analyzer::ForwardDecryptedData(const std::vector<u_char>& data, bool is_orig)
{
if ( ! pia )
{
pia = new analyzer::pia::PIA_TCP(Conn());
if ( AddChildAnalyzer(pia) )
{
pia->FirstPacket(true, nullptr);
pia->FirstPacket(false, nullptr);
}
else
reporter->Error("Could not initialize PIA");
}
ForwardStream(data.size(), data.data(), is_orig);
}
} // namespace zeek::analyzer::ssl } // namespace zeek::analyzer::ssl

117
src/analyzer/protocol/ssl/SSL.h
View file

@ -1,5 +1,6 @@
#pragma once #pragma once
#include "zeek/analyzer/protocol/pia/PIA.h"
#include "zeek/analyzer/protocol/ssl/events.bif.h" #include "zeek/analyzer/protocol/ssl/events.bif.h"
#include "zeek/analyzer/protocol/tcp/TCP.h" #include "zeek/analyzer/protocol/tcp/TCP.h"
@ -24,6 +25,9 @@ namespace zeek::analyzer::ssl
class SSL_Analyzer final : public analyzer::tcp::TCP_ApplicationAnalyzer class SSL_Analyzer final : public analyzer::tcp::TCP_ApplicationAnalyzer
{ {
// let binpac forward encrypted TLS application data to us.
friend class binpac::SSL::SSL_Conn;
public: public:
explicit SSL_Analyzer(Connection* conn); explicit SSL_Analyzer(Connection* conn);
~SSL_Analyzer() override; ~SSL_Analyzer() override;
@ -45,10 +49,123 @@ public:
static analyzer::Analyzer* Instantiate(Connection* conn) { return new SSL_Analyzer(conn); } static analyzer::Analyzer* Instantiate(Connection* conn) { return new SSL_Analyzer(conn); }
/**
* Set the secret that should be used to derive keys for the
* connection. (For TLS 1.2 this is the pre-master secret)
*
* Please note that these functions currently are hardcoded to only work with a single TLS 1.2
* ciphersuite (TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384).
*
* @param secret The secret to set
*/
void SetSecret(const StringVal& secret);
/**
* Set the secret that should be used to derive keys for the
* connection. (For TLS 1.2 this is the pre-master secret)
*
* Please note that these functions currently are hardcoded to only work with a single TLS 1.2
* cuphersuite (TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384).
*
* @param len Length of the secret bytes
*
* @param data Pointer to the secret bytes
*/
void SetSecret(size_t len, const u_char* data);
/**
* Set the decryption keys that should be used to decrypt
* TLS application data in the connection.
*
* Please note that these functions currently are hardcoded to only work with a single TLS 1.2
* cuphersuite (TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384).
*
* @param keys The key buffer as derived via TLS PRF (for
* AES_GCM this should be 72 bytes in length)
*/
void SetKeys(const StringVal& keys);
/**
* Set the decryption keys that should be used to decrypt
* TLS application data in the connection.
*
* Please note that these functions currently are hardcoded to only work with a single TLS 1.2
* cuphersuite (TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384).
*
* @param keys The key buffer as derived via TLS PRF (for
* AES_GCM this should be 72 bytes in length)
*/
void SetKeys(const std::vector<u_char> newkeys);
protected: protected:
/**
* Try to decrypt TLS application data from a packet. Requires secret or keys to be set prior.
*
* Please note that these functions currently are hardcoded to only work with a single TLS 1.2
* cuphersuite (TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384).
*
* @param len Length of the encrypted bytes to decrypt
*
* @param data Pointer to the encrypted bytes to decrypt
*
* @param is_orig Direction of the connection
*
* @param content_type Content type as given in the TLS packet
*
* @param raw_tls_version Raw TLS version as given in the TLS packets
*
* @return True if decryption succeeded and data was forwarded.
*/
bool TryDecryptApplicationData(int len, const u_char* data, bool is_orig, uint8_t content_type,
uint16_t raw_tls_version);
/**
* TLS 1.2 pseudo random function (PRF) used to expand the pre-master secret and derive keys.
* The seed is obtained by concatinating rnd1 and rnd2.
*
* Please note that these functions currently are hardcoded to only work with a single TLS 1.2
* cuphersuite (TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384).
*
* @param secret Secret as defined in the TLS RFC
*
* @param label Label as defined in the TLS RFC
*
* @param First part of the seed
*
* @param rnd2 Second part of the seed
*
* @param rnd2_len Length of the second part of the seed
*
* @param requested_len Length indicating how many bytes should be derived
*
* @return The derived bytes, if the operation succeeds.
*/
std::optional<std::vector<u_char>> TLS12_PRF(const std::string& secret,
const std::string& label, const std::string& rnd1,
const std::string& rnd2, size_t requested_len);
/**
* Forward decrypted TLS application data to child analyzers.
*
* @param data Data to forward
*
* @param is_orig Direction of the connection
*/
void ForwardDecryptedData(const std::vector<u_char>& data, bool is_orig);
binpac::SSL::SSL_Conn* interp; binpac::SSL::SSL_Conn* interp;
binpac::TLSHandshake::Handshake_Conn* handshake_interp; binpac::TLSHandshake::Handshake_Conn* handshake_interp;
bool had_gap; bool had_gap;
// client and server sequence number, used for TLS 1.2 decryption
int c_seq;
int s_seq;
// secret, for decyption
std::string secret;
// derived keys, for decryption
std::vector<u_char> keys;
// PIA, for decrypted data
zeek::analyzer::pia::PIA_TCP* pia;
}; };
} // namespace zeek::analyzer::ssl } // namespace zeek::analyzer::ssl

53
src/analyzer/protocol/ssl/functions.bif
View file

@ -9,10 +9,59 @@
## finished succesfully). ## finished succesfully).
## ##
## c: The SSL connection. ## c: The SSL connection.
function set_ssl_established%(c: connection%): any ##
## Returns: T on success, F on failure.
function set_ssl_established%(c: connection%): bool
%{ %{
zeek::analyzer::Analyzer* sa = c->FindAnalyzer("SSL"); zeek::analyzer::Analyzer* sa = c->FindAnalyzer("SSL");
if ( sa ) if ( sa )
{
static_cast<zeek::analyzer::ssl::SSL_Analyzer*>(sa)->StartEncryption(); static_cast<zeek::analyzer::ssl::SSL_Analyzer*>(sa)->StartEncryption();
return nullptr; return zeek::val_mgr->True();
}
return zeek::val_mgr->False();
%}
## Set the secret that should be used to derive keys for the connection.
## (For TLS 1.2 this is the pre-master secret).
##
## c: The affected connection
##
## secret: secret to set
##
## Returns: T on success, F on failure.
function set_secret%(c: connection, secret: string%): bool
%{
analyzer::Analyzer* sa = c->FindAnalyzer("SSL");
if ( sa )
{
static_cast<zeek::analyzer::ssl::SSL_Analyzer*>(sa)->SetSecret(*secret);
return zeek::val_mgr->True();
}
return zeek::val_mgr->False();
%}
## Set the decryption keys that should be used to decrypt
## TLS application data in the connection.
##
## c: The affected connection
##
## keys: The key buffer as derived via TLS PRF.
##
## Returns: T on success, F on failure.
function set_keys%(c: connection, keys: string%): bool
%{
analyzer::Analyzer* sa = c->FindAnalyzer("SSL");
if ( sa )
{
static_cast<zeek::analyzer::ssl::SSL_Analyzer*>(sa)->SetKeys(*keys);
return zeek::val_mgr->True();
}
return zeek::val_mgr->False();
%} %}

54
src/analyzer/protocol/ssl/proc-client-hello.pac
View file

@ -1,54 +0,0 @@
function proc_client_hello(
version : uint16, ts : double,
client_random : bytestring,
session_id : uint8[],
cipher_suites16 : uint16[],
cipher_suites24 : uint24[],
compression_methods: uint8[]) : bool
%{
if ( ! version_ok(version) )
{
zeek_analyzer()->AnalyzerViolation(zeek::util::fmt("unsupported client SSL version 0x%04x", version));
zeek_analyzer()->SetSkip(true);
}
else
zeek_analyzer()->AnalyzerConfirmation();
if ( ssl_client_hello )
{
vector<int> cipher_suites;
if ( cipher_suites16 )
std::copy(cipher_suites16->begin(), cipher_suites16->end(), std::back_inserter(cipher_suites));
else
std::transform(cipher_suites24->begin(), cipher_suites24->end(), std::back_inserter(cipher_suites), to_int());
auto cipher_vec = zeek::make_intrusive<zeek::VectorVal>(zeek::id::index_vec);
for ( unsigned int i = 0; i < cipher_suites.size(); ++i )
{
auto ciph = zeek::val_mgr->Count(cipher_suites[i]);
cipher_vec->Assign(i, ciph);
}
auto comp_vec = zeek::make_intrusive<zeek::VectorVal>(zeek::id::index_vec);
if ( compression_methods )
{
for ( unsigned int i = 0; i < compression_methods->size(); ++i )
{
auto comp = zeek::val_mgr->Count((*compression_methods)[i]);
comp_vec->Assign(i, comp);
}
}
zeek::BifEvent::enqueue_ssl_client_hello(zeek_analyzer(), zeek_analyzer()->Conn(),
version, record_version(), ts,
zeek::make_intrusive<zeek::StringVal>(client_random.length(),
(const char*) client_random.data()),
{zeek::AdoptRef{}, to_string_val(session_id)},
std::move(cipher_vec), std::move(comp_vec));
}
return true;
%}

40
src/analyzer/protocol/ssl/proc-server-hello.pac
View file

@ -1,40 +0,0 @@
function proc_server_hello(
version : uint16, v2 : bool,
server_random : bytestring,
session_id : uint8[],
cipher_suites16 : uint16[],
cipher_suites24 : uint24[],
comp_method : uint8) : bool
%{
if ( ! version_ok(version) )
{
zeek_analyzer()->AnalyzerViolation(zeek::util::fmt("unsupported server SSL version 0x%04x", version));
zeek_analyzer()->SetSkip(true);
}
if ( ssl_server_hello )
{
vector<int>* ciphers = new vector<int>();
if ( cipher_suites16 )
std::copy(cipher_suites16->begin(), cipher_suites16->end(), std::back_inserter(*ciphers));
else
std::transform(cipher_suites24->begin(), cipher_suites24->end(), std::back_inserter(*ciphers), to_int());
uint32 ts = 0;
if ( v2 == 0 && server_random.length() >= 4 )
ts = ntohl(*((uint32*)server_random.data()));
zeek::BifEvent::enqueue_ssl_server_hello(zeek_analyzer(),
zeek_analyzer()->Conn(),
version, record_version(), ts,
zeek::make_intrusive<zeek::StringVal>(server_random.length(),
(const char*) server_random.data()),
{zeek::AdoptRef{}, to_string_val(session_id)},
ciphers->size()==0 ? 0 : ciphers->at(0), comp_method);
delete ciphers;
}
return true;
%}

98
src/analyzer/protocol/ssl/ssl-analyzer.pac
View file

@ -2,10 +2,104 @@
refine connection SSL_Conn += { refine connection SSL_Conn += {
%include proc-client-hello.pac
%include proc-server-hello.pac
%include proc-certificate.pac %include proc-certificate.pac
function proc_client_hello(
version : uint16, ts : double,
client_random : bytestring,
session_id : uint8[],
cipher_suites16 : uint16[],
cipher_suites24 : uint24[],
compression_methods: uint8[]) : bool
%{
if ( ! version_ok(version) )
{
zeek_analyzer()->AnalyzerViolation(zeek::util::fmt("unsupported client SSL version 0x%04x", version));
zeek_analyzer()->SetSkip(true);
}
else
zeek_analyzer()->AnalyzerConfirmation();
if ( ssl_client_hello )
{
vector<int> cipher_suites;
if ( cipher_suites16 )
std::copy(cipher_suites16->begin(), cipher_suites16->end(), std::back_inserter(cipher_suites));
else
std::transform(cipher_suites24->begin(), cipher_suites24->end(), std::back_inserter(cipher_suites), to_int());
auto cipher_vec = zeek::make_intrusive<zeek::VectorVal>(zeek::id::index_vec);
for ( unsigned int i = 0; i < cipher_suites.size(); ++i )
{
auto ciph = zeek::val_mgr->Count(cipher_suites[i]);
cipher_vec->Assign(i, ciph);
}
auto comp_vec = zeek::make_intrusive<zeek::VectorVal>(zeek::id::index_vec);
if ( compression_methods )
{
for ( unsigned int i = 0; i < compression_methods->size(); ++i )
{
auto comp = zeek::val_mgr->Count((*compression_methods)[i]);
comp_vec->Assign(i, comp);
}
}
zeek::BifEvent::enqueue_ssl_client_hello(zeek_analyzer(), zeek_analyzer()->Conn(),
version, record_version(), ts,
zeek::make_intrusive<zeek::StringVal>(client_random.length(),
(const char*) client_random.data()),
{zeek::AdoptRef{}, to_string_val(session_id)},
std::move(cipher_vec), std::move(comp_vec));
}
return true;
%}
function proc_server_hello(
version : uint16, v2 : bool,
server_random : bytestring,
session_id : uint8[],
cipher_suites16 : uint16[],
cipher_suites24 : uint24[],
comp_method : uint8) : bool
%{
if ( ! version_ok(version) )
{
zeek_analyzer()->AnalyzerViolation(zeek::util::fmt("unsupported server SSL version 0x%04x", version));
zeek_analyzer()->SetSkip(true);
}
if ( ssl_server_hello )
{
vector<int>* ciphers = new vector<int>();
if ( cipher_suites16 )
std::copy(cipher_suites16->begin(), cipher_suites16->end(), std::back_inserter(*ciphers));
else
std::transform(cipher_suites24->begin(), cipher_suites24->end(), std::back_inserter(*ciphers), to_int());
uint32 ts = 0;
if ( v2 == 0 && server_random.length() >= 4 )
ts = ntohl(*((uint32*)server_random.data()));
zeek::BifEvent::enqueue_ssl_server_hello(zeek_analyzer(),
zeek_analyzer()->Conn(),
version, record_version(), ts,
zeek::make_intrusive<zeek::StringVal>(server_random.length(),
(const char*) server_random.data()),
{zeek::AdoptRef{}, to_string_val(session_id)},
ciphers->size()==0 ? 0 : ciphers->at(0), comp_method);
delete ciphers;
}
return true;
%}
function proc_v2_certificate(is_orig: bool, cert : bytestring) : bool function proc_v2_certificate(is_orig: bool, cert : bytestring) : bool
%{ %{
vector<bytestring>* cert_list = new vector<bytestring>(1,cert); vector<bytestring>* cert_list = new vector<bytestring>(1,cert);

15
src/analyzer/protocol/ssl/ssl-dtls-analyzer.pac
View file

@ -13,10 +13,12 @@ refine connection SSL_Conn += {
%member{ %member{
int established_; int established_;
int decryption_failed_;
%} %}
%init{ %init{
established_ = false; established_ = false;
decryption_failed_ = false;
%} %}
%cleanup{ %cleanup{
@ -43,7 +45,7 @@ refine connection SSL_Conn += {
return true; return true;
%} %}
function proc_ciphertext_record(rec : SSLRecord) : bool function proc_ciphertext_record(rec : SSLRecord, cont: const_bytestring) : bool
%{ %{
if ( established_ == false && determine_tls13() == 1 ) if ( established_ == false && determine_tls13() == 1 )
{ {
@ -62,8 +64,17 @@ refine connection SSL_Conn += {
} }
if ( ssl_encrypted_data ) if ( ssl_encrypted_data )
{
zeek::BifEvent::enqueue_ssl_encrypted_data(zeek_analyzer(), zeek::BifEvent::enqueue_ssl_encrypted_data(zeek_analyzer(),
zeek_analyzer()->Conn(), ${rec.is_orig}, ${rec.raw_tls_version}, ${rec.content_type}, ${rec.length}); zeek_analyzer()->Conn(), ${rec.is_orig}, ${rec.raw_tls_version}, ${rec.content_type}, ${rec.length});
}
if ( rec->content_type() == APPLICATION_DATA && decryption_failed_ == false )
{
// If decryption of one packet fails, do not try to decrypt future packets.
if ( ! zeek_analyzer()->TryDecryptApplicationData(cont.length(), cont.begin(), rec->is_orig(), rec->content_type(), rec->raw_tls_version()) )
decryption_failed_ = true;
}
return true; return true;
%} %}
@ -123,7 +134,7 @@ refine typeattr UnknownRecord += &let {
}; };
refine typeattr CiphertextRecord += &let { refine typeattr CiphertextRecord += &let {
proc : bool = $context.connection.proc_ciphertext_record(rec); proc : bool = $context.connection.proc_ciphertext_record(rec, cont);
} }
refine typeattr PlaintextRecord += &let { refine typeattr PlaintextRecord += &let {

101
src/analyzer/protocol/ssl/tls-handshake-analyzer.pac
View file

@ -25,10 +25,107 @@
refine connection Handshake_Conn += { refine connection Handshake_Conn += {
%include proc-client-hello.pac
%include proc-server-hello.pac
%include proc-certificate.pac %include proc-certificate.pac
function proc_client_hello(
version : uint16, ts : double,
client_random : bytestring,
session_id : uint8[],
cipher_suites16 : uint16[],
cipher_suites24 : uint24[],
compression_methods: uint8[]) : bool
%{
if ( ! version_ok(version) )
{
zeek_analyzer()->AnalyzerViolation(zeek::util::fmt("unsupported client SSL version 0x%04x", version));
zeek_analyzer()->SetSkip(true);
}
else
zeek_analyzer()->AnalyzerConfirmation();
if ( ssl_client_hello )
{
vector<int> cipher_suites;
if ( cipher_suites16 )
std::copy(cipher_suites16->begin(), cipher_suites16->end(), std::back_inserter(cipher_suites));
else
std::transform(cipher_suites24->begin(), cipher_suites24->end(), std::back_inserter(cipher_suites), to_int());
auto cipher_vec = zeek::make_intrusive<zeek::VectorVal>(zeek::id::index_vec);
for ( unsigned int i = 0; i < cipher_suites.size(); ++i )
{
auto ciph = zeek::val_mgr->Count(cipher_suites[i]);
cipher_vec->Assign(i, ciph);
}
auto comp_vec = zeek::make_intrusive<zeek::VectorVal>(zeek::id::index_vec);
if ( compression_methods )
{
for ( unsigned int i = 0; i < compression_methods->size(); ++i )
{
auto comp = zeek::val_mgr->Count((*compression_methods)[i]);
comp_vec->Assign(i, comp);
}
}
set_client_random(client_random);
set_gmt_unix_time(ts);
zeek::BifEvent::enqueue_ssl_client_hello(zeek_analyzer(), zeek_analyzer()->Conn(),
version, record_version(), ts,
zeek::make_intrusive<zeek::StringVal>(client_random.length(),
(const char*) client_random.data()),
{zeek::AdoptRef{}, to_string_val(session_id)},
std::move(cipher_vec), std::move(comp_vec));
}
return true;
%}
function proc_server_hello(
version : uint16, v2 : bool,
server_random : bytestring,
session_id : uint8[],
cipher_suites16 : uint16[],
cipher_suites24 : uint24[],
comp_method : uint8) : bool
%{
if ( ! version_ok(version) )
{
zeek_analyzer()->AnalyzerViolation(zeek::util::fmt("unsupported server SSL version 0x%04x", version));
zeek_analyzer()->SetSkip(true);
}
if ( ssl_server_hello )
{
vector<int>* ciphers = new vector<int>();
if ( cipher_suites16 )
std::copy(cipher_suites16->begin(), cipher_suites16->end(), std::back_inserter(*ciphers));
else
std::transform(cipher_suites24->begin(), cipher_suites24->end(), std::back_inserter(*ciphers), to_int());
uint32 ts = 0;
if ( v2 == 0 && server_random.length() >= 4 )
ts = ntohl(*((uint32*)server_random.data()));
set_server_random(server_random);
zeek::BifEvent::enqueue_ssl_server_hello(zeek_analyzer(),
zeek_analyzer()->Conn(),
version, record_version(), ts,
zeek::make_intrusive<zeek::StringVal>(server_random.length(),
(const char*) server_random.data()),
{zeek::AdoptRef{}, to_string_val(session_id)},
ciphers->size()==0 ? 0 : ciphers->at(0), comp_method);
delete ciphers;
}
return true;
%}
function proc_session_ticket_handshake(rec: SessionTicketHandshake, is_orig: bool): bool function proc_session_ticket_handshake(rec: SessionTicketHandshake, is_orig: bool): bool
%{ %{
if ( ssl_session_ticket_handshake ) if ( ssl_session_ticket_handshake )

35
src/analyzer/protocol/ssl/tls-handshake-protocol.pac
View file

@ -943,6 +943,9 @@ refine connection Handshake_Conn += {
uint32 chosen_cipher_; uint32 chosen_cipher_;
uint16 chosen_version_; uint16 chosen_version_;
uint16 record_version_; uint16 record_version_;
bytestring client_random_;
bytestring server_random_;
uint32 gmt_unix_time_;
%} %}
%init{ %init{
@ -950,6 +953,12 @@ refine connection Handshake_Conn += {
chosen_version_ = UNKNOWN_VERSION; chosen_version_ = UNKNOWN_VERSION;
record_version_ = 0; record_version_ = 0;
gmt_unix_time_ = 0;
%}
%cleanup{
client_random_.free();
server_random_.free();
%} %}
function chosen_cipher() : int %{ return chosen_cipher_; %} function chosen_cipher() : int %{ return chosen_cipher_; %}
@ -983,5 +992,31 @@ refine connection Handshake_Conn += {
record_version_ = version; record_version_ = version;
return true; return true;
%} %}
function client_random() : bytestring %{ return client_random_; %}
function set_client_random(client_random: bytestring) : bool
%{
client_random_.free();
client_random_.init(client_random.data(), client_random.length());
return true;
%}
function server_random() : bytestring %{ return server_random_; %}
function set_server_random(server_random: bytestring) : bool
%{
server_random_.free();
server_random_.init(server_random.data(), server_random.length());
return true;
%}
function gmt_unix_time() : uint32 %{ return gmt_unix_time_; %}
function set_gmt_unix_time(ts: uint32) : bool
%{
gmt_unix_time_ = ts;
return true;
%}
}; };

4
testing/btest/Baseline/coverage.bare-mode-errors/errors
View file

@ -1,9 +1,9 @@
### BTest baseline data generated by btest-diff. Do not edit. Use "btest -U/-u" to update. Requires BTest >= 0.63. ### BTest baseline data generated by btest-diff. Do not edit. Use "btest -U/-u" to update. Requires BTest >= 0.63.
### NOTE: This file has been sorted with diff-sort. ### NOTE: This file has been sorted with diff-sort.
warning in <...>/extract-certs-pem.zeek, line 1: deprecated script loaded from <...>/__load__.zeek:12 "Remove in v5.1. Use log-certs-base64.zeek instead." warning in <...>/extract-certs-pem.zeek, line 1: deprecated script loaded from <...>/__load__.zeek:13 "Remove in v5.1. Use log-certs-base64.zeek instead."
warning in <...>/extract-certs-pem.zeek, line 1: deprecated script loaded from command line arguments "Remove in v5.1. Use log-certs-base64.zeek instead." warning in <...>/extract-certs-pem.zeek, line 1: deprecated script loaded from command line arguments "Remove in v5.1. Use log-certs-base64.zeek instead."
warning in <...>/log-ocsp.zeek, line 1: deprecated script loaded from <...>/test-all-policy.zeek:61 ("Remove in v5.1. OCSP logging is now enabled by default") warning in <...>/log-ocsp.zeek, line 1: deprecated script loaded from <...>/test-all-policy.zeek:61 ("Remove in v5.1. OCSP logging is now enabled by default")
warning in <...>/log-ocsp.zeek, line 1: deprecated script loaded from <...>/test-all-policy.zeek:61 ("Remove in v5.1. OCSP logging is now enabled by default") warning in <...>/log-ocsp.zeek, line 1: deprecated script loaded from <...>/test-all-policy.zeek:61 ("Remove in v5.1. OCSP logging is now enabled by default")
warning in <...>/log-ocsp.zeek, line 1: deprecated script loaded from command line arguments ("Remove in v5.1. OCSP logging is now enabled by default") warning in <...>/log-ocsp.zeek, line 1: deprecated script loaded from command line arguments ("Remove in v5.1. OCSP logging is now enabled by default")
warning in <...>/notary.zeek, line 1: deprecated script loaded from <...>/__load__.zeek:4 ("Remove in v5.1. Please switch to other more modern approaches like SCT validation (validate-sct.zeek).") warning in <...>/notary.zeek, line 1: deprecated script loaded from <...>/__load__.zeek:5 ("Remove in v5.1. Please switch to other more modern approaches like SCT validation (validate-sct.zeek).")
warning in <...>/notary.zeek, line 1: deprecated script loaded from command line arguments ("Remove in v5.1. Please switch to other more modern approaches like SCT validation (validate-sct.zeek).") warning in <...>/notary.zeek, line 1: deprecated script loaded from command line arguments ("Remove in v5.1. Please switch to other more modern approaches like SCT validation (validate-sct.zeek).")

11
testing/btest/Baseline/scripts.policy.protocols.ssl.decryption/http.log Normal file
View file

@ -0,0 +1,11 @@
### BTest baseline data generated by btest-diff. Do not edit. Use "btest -U/-u" to update. Requires BTest >= 0.63.
#separator \x09
#set_separator ,
#empty_field (empty)
#unset_field -
#path http
#open XXXX-XX-XX-XX-XX-XX
#fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p trans_depth method host uri referrer version user_agent origin request_body_len response_body_len status_code status_msg info_code info_msg tags username password proxied orig_fuids orig_filenames orig_mime_types resp_fuids resp_filenames resp_mime_types
#types time string addr port addr port count string string string string string string string count count count string count string set[enum] string string set[string] vector[string] vector[string] vector[string] vector[string] vector[string] vector[string]
XXXXXXXXXX.XXXXXX CHhAvVGS1DHFjwGM9 192.168.100.70 48216 193.99.144.80 443 1 GET heise.de / - 1.1 Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Firefox/78.0 - 0 229 301 Moved Permanently - - (empty) - - - - - - FaOfPL638bbaQ1KMh - text/html
#close XXXX-XX-XX-XX-XX-XX

BIN
testing/btest/Traces/tls/tls-1.2-stream-keylog.pcap Normal file
View file

Binary file not shown.

BIN
testing/btest/Traces/tls/tls12-decryption.pcap Normal file
View file

Binary file not shown.

42
testing/btest/scripts/policy/protocols/ssl/decryption-keylog.zeek Normal file
View file

@ -0,0 +1,42 @@
# @TEST-REQUIRES: grep -q "#define OPENSSL_HAVE_KDF_H" $BUILD/zeek-config.h
# @TEST-EXEC: ZEEK_TLS_KEYLOG_FILE=keylogfile.log zeek -B dpd -C -r $TRACES/tls/tls-1.2-stream-keylog.pcap %INPUT
@TEST-START-FILE keylogfile.log
#fields client_random secret
\x0e\x78\x2d\x35\x63\x95\x5d\x8a\x30\xa9\xcf\xb6\x4f\x47\xf3\x96\x34\x8a\x1e\x79\x1a\xa2\x32\x55\xe2\x2f\xc5\x7a \x34\x4f\x12\x65\xbf\x43\x40\xb3\x61\x6b\xa0\x16\x5d\x2b\x4d\xb9\xb1\xe8\x4a\x3d\xa2\x42\x0e\x38\xab\x01\x50\x62\x84\xcc\x34\xcd\xe0\x34\x10\xfe\x1a\x02\x30\x49\x74\x6c\x46\x43\xa7\x0c\x67\x0d
\x24\x8c\x7e\x24\xee\xfb\x13\xcd\xee\xde\xb1\xf4\xb6\xd6\xd5\xee\x67\x8d\xd3\xff\xc7\xe7\x39\x23\x18\x3f\x99\xb4 \xe7\xed\x24\x26\x0d\x25\xd9\xfd\xf5\x0f\xc0\xf4\x56\x51\x0e\x4e\xec\x7f\x58\x9c\xaf\x39\x25\x14\x16\xa6\x71\xdd\xea\xfe\xe9\xc0\x93\xbe\x89\x4c\xab\xcc\xff\xb2\xf0\x9a\xea\x98\xf5\xb2\x53\x1e
\x57\xd7\xc7\x7a\x2d\x5e\x35\x29\x2c\xd7\xe7\x94\xee\xf8\x6f\x31\x45\xf6\xbe\x25\x08\xed\x1d\x92\xd2\x0b\x9b\x04 \xc1\x93\x17\x93\xd9\x7d\xd2\x98\xb3\xe0\xdb\x2c\x5d\xbe\x71\x31\xa7\x9a\xf5\x91\xf9\x87\x90\xee\xb7\x79\x9f\x6b\xb4\x1f\x47\xa7\x69\x62\x4b\xa3\x99\x0c\xa9\x43\xf9\xea\x3b\x4d\x5f\x2f\xfe\xfb
\x30\xd7\xb8\x92\xc1\xec\x17\x90\x5b\x0f\xcb\xda\xe6\x42\xb2\x09\x4c\xdd\x7d\x2e\xa1\x9f\x1a\x3b\x70\x23\x7d\xf2 \xc1\x93\x17\x93\xd9\x7d\xd2\x98\xb3\xe0\xdb\x2c\x5d\xbe\x71\x31\xa7\x9a\xf5\x91\xf9\x87\x90\xee\xb7\x79\x9f\x6b\xb4\x1f\x47\xa7\x69\x62\x4b\xa3\x99\x0c\xa9\x43\xf9\xea\x3b\x4d\x5f\x2f\xfe\xfb
\x49\xc7\x71\x25\xdc\xb0\xa7\xbc\xd6\xb6\x67\x5c\x30\x58\x8d\xad\x47\x5a\x93\x60\xac\xa5\x78\xf5\x62\x7e\xff\x62 \xc1\x93\x17\x93\xd9\x7d\xd2\x98\xb3\xe0\xdb\x2c\x5d\xbe\x71\x31\xa7\x9a\xf5\x91\xf9\x87\x90\xee\xb7\x79\x9f\x6b\xb4\x1f\x47\xa7\x69\x62\x4b\xa3\x99\x0c\xa9\x43\xf9\xea\x3b\x4d\x5f\x2f\xfe\xfb
\x38\x1c\x49\xcc\xf9\x62\xd0\x5c\xf0\xd4\xe2\xd5\xa1\x15\xc1\x5e\x8d\x02\xcc\x50\xed\x6c\x90\x63\x73\x9d\xfb\x96 \xdc\xf5\xfc\x10\xf2\xb3\x8b\xd8\x87\xae\xcf\xb5\xcd\x1a\xe3\xa8\x06\x8e\x85\xfc\xbb\xfc\x22\xec\x0f\x79\x99\x04\x13\x5b\x6b\x03\x52\x02\xee\xe9\x04\x59\x78\x44\xf1\xf3\xc8\xac\x22\x68\x6c\x7e
\x61\x9e\x08\x51\xee\x36\x3c\x2c\xf3\x71\x87\x22\x82\x27\xca\x4e\x68\x0f\x9a\x7c\x0b\xd1\x50\x69\xaa\x7a\x59\x70 \xad\x03\xce\xda\x48\x90\xfa\x58\x1e\x98\x9f\x5e\x38\x62\x02\x3e\x2a\x4e\x3e\x8a\xd8\x13\x25\x23\x8d\x90\x80\x66\xe1\xd3\x5c\xc8\x75\x97\x9e\x34\xc0\x8e\x6f\xdf\xd9\xd8\xc6\xf3\x56\xe3\x85\xc1
\xcb\x3f\x93\xd2\x55\xcb\xb6\x56\x25\x87\xf0\xdd\x01\x02\x12\xfd\xee\x9d\x23\x3a\xff\x64\xe6\xed\x36\xcd\x5c\x45 \x0d\x36\xfa\xaa\x2e\xad\xbd\xa2\xa8\x09\x5f\x95\x1d\xe1\xcb\xac\x46\xb8\x1b\x00\x8f\xbf\x39\x1d\x91\x95\x1b\x34\x85\x47\x6b\xab\x73\x28\x8a\x1e\x17\xcd\x0c\xe8\x0e\x0f\xc0\x40\x1d\xbe\x9e\x3f
\xf9\x7e\x7d\x38\x56\xe2\xfc\xcb\xbe\x80\x79\x8e\xc2\xe3\xf5\x15\x25\x10\x82\xad\x63\xbb\xc7\xc2\x31\xd8\xbe\xe0 \x9a\x7c\xf9\x46\xa0\x47\x18\xa1\x9f\x4d\x20\xc3\xf8\x0c\x1c\xf8\xc8\x23\xc3\xe2\xb1\xc3\x37\xef\x64\x32\x2d\x75\x1b\x41\x05\x43\x31\x5f\x6e\xcf\x7d\xbf\x45\xec\x9b\xe1\x94\xa3\xcc\x7c\x1a\x0f
\x57\x97\x63\x67\xf2\xea\x9c\x95\x46\x7a\x6c\xc5\x59\xda\x6f\xeb\xbc\x44\x2e\x11\x3a\xc5\xea\xa7\xed\x97\xad\x38 \x0e\x5e\xc0\x6c\xa5\x4e\xe3\x86\x05\x5a\xaa\x97\x1c\x7e\x09\x39\xba\x3e\x1f\xb1\x62\x4d\x0a\x5b\x9c\x0c\xae\x97\x5f\x0e\x25\xbc\x4c\x51\x21\xfa\x34\x5e\xa1\x26\x47\xc4\x7a\x5a\x1c\xe5\xbd\xce
\x70\x18\x17\x27\xd6\xe2\x04\xd1\xd8\xa5\xb8\x2a\x05\x01\xaf\x7b\x13\x6d\x3a\x9c\x56\x6c\x32\x5b\x3f\xef\xb5\x04 \x92\x3d\x8a\x93\xba\xc5\x54\xc1\x04\x9a\x8d\xeb\x63\x28\x8c\xd7\x4d\x60\x51\xb0\x7a\x10\x67\x84\x8d\xac\x15\xc8\x75\xf2\x5c\x2a\x60\xe3\x38\xde\xb3\x27\x37\x44\xb1\x53\xe6\x9d\x42\x06\x0e\x18
\x4f\x12\x67\xb1\x13\xdc\x1a\x3e\x5d\xee\xbf\xff\xa7\x4d\xaa\xa1\x96\xff\x43\x0a\x30\xbe\x04\x07\x60\x29\x5f\x5e \x1d\x61\x52\xa6\x1e\x86\x75\x53\x04\xb8\x8e\x12\x6f\xdb\xa4\x49\x05\xeb\x5e\x4b\x33\xf6\xaf\xee\x67\x20\x37\xfd\x84\x48\x9a\xaa\x62\xa6\xb2\x64\x0f\x62\x87\x12\xe8\x05\x98\xae\x0c\xbf\xae\x5f
\xfe\x13\x61\x60\x80\x41\x0b\x9d\xc2\xcc\xc2\xc3\x00\xab\x20\x6b\xb8\x43\xc4\xc4\x22\x81\x1f\x15\xd4\xed\x34\xc3 \x39\xfb\x4d\x9c\x1d\xff\x4d\xe4\x1c\x86\xf9\x67\x9b\x32\xca\xa3\x99\x9c\x91\xcd\x7a\xf5\x4d\xc5\x58\x98\x1c\xcf\xf6\xd9\xa7\x4c\x92\x6e\x93\x7f\x98\x02\x96\x22\x20\x52\x5e\x9d\xe0\xec\x4a\xc1
\x92\xc2\x33\xdd\xf3\xf4\x31\xd6\x0c\x9b\x90\x86\x6a\xde\x5d\x80\x32\x22\xb8\x18\x45\xf5\x11\x72\xa0\x4f\xe9\x65 \xda\x22\x06\x86\xef\x25\x99\xb4\x65\x2c\x45\x94\x73\xcd\xe9\xc6\x64\x55\x84\x21\x42\x35\x86\x57\x9a\x60\xd4\xc7\x88\xd8\x1b\x3a\xbe\xdf\x53\x7b\xd7\x9c\xf9\x29\x47\x05\x07\x0f\x23\x3b\x22\xc4
\x39\x8e\xeb\xdf\x69\xd9\xe3\xe2\xce\xd8\xe9\xb2\x93\xa6\xb7\x58\x30\x9b\xaf\x14\x98\xbd\x27\xa0\xe1\x12\x54\x3f \xa9\xcc\x51\xa6\x83\xf1\xbb\x6b\x37\xf0\xe2\x8b\xa5\xea\x31\xc8\xdc\x19\x5e\xb1\xaf\xa0\x5c\x51\xa1\x4a\x73\x22\xc0\x24\xf1\x41\x4a\xd9\x15\x16\xa8\x83\x38\x84\xe1\xca\x9d\xf0\xd5\x35\x40\x73
\xdc\xf5\x87\xb0\x6d\x66\xd6\xab\x66\x34\xd7\x64\xc8\x51\xa1\x22\xe3\x97\x3d\x4b\x16\xee\x8e\x1e\x0b\xfb\xfc\x13 \xd5\xaf\x0d\xed\x74\x58\x8d\xe8\x97\x6d\xa0\xb2\x46\x83\x58\x0f\x52\xbc\xc7\x66\xb1\x19\x74\x70\x0d\xaa\xd1\x10\x9b\x71\x53\xe6\x80\x34\x5d\x81\xd2\x86\x8a\x33\xfc\x62\x88\xa7\x80\xac\x63\xb6
\x51\xcb\xcc\x61\xae\xd0\xeb\x08\x75\x09\xde\x68\x3c\x36\x03\xf5\xa3\xd5\xa5\x15\xdc\x3e\x87\xdb\xcf\xc7\x7a\x1e \x25\x90\xa9\x7e\x5a\x93\xe9\xdd\x61\x6c\x46\xf2\xf6\x03\x7c\x19\xb1\xf5\x9a\x4a\x6c\x58\x71\x8e\xfe\xa4\xfe\xa6\x30\x70\x5f\xaf\xd4\xf9\xb9\x3a\x16\xa8\x0f\x69\x8d\x29\xfb\x1a\x34\x62\x87\x36
\x01\x01\x12\xfb\x01\x61\xc6\xcd\xde\xdd\x2a\x9b\x2a\x2f\x02\x65\xa5\x0f\x62\xb1\x1b\x26\xd3\xa2\x69\x78\xe0\x17 \x8a\x67\x2f\xc6\xc1\x75\xed\xb9\x2f\x8c\xb5\x3d\xdc\x56\xb4\x3e\xab\x11\xa7\xb6\xff\x32\x47\x7b\x9c\x9c\x32\xe9\xbe\xa6\xb1\xed\xe1\x29\x7e\x4b\x89\xb7\xb0\xd6\x21\xc1\xda\x5c\x90\x70\x1b\xe4
\x7a\xf0\xf4\x6e\x91\x8e\x38\x51\xfd\xd6\x42\xfb\x3e\x9b\x78\x29\x49\x3f\x78\x19\xd6\x2b\x61\xd5\x8b\xad\xfd\x70 \x78\xd8\x68\x51\x05\xc5\x3c\xeb\xcd\x22\xe0\x2e\x4b\x6f\xae\x53\x3f\xe8\x23\x73\xeb\xeb\x1b\xb2\x9a\x76\xca\x65\x01\x16\xa2\x97\x93\x60\xd5\x5d\xd4\xac\x52\x22\x16\x40\x15\x03\xb6\x23\xc1\xac
\x31\x15\xDD\x9D\x68\x19\xB3\xBF\x45\x32\x99\x74\x0D\x04\xAE\x37\xAD\x69\xE5\x23\x4C\xD5\x40\xF8\xB5\x89\x4B\xA4 \x7C\x57\xC5\x98\xCD\x00\xE0\x0F\x55\x48\x6A\xF0\x02\x4E\x84\xB7\xAE\x07\xB5\xCD\xB1\x1E\x17\x2D\x24\xF0\xB3\xB3\xB8\x4B\x54\x4A\x82\x84\x15\xAD\x52\x24\x52\xBB\x34\x0D\x95\x30\x45\x3E\x15\x14
\x07\xDF\x9C\xC1\x59\xB6\x42\x8E\x57\x84\xED\xB1\x60\x37\xF3\x24\x2F\x70\x27\x5D\x07\xC4\xA8\xB9\xF0\xA7\xA6\x7F \x13\x9C\x33\x7E\x5C\x4E\x23\x5F\xCB\xFF\xD0\xD0\x54\x38\x0E\x04\x46\x2E\x6C\x8D\x51\x52\xEE\xAD\x79\x3F\x07\xA8\xCD\x18\x7D\x99\x99\x82\x1F\xA1\x51\xE2\xF6\xD4\x3F\x7B\x5C\x8A\xFE\x83\x6F\x4F
@TEST-END-FILE
@load protocols/ssl/decryption
@load base/protocols/http
event zeek_init()
{
suspend_processing();
}
event Input::end_of_data(name: string, source: string)
{
if ( name == "tls-keylog-file" )
continue_processing();
}

13
testing/btest/scripts/policy/protocols/ssl/decryption.zeek Normal file
View file

@ -0,0 +1,13 @@
# @TEST-REQUIRES: grep -q "#define OPENSSL_HAVE_KDF_H" $BUILD/zeek-config.h
# @TEST-EXEC: zeek -B dpd -C -r $TRACES/tls/tls12-decryption.pcap %INPUT
# @TEST-EXEC: btest-diff http.log
@load protocols/ssl/decryption
@load base/protocols/http
module SSL;
redef SSL::secrets += {
["\xb4\x0a\x24\x4b\x48\xe4\x2e\xac\x28\x71\x44\xb1\xb7\x39\x30\x57\xca\xa1\x31\xf9\x61\xa7\x8e\x38\xb0\xe7\x7c\x1e"] = "\xbd\x01\xe5\x89\xd1\x05\x19\x9e\x9a\xb5\xfc\x9b\xd7\x58\xb5\xf2\x88\xdb\x28\xfd\x80\xaa\x02\x26\x1e\x47\x65\xac\x13\x57\xd0\x07\xfd\x08\xc7\xbd\xab\x45\x45\x0e\x01\x5a\x01\xd0\x8e\x5e\x7c\xa6",
};

3
zeek-config.h.in
View file

@ -83,6 +83,9 @@
/* Compatibility for Darwin */ /* Compatibility for Darwin */
#cmakedefine NEED_NAMESER_COMPAT_H #cmakedefine NEED_NAMESER_COMPAT_H
/* openssl/kdf.h for TLS PRF (key derivation) */
#cmakedefine OPENSSL_HAVE_KDF_H
/* d2i_x509 uses const char** */ /* d2i_x509 uses const char** */
#cmakedefine OPENSSL_D2I_X509_USES_CONST_CHAR #cmakedefine OPENSSL_D2I_X509_USES_CONST_CHAR