mirror of
https://github.com/zeek/zeek.git
synced 2025-10-15 21:18:20 +00:00
Copy docs into Zeek repo directly
This is based on commit 99e6942efec5feff50523f6b2a1f5868f19ab638 from the zeek-docs repo.
This commit is contained in:
Tim Wojtulewicz
parent
979a98c73c
commit
adce4e604a
1075 changed files with 169492 additions and 1 deletions
165
doc/frameworks/tls-decryption.rst
Normal file
165
doc/frameworks/tls-decryption.rst
Normal file
|
|
@ -0,0 +1,165 @@
|
|||
|
||||
.. _framework-tls-decryption:
|
||||
|
||||
==============
|
||||
TLS Decryption
|
||||
==============
|
||||
|
||||
.. rst-class:: opening
|
||||
|
||||
|
||||
Zeek has limited support for decrypting TLS connections, if the necessary key material is
|
||||
available. If decryption is possible, Zeek can forward the decrypted data to other analyzers - like
|
||||
the HTTP analyzer.
|
||||
|
||||
Note that this support is currently limited to a single version of TLS and a single cipher suite.
|
||||
Zeek can currently only decrypt TLS 1.2 connections that use the TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
|
||||
cipher. Any other TLS version or cipher will not be decrypted. We do not currently plan to extend
|
||||
this support to other versions of TLS, or to other ciphersuites.
|
||||
|
||||
Capturing and decrypting a trace file
|
||||
=====================================
|
||||
|
||||
The most common use-case for TLS decryption is to capture a trace file together with the necessary key material
|
||||
to allow Zeek to decrypt it. In principle, it is possible to allow Zeek to decrypt TLS connections in live traffic.
|
||||
However, this kind of setup is much more complex and will require the user to set up a way to transport the
|
||||
key material to Zeek in real-time. We will talk a bit about the possibility of this below.
|
||||
|
||||
Capturing a trace file with keys
|
||||
--------------------------------
|
||||
|
||||
To be able to decrypt the TLS connections contained in a trace file, we need access to the symmetric keys that
|
||||
were used to encrypt the connection. Specifically, for our supported version of TLS, we need the pre-master secret
|
||||
for the TLS connection.
|
||||
|
||||
Firefox and Chrome allow users to capture the key material of all TLS connections that they perform by setting
|
||||
the ``SSLKEYLOGFILE`` environment variable. For example, on Mac OS, you can instruct Firefox to record the TLS
|
||||
key material by starting it in the following way:
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
export SSLKEYLOGFILE=$HOME/keylogfile.txt
|
||||
open -a firefox
|
||||
|
||||
After running Firefox like this, and accessing some web pages, you should end up with the file ``keylogfile.txt``
|
||||
in your home directory that contains lines like this::
|
||||
|
||||
# SSL/TLS secrets log file, generated by NSS
|
||||
CLIENT_RANDOM 47d1becb619e0851ee363c2cf37187228227ca4e680f9a7c0bd15069aa7a5970 ad03ceda4890fa581e989f5e3862023e2a4e3e8ad81325238d908066e1d35cc875979e34c08e6fdfd9d8c6f356e385c1
|
||||
CLIENT_RANDOM 2095006fcb3f93d255cbb6562587f0dd010212fdee9d233aff64e6ed36cd5c45 0d36faaa2eadbda2a8095f951de1cbac46b81b008fbf391d91951b3485476bab73288a1e17cd0ce80e0fc0401dbe9e3f
|
||||
CLIENT_RANDOM 8f58b32bf97e7d3856e2fccbbe80798ec2e3f515251082ad63bbc7c231d8bee0 9a7cf946a04718a19f4d20c3f80c1cf8c823c3e2b1c337ef64322d751b410543315f6ecf7dbf45ec9be194a3cc7c1a0f
|
||||
|
||||
These log lines contain the pre-master secrets for the connections that your browser established. The secrets
|
||||
are indexed with the client random of the connections. This allows applications (like Zeek) to identify which
|
||||
secret to use to decrypt a connection.
|
||||
|
||||
If you capture this key log file together with a trace-file, you will be able to decrypt the sessions using Zeek
|
||||
(assuming they use a supported TLS version and ciphersuite).
|
||||
|
||||
Decrypting a trace file
|
||||
-----------------------
|
||||
|
||||
The next step is to convert the keylogfile into a format that can be ingested by the Zeek. This bash-script
|
||||
will perform the conversion:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
#!/usr/bin/env bash
|
||||
|
||||
if [ $# -ne 1 ]; then
|
||||
echo "Script expects one argument (key log filename)" >/dev/stderr
|
||||
exit -1
|
||||
fi
|
||||
|
||||
FILE=$1
|
||||
|
||||
if [ ! -f ${FILE} ]; then
|
||||
echo "${FILE} does not exist or is not readable" >/dev/stderr
|
||||
exit -1
|
||||
fi
|
||||
|
||||
echo "#fields client_random secret"
|
||||
grep CLIENT_RANDOM ${FILE} | sed 's/^CLIENT_RANDOM ........\(.*\) \(.*\)$/\1 \2/' | sed 's/[A-Za-z0-9][A-Za-z0-9]/\\x&/g'
|
||||
|
||||
Note that the script just converts the keylog file in a standard Zeek tsv-file. Furthermore, it removes
|
||||
the first 16 characters of the CLIENT_RANDOM; this is needed due to a design-choice of Zeek that makes accessing
|
||||
the first 8 bytes (equivalent to 16 hex-characters) of the client random inconvenient - thus these bytes are not
|
||||
used for matching.
|
||||
|
||||
If you run the bash script on the ``keylogfile.txt`` you created earlier, you will get a Zeek tsv-file.
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
./convert-keylog.sh ~/keylogfile.txt > ~/keylogfile.log
|
||||
|
||||
cat ~/keylogfile.log
|
||||
#fields client_random secret
|
||||
\x0e\x78\x2d\x35\x63\x95\x5d\x8a\x30\xa9\xcf\xb6\x4f\x47\xf3\x96\x34\x8a\x1e\x79\x1a\xa2\x32\x55\xe2\x2f\xc5\x7a \x34\x4f\x12\x65\xbf\x43\x40\xb3\x61\x6b\xa0\x16\x5d\x2b\x4d\xb9\xb1\xe8\x4a\x3d\xa2\x42\x0e\x38\xab\x01\x50\x62\x84\xcc\x34\xcd\xe0\x34\x10\xfe\x1a\x02\x30\x49\x74\x6c\x46\x43\xa7\x0c\x67\x0d
|
||||
\x24\x8c\x7e\x24\xee\xfb\x13\xcd\xee\xde\xb1\xf4\xb6\xd6\xd5\xee\x67\x8d\xd3\xff\xc7\xe7\x39\x23\x18\x3f\x99\xb4 \xe7\xed\x24\x26\x0d\x25\xd9\xfd\xf5\x0f\xc0\xf4\x56\x51\x0e\x4e\xec\x7f\x58\x9c\xaf\x39\x25\x14\x16\xa6\x71\xdd\xea\xfe\xe9\xc0\x93\xbe\x89\x4c\xab\xcc\xff\xb2\xf0\x9a\xea\x98\xf5\xb2\x53\x1e
|
||||
\x57\xd7\xc7\x7a\x2d\x5e\x35\x29\x2c\xd7\xe7\x94\xee\xf8\x6f\x31\x45\xf6\xbe\x25\x08\xed\x1d\x92\xd2\x0b\x9b\x04 \xc1\x93\x17\x93\xd9\x7d\xd2\x98\xb3\xe0\xdb\x2c\x5d\xbe\x71\x31\xa7\x9a\xf5\x91\xf9\x87\x90\xee\xb7\x79\x9f\x6b\xb4\x1f\x47\xa7\x69\x62\x4b\xa3\x99\x0c\xa9\x43\xf9\xea\x3b\x4d\x5f\x2f\xfe\xfb
|
||||
|
||||
Now we can run Zeek on the trace-file that we recorded. We need a small additional script for this, which
|
||||
stops processing while the TLS keylog file is loaded. It also loads the required policy script.
|
||||
|
||||
.. literalinclude:: tls_decryption-1-suspend-processing.zeek
|
||||
:caption:
|
||||
:language: zeek
|
||||
:linenos:
|
||||
:tab-width: 4
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
$ export ZEEK_TLS_KEYLOG_FILE=~/keylogfile.log
|
||||
$ zeek -C -r tls/tls-1.2-stream-keylog.pcap tls_decryption-1-suspend-processing.zeek
|
||||
|
||||
$ cat conn.log
|
||||
#separator \x09
|
||||
#set_separator ,
|
||||
#empty_field (empty)
|
||||
#unset_field -
|
||||
#path conn
|
||||
#open 2022-03-01-16-57-26
|
||||
#fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p proto service duration orig_bytes resp_bytes conn_state local_orig local_resp missed_bytes history orig_pkts orig_ip_bytes resp_pkts resp_ip_bytes tunnel_parents
|
||||
#types time string addr port addr port enum string interval count count string bool bool count string count count count count set[string]
|
||||
1646150638.631834 CTy5Us4OUaTOcyrPvc 192.168.20.12 60679 193.99.144.85 443 tcp http,ssl 7.246461 10853 151695 SF - - 0 ShADadFf 98 15961 139 158931 -
|
||||
#close 2022-03-01-16-57-26
|
||||
|
||||
$ cat http.log
|
||||
#separator \x09
|
||||
#set_separator ,
|
||||
#empty_field (empty)
|
||||
#unset_field -
|
||||
#path http
|
||||
#open 2022-03-01-16-57-25
|
||||
#fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p trans_depth method host uri referrer version user_agent origin request_body_len response_body_len status_code status_msg info_code info_msg tags username password proxied orig_fuids orig_filenames orig_mime_types resp_fuids resp_filenames resp_mime_types
|
||||
#types time string addr port addr port count string string string string string string string count count count string count string set[enum] string string set[string] vector[string] vector[string] vector[string] vector[string] vector[string] vector[string]
|
||||
1646150638.735969 CTy5Us4OUaTOcyrPvc 192.168.20.12 60679 193.99.144.85 443 1 GET www.heise.de /assets/akwa/v24/js/akwa.js?.ltc.c61e84978682308f631c https://www.heise.de/ 1.1 Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:97.0) Gecko/20100101 Firefox/97.0 - 0 375340 200 OK - - (empty) - - - - - - FSJiWr34wfIujxxtm3 - text/plain
|
||||
1646150638.944774 CTy5Us4OUaTOcyrPvc 192.168.20.12 60679 193.99.144.85 443 2 GET www.heise.de /assets/heise/images/mit_technology_review_singleline.b768.ltc.svg https://www.heise.de/ 1.1 Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:97.0) Gecko/20100101 Firefox/97.0 - 0 3430 200 OK - - (empty) - - - - - - FgivhC1pvnYeQS4u18 - text/plain
|
||||
1646150638.976118 CTy5Us4OUaTOcyrPvc 192.168.20.12 60679 193.99.144.85 443 3 GET www.heise.de /assets/heise/hobell/css/hobell.css?.ltc.3746e7e49abafa23b5fb https://www.heise.de/ 1.1 Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:97.0) Gecko/20100101 Firefox/97.0 - 0 85280 200 OK - - (empty) - - - - - - FyvBkl2nwRXf0hkDO1 - text/plain
|
||||
...
|
||||
|
||||
Now :file:`conn.log` shows that the HTTP as well as the SSL analyzers were attached. :file:`http.log` shows the
|
||||
information from the decrypted HTTP session.
|
||||
|
||||
If you try this yourself note that today a lot of encrypted Internet traffic uses HTTP/2. Zeek currently does not
|
||||
ship with an HTTP/2 parser by default. If you capture your own traffic make sure that your browser uses
|
||||
HTTP/1. Alternatively, you can add an HTTP/2 analyzer to Zeek, e.g. using a package.
|
||||
|
||||
Decrypting live traffic
|
||||
=======================
|
||||
|
||||
In principle, it is possible to decrypt live traffic using this approach. When you want to do this, you have to supply the secrets
|
||||
to Zeek as the connections are happening. Note that there are timing constraints here - the secrets should arrive at the Zeek instance
|
||||
that will decrypt the traffic before encrypted application data is exchanged.
|
||||
|
||||
The :doc:`/scripts/policy/protocols/ssl/decryption.zeek` policy script sets up a two events for this purpose. You can send key material
|
||||
to the Zeek worker in question via Broker, using the ``/zeek/tls/decryption`` topic. The two events used for this are
|
||||
:zeek:see:`SSL::add_keys` and :zeek:see:`SSL::add_secret`.
|
||||
|
||||
TLS Decryption API
|
||||
==================
|
||||
|
||||
If the policy script does not suit your use-case, you can use the TLS decryption API directly to decrypt a connection. You can use either the
|
||||
:zeek:see:`set_secret` or the :zeek:see:`set_keys` functions to provide the decryption keys for an ongoing SSL connection.
|
||||
|
||||
Note that you will have to make sure to set :zeek:see:`SSL::disable_analyzer_after_detection` to false if you use this functionality directly.
|
||||
Loading…
Add table
Add a link
Reference in a new issue