Switch file UID hashing from md5 to highwayhash.

This commit switches UID hashing from md5 to a highway hash. It also
moves the salt value out of the file plugin - and makes it
installation-specific instead - it is moved to the global namespace.

There now are digest hash functions to make "static"
installation-specific hashes that are stable over workers available to
everyone; hashes can be 64, 128 or 256 bits in size.

Due to the fact that we switch the file hashing algorithm, all file
hashes change.

The underlyigng algorithm that is used for hashing is highwayhash-128,
which is significantly faster than md5.
This commit is contained in:
Johanna Amann 2020-04-30 10:03:56 -07:00
parent bc546634d1
commit 3bce313b12
153 changed files with 953 additions and 799 deletions

View file

@ -3,8 +3,8 @@
#empty_field (empty)
#unset_field -
#path ssl
#open 2016-07-13-16-16-58
#open 2020-04-30-00-47-36
#fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p version cipher curve server_name resumed last_alert next_protocol established cert_chain_fuids client_cert_chain_fuids subject issuer client_subject client_issuer
#types time string addr port addr port string string string string bool string string bool vector[string] vector[string] string string string string
1335538392.319381 CHhAvVGS1DHFjwGM9 192.168.1.105 62045 74.125.224.79 443 TLSv10 TLS_ECDHE_RSA_WITH_RC4_128_SHA secp256r1 ssl.gstatic.com F - - T F6wfNWn8LR755SYo7,FJl60T1mOolaez9T0h (empty) CN=*.gstatic.com,O=Google Inc,L=Mountain View,ST=California,C=US CN=Google Internet Authority,O=Google Inc,C=US - -
#close 2016-07-13-16-16-58
1335538392.319381 CHhAvVGS1DHFjwGM9 192.168.1.105 62045 74.125.224.79 443 TLSv10 TLS_ECDHE_RSA_WITH_RC4_128_SHA secp256r1 ssl.gstatic.com F - - T FHRcwD4vPs7uj2D7m6,FjrkUT3ceLKRNN1Tnb (empty) CN=*.gstatic.com,O=Google Inc,L=Mountain View,ST=California,C=US CN=Google Internet Authority,O=Google Inc,C=US - -
#close 2020-04-30-00-47-36

View file

@ -3,9 +3,9 @@
#empty_field (empty)
#unset_field -
#path x509
#open 2016-07-13-16-16-58
#open 2020-04-30-00-47-36
#fields ts id certificate.version certificate.serial certificate.subject certificate.issuer certificate.not_valid_before certificate.not_valid_after certificate.key_alg certificate.sig_alg certificate.key_type certificate.key_length certificate.exponent certificate.curve san.dns san.uri san.email san.ip basic_constraints.ca basic_constraints.path_len
#types time string count string string string time time string string string count string string vector[string] vector[string] vector[string] vector[addr] bool count
1335538392.343624 F6wfNWn8LR755SYo7 3 36F5DA5300000000505E CN=*.gstatic.com,O=Google Inc,L=Mountain View,ST=California,C=US CN=Google Internet Authority,O=Google Inc,C=US 1334102677.000000 1365639277.000000 rsaEncryption sha1WithRSAEncryption rsa 1024 65537 - *.gstatic.com,gstatic.com,*.metric.gstatic.com - - - - -
1335538392.343624 FJl60T1mOolaez9T0h 3 0B6771 CN=Google Internet Authority,O=Google Inc,C=US OU=Equifax Secure Certificate Authority,O=Equifax,C=US 1244493807.000000 1370634207.000000 rsaEncryption sha1WithRSAEncryption rsa 1024 65537 - - - - - T 0
#close 2016-07-13-16-16-58
1335538392.343624 FHRcwD4vPs7uj2D7m6 3 36F5DA5300000000505E CN=*.gstatic.com,O=Google Inc,L=Mountain View,ST=California,C=US CN=Google Internet Authority,O=Google Inc,C=US 1334102677.000000 1365639277.000000 rsaEncryption sha1WithRSAEncryption rsa 1024 65537 - *.gstatic.com,gstatic.com,*.metric.gstatic.com - - - - -
1335538392.343624 FjrkUT3ceLKRNN1Tnb 3 0B6771 CN=Google Internet Authority,O=Google Inc,C=US OU=Equifax Secure Certificate Authority,O=Equifax,C=US 1244493807.000000 1370634207.000000 rsaEncryption sha1WithRSAEncryption rsa 1024 65537 - - - - - T 0
#close 2020-04-30-00-47-36