Switch file UID hashing from md5 to highwayhash.

This commit switches UID hashing from md5 to a highway hash. It also
moves the salt value out of the file plugin - and makes it
installation-specific instead - it is moved to the global namespace.

There now are digest hash functions to make "static"
installation-specific hashes that are stable over workers available to
everyone; hashes can be 64, 128 or 256 bits in size.

Due to the fact that we switch the file hashing algorithm, all file
hashes change.

The underlyigng algorithm that is used for hashing is highwayhash-128,
which is significantly faster than md5.
This commit is contained in:
Johanna Amann 2020-04-30 10:03:56 -07:00
parent bc546634d1
commit 3bce313b12
153 changed files with 953 additions and 799 deletions

View file

@ -3,14 +3,14 @@
#empty_field (empty)
#unset_field -
#path smb_files
#open 2019-02-14-15-17-09
#open 2020-04-30-00-47-31
#fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p fuid action path name size prev_name times.modified times.accessed times.created times.changed
#types time string addr port addr port string enum string string count string time time time time
1549644186.686127 CHhAvVGS1DHFjwGM9 169.254.128.18 49155 169.254.128.15 445 - SMB::FILE_OPEN - pythonfile 16 - 1549643138.282481 1549643183.156000 1549643138.280000 1549643138.282481
1549644186.686127 CHhAvVGS1DHFjwGM9 169.254.128.18 49155 169.254.128.15 445 - SMB::FILE_READ - pythonfile 16 - 1549643138.282481 1549643183.156000 1549643138.280000 1549643138.282481
1549644186.686127 CHhAvVGS1DHFjwGM9 169.254.128.18 49155 169.254.128.15 445 FG403EpKSkh5CwCre SMB::FILE_READ - pythonfile 16 - 1549643138.282481 1549643183.156000 1549643138.280000 1549643138.282481
1549644186.686127 CHhAvVGS1DHFjwGM9 169.254.128.18 49155 169.254.128.15 445 FvOchP1DvxPt75ql7b SMB::FILE_READ - pythonfile 16 - 1549643138.282481 1549643183.156000 1549643138.280000 1549643138.282481
1549644186.692584 CHhAvVGS1DHFjwGM9 169.254.128.18 49155 169.254.128.15 445 - SMB::FILE_OPEN - pythonfile2 0 - 1549644186.688000 1549644186.688000 1549644186.688000 1549644186.688000
1549644186.692584 CHhAvVGS1DHFjwGM9 169.254.128.18 49155 169.254.128.15 445 - SMB::FILE_WRITE - pythonfile2 0 - 1549644186.688000 1549644186.688000 1549644186.688000 1549644186.688000
1549644186.692584 CHhAvVGS1DHFjwGM9 169.254.128.18 49155 169.254.128.15 445 FLCGB5TxPTWKKeQf4 SMB::FILE_WRITE - pythonfile2 0 - 1549644186.688000 1549644186.688000 1549644186.688000 1549644186.688000
1549644186.692584 CHhAvVGS1DHFjwGM9 169.254.128.18 49155 169.254.128.15 445 FRCqNs3XdP1aPvzhvf SMB::FILE_WRITE - pythonfile2 0 - 1549644186.688000 1549644186.688000 1549644186.688000 1549644186.688000
1549644187.702245 CHhAvVGS1DHFjwGM9 169.254.128.18 49155 169.254.128.15 445 - SMB::FILE_OPEN - <share_root> 0 - 1549644186.688000 1549644187.700000 1549644186.688000 1549644186.688000
#close 2019-02-14-15-17-09
#close 2020-04-30-00-47-31