FileAnalysis: add custom libmagic database.

- It's derived from the magic database of libmagic 5.14, but with most
  everything not related to mime types removed.

- The custom database is always used by default for mime detection, but
  the more verbose file type detection will fall back on the default
  libmagic installation's database.  The result is: mime type strings
  are now guaranteed to be consistent across platforms, but the verbose
  file type descriptions are not.

- The custom database gets installed in $prefix/share/bro/magic, and
  should even be extensible if files with new patterns are added inside
  the directory.

- The search path for the mime magic database can be controlled via
  BROMAGIC environment variable.

- Remove mime_desc field from ftp.log.

- Stop using the mime/file type canonifier with unit tests.

- libmagic >= 5.04 is now a requirement.
This commit is contained in:
Jon Siwek 2013-04-12 11:58:19 -05:00
parent b8c98b8bf7
commit 037d582b0e
106 changed files with 2951 additions and 174 deletions

View file

@ -2,8 +2,8 @@ FILE_NEW
aFQKI8SPOL2, 0, 0
FILE_BOF_BUFFER
/*^J********
FILE_TYPE
mime type is set
MIME_TYPE
text/plain
FILE_STATE_REMOVE
aFQKI8SPOL2, 2675, 0
[orig_h=192.168.1.104, orig_p=1673/tcp, resp_h=63.245.209.11, resp_p=80/tcp]
@ -15,8 +15,8 @@ FILE_NEW
CCU3vUEr06l, 0, 0
FILE_BOF_BUFFER
//-- Google
FILE_TYPE
mime type is set
MIME_TYPE
text/plain
FILE_STATE_REMOVE
CCU3vUEr06l, 21421, 0
[orig_h=192.168.1.104, orig_p=1673/tcp, resp_h=63.245.209.11, resp_p=80/tcp]
@ -28,8 +28,8 @@ FILE_NEW
HCzA0dVwDPj, 0, 0
FILE_BOF_BUFFER
GIF89a^D\0^D\0\xb3
FILE_TYPE
mime type is set
MIME_TYPE
image/gif
FILE_STATE_REMOVE
HCzA0dVwDPj, 94, 0
[orig_h=192.168.1.104, orig_p=1673/tcp, resp_h=63.245.209.11, resp_p=80/tcp]
@ -42,8 +42,8 @@ FILE_NEW
a1Zu1fteVEf, 0, 0
FILE_BOF_BUFFER
\x89PNG^M^J^Z^J\0\0\0
FILE_TYPE
mime type is set
MIME_TYPE
image/png
FILE_STATE_REMOVE
a1Zu1fteVEf, 2349, 0
[orig_h=192.168.1.104, orig_p=1673/tcp, resp_h=63.245.209.11, resp_p=80/tcp]
@ -56,8 +56,8 @@ FILE_NEW
xXlF7wFdsR, 0, 0
FILE_BOF_BUFFER
\x89PNG^M^J^Z^J\0\0\0
FILE_TYPE
mime type is set
MIME_TYPE
image/png
FILE_STATE_REMOVE
xXlF7wFdsR, 27579, 0
[orig_h=192.168.1.104, orig_p=1673/tcp, resp_h=63.245.209.11, resp_p=80/tcp]