Merge remote-tracking branch 'origin/topic/dev/non-ascii-logging'

* origin/topic/dev/non-ascii-logging:
  Removed Policy Script for UTF-8 Logs
  Commented out UTF-8 Script in Test All Policy
  Minor Style Tweak
  Use getNumBytesForUTF8 method to determine number of bytes
  Added Jon's test cases as unit tests
  Prioritizes escaping predefined Escape Sequences over Unescaping UTF-8 Sequences
  Added additional check to confirm anything unescaping is a multibyte UTF-8 sequence, addressing the test case Jon brought up
  Added optional script and redef bool to enable utf-8 in ASCII logs
  Initial Commit, removed std::isprint check to escape

Made minor code format and logic adjustments during merge.
This commit is contained in:
Jon Siwek 2019-07-30 19:36:56 -07:00
commit d1770853b3
16 changed files with 181 additions and 3 deletions

View file

@ -26,6 +26,12 @@ export {
## This option is also available as a per-filter ``$config`` option.
const use_json = F &redef;
## If true, valid UTF-8 sequences will pass through unescaped and be
## written into logs.
##
## This option is also available as a per-filter ``$config`` option.
const enable_utf_8 = F &redef;
## Define the gzip level to compress the logs. If 0, then no gzip
## compression is performed. Enabling compression also changes
## the log file name extension to include ".gz".