* origin/topic/dev/non-ascii-logging:
Removed Policy Script for UTF-8 Logs
Commented out UTF-8 Script in Test All Policy
Minor Style Tweak
Use getNumBytesForUTF8 method to determine number of bytes
Added Jon's test cases as unit tests
Prioritizes escaping predefined Escape Sequences over Unescaping UTF-8 Sequences
Added additional check to confirm anything unescaping is a multibyte UTF-8 sequence, addressing the test case Jon brought up
Added optional script and redef bool to enable utf-8 in ASCII logs
Initial Commit, removed std::isprint check to escape
Made minor code format and logic adjustments during merge.
This allows one to tune the number of protocol violations to tolerate
from any given analyzer type before just disabling a given instance
of it.
Also removes the "disabled_aids" field from the DPD::Info record
since it serves no purpose: in this case, calling disable_analyzer
multiple times for the same analyzer is a no-op.
It may generally be better for our default use-case, as workers may
save a few percent cpu utilization as this policy does not have to
use any polling like the stealing policy does.
This also helps avoid a potential issue with the implementation of
spinlocks used in the work-stealing policy in current CAF versions,
where there's some conditions where lock contention causes a thread
to spin for long periods without relinquishing the cpu to others.
This signature is relevant for process dumps on Windows that could be extracted by various tools. The unencrypted transmission of the dump of a critical system process (for example, lsass.exe) via network would be detected by this rule.
In the past they were processed on the manager - which requires big
records to be sent around.
This has a potential of incompatibilities if someone relied on global
state for notice processing.
GH-214
* "bro_is_terminating" is now "zeek_is_terminating"
* "bro_version" is now "zeek_version"
The old function names still exist for now, but are deprecated.
These are no longer loaded by default due to the performance impact they
cause simply by being loaded (they have event handlers for commonly
generated events) and they aren't generally useful enough to justify it.
For backward compatibility when reading values, we first check
the ZEEK-prefixed value, and if not set, then check the corresponding
BRO-prefixed value.
Broker::subscribe() after Broker::peer() may result in losing messages,
always best to do the reverse order.
Also possibly improved chance of unstable unit test output order.
* origin/topic/robin/gh-239:
Undo a change to btest.cfg from a recent commit
Updating submodule.
Fix zeek-wrapper
Update for renaming BroControl to ZeekControl.
Updating submodule.
GH-239: Rename bro to zeek, bro-config to zeek-config, and bro-path-dev to zeek-path-dev.
* All "Broxygen" usages have been replaced in
code, documentation, filenames, etc.
* Sphinx roles/directives like ":bro:see" are now ":zeek:see"
* The "--broxygen" command-line option is now "--zeexygen"
This changes many weird names to move non-static content from the
weird name into the "addl" field to help ensure the total number of
weird names is reasonably bounded. Note the net_weird and flow_weird
events do not have an "addl" parameter, so information may no longer
be available in those cases -- to make it available again we'd need
to either (1) define new events that contain such a parameter, or
(2) change net_weird/flow_weird event signature (which is a breaking
change for user-code at the moment).
Also, the generic handling of binpac exceptions for analyzers which
to not otherwise catch and handle them has been changed from a Weird
to a ProtocolViolation.
Finally, a new "file_weird" event has been added for reporting
weirdness found during file analysis.
This introduces the following redefinable string constants, empty by
default:
- InputAscii::path_prefix
- InputBinary::path_prefix
- Intel::path_prefix
When using ASCII or binary reades in the Input/Intel Framework with an
input stream source that does not have an absolute path, these
constants cause Zeek to prefix the resulting paths accordingly. For
example, in the following the location on disk from which Zeek loads
the input becomes "/path/to/input/whitelist.data":
redef InputAscii::path_prefix = "/path/to/input";
event bro_init()
{
Input::add_table([$source="whitelist.data", ...]);
}
These path prefixes can be absolute or relative. When an input stream
source already uses an absolute path, this path is preserved and the
new variables have no effect (i.e., we do not affect configurations
already using absolute paths).
Since the Intel framework builds upon the Input framework, the first
two paths also affect Intel file locations. If this is undesirable,
the Intel::path_prefix variable allows specifying a separate path:
when its value is absolute, the resulting source seen by the Input
framework is absolute, therefore no further changes to the paths
happen.
* Better parsing/error-checking of VXLAN and encapsulated packet headers
* Add/implement the "vxlan_packet" event
* Add "Tunnel::vxlan_ports" option to tune the set of VXLAN ports to
analyze/decapsulate
* Add "Tunnel::validate_vxlan_checksums" option to allow for tuning of how
checksums associated with the outer UDP header of a possible VXLAN
tunnel are handled
Fixes GH-250
Disabling this option allows one to read pcaps, but still initiate
Broker peerings and automatically exit when done processing the pcap
file. The default behavior would normally cause Broker::peer() to
prevent shutting the process down even after done reading the pcap.
Still finding it to not be foolproof enough to enable generally for all
nodes in a cluster. Specific/advanced use-cases may still consider
enabling, possibly just for specific nodes.
* origin/topic/johanna/weird-options:
Update test baselines (weird options)
Weird settings: make constants into options.
Permit weird sampling rate of 0.
