* is_valid_ip() is now implemented as a BIF instead of in
base/utils/addrs
* The IPv4 and IPv6 regular expressions provided by base/utils/addrs
have been improved/corrected (previously they could possibly match
some invalid IPv4 decimals, or various "zero compressed" IPv6 strings
with too many hextets)
* extract_ip_addresses() should give better results as a result of
the above two points
* 'smb2_write_response' of https://github.com/mauropalumbo75/zeek:
smb2_write_response event added
Fixed the unit test to ignore bad checksums in the pcap
DTLS now only outputs protocol violations once it saw something that
looked like a DTLS connection (at least a client hello). Before the
danger that it misinterprets something is too high.
It has a configurable number of invalid packets that it can skip over
(because other protocols might be interleaved with the connection) and a
maximum amount of Protocol violations that it outputs because of wrong
packet versions.
Mostly rewrote the parsing logic to support incremental parsing and
to support parsing of client messages. Though I did not add events
for client messages, that's easy to add later.
Parsing now stops for both client and server if either encounters
any parsing error or invalid state.
After a complete handshake, server messages are no longer parsed.
Support for that is incomplete and not sure it's that useful anyway
since it mostly contains pixel data.
This changes many weird names to move non-static content from the
weird name into the "addl" field to help ensure the total number of
weird names is reasonably bounded. Note the net_weird and flow_weird
events do not have an "addl" parameter, so information may no longer
be available in those cases -- to make it available again we'd need
to either (1) define new events that contain such a parameter, or
(2) change net_weird/flow_weird event signature (which is a breaking
change for user-code at the moment).
Also, the generic handling of binpac exceptions for analyzers which
to not otherwise catch and handle them has been changed from a Weird
to a ProtocolViolation.
Finally, a new "file_weird" event has been added for reporting
weirdness found during file analysis.
* 'topic/jgras/intel-filter' of https://github.com/J-Gras/zeek:
Added new intel policy script to policy test.
Added test for intel removal policy script.
Added policy script for intel removal.
Added test for intel item filtering.
Added hook to filter intelligence items.
* 'smb3-negotiate-response' of https://github.com/mauropalumbo75/zeek:
added test and pcap files for smb 3.1.1 negotiate-response
smb3.1.1 additions to negotiate-response command
I made several modifications:
- Code format, style, naming changes
- For completeness/correctness, I added parsing support for the remaining
context type structures.
- Moved the optional padding before the NegotiateContextList field to
also require the 0x0311 dialect version (some failures in
pre-existing unit tests pointed this out as an issue)
* 'smb3-transform-header' of https://github.com/mauropalumbo75/zeek:
clean up, test and pcap for transform_header added
added smb2-com-transform-header for smb3.x
This introduces the following redefinable string constants, empty by
default:
- InputAscii::path_prefix
- InputBinary::path_prefix
- Intel::path_prefix
When using ASCII or binary reades in the Input/Intel Framework with an
input stream source that does not have an absolute path, these
constants cause Zeek to prefix the resulting paths accordingly. For
example, in the following the location on disk from which Zeek loads
the input becomes "/path/to/input/whitelist.data":
redef InputAscii::path_prefix = "/path/to/input";
event bro_init()
{
Input::add_table([$source="whitelist.data", ...]);
}
These path prefixes can be absolute or relative. When an input stream
source already uses an absolute path, this path is preserved and the
new variables have no effect (i.e., we do not affect configurations
already using absolute paths).
Since the Intel framework builds upon the Input framework, the first
two paths also affect Intel file locations. If this is undesirable,
the Intel::path_prefix variable allows specifying a separate path:
when its value is absolute, the resulting source seen by the Input
framework is absolute, therefore no further changes to the paths
happen.
Essentially, it will now process/parse priority values if they are
there, or else just accept whatever remaining data/text is there as the
syslog message. Reasoning is that there's syslog producers out there
that may have simply forgotten/neglected to send the priority value
and technically won't conform to what the standard says, though we can
infer the intent (some syslog consumers already may do similarly, but
I didn't verify).
The "orig_fuids", "orig_filenames", "orig_mime_types" http.log fields as
well as their "resp" counterparts are now limited to having
"HTTP::max_files_orig" or "HTTP::max_files_resp" entries, which are 15
by default. The limit can also be ignored case-by-case via the
"HTTP::max_files_policy" hook.
Fixes GH-289
This change ignores leading/trailing whitespaces for a couple of
data-types (bool, port, subnet, addr) and just parses them as if the
whitespace was not present.
* origin/master:
Fix SumStats "last" plugin in cluster mode
Remove unnessary check
Support appending to vector of any
Fix coding conventions nits/typos
Updating submodule(s).
Revert "Switch frag_timeout from redef to option"
Improve error handling in x509_ocsp_verify function
Updating submodule(s).
Updating submodule(s).
Update baseline for plugins.hooks for redef -> option changes
Switch frag_timeout from redef to option
Switch GridFTP options from redef to option
Updating submodule(s).
Fix a unit test relying on a bash-ism
* origin/master: (23 commits)
Improve Travis script to show multiple core dump stacks
Improve a weird stats unit test
Updating submodule(s).
Fix Travis script typo
Add more debug output to Travis script
Add core file search and stack trace output for Travis builds
Updating submodule(s).
Update license year for 2018
GH-199: change `bro --help` exit status from 1 to 0
Updating submodule(s).
Add a test with an encrypted MySQL connection
Fix parsing of MySQL NUL Strings, where we now require it to have a NUL value at the end.
Updating submodule(s).
Add missing record field comment
Add missing record field comments
Fix minor documentation mistakes
Updating submodule(s).
Updating submodule(s).
Update btest baselines for fix in 46f727a6fa
Generate ssh_auth_attempted for the 'none' authentication method.
...
* origin/master:
Fix typo in Sessions.h
Update baselines for SSH capabilities fix
Fix SSH analyzer bug where is_server in capabilities is wrong.
Revert DNS query 255 from ANY to *
Missing commas...
Fix typo in dce-rpc consts
Refactor to use consistent numeric type in dce_rpc. Add missing DNS query type codes
Add DCE_RPC exchange_mapi operations to relevant consts.bro file
GH-186: fix JSON formatting of timestamps before Unix epoch
Fix test baseline for plugin skeleton update
Fix crash when modifying a table from within its &expire_func
Convert site::local_nets, etc. into options.
GH-184: add `bro-config --build_type`, outputs CMake build type
Updating submodule(s).
Add return value checks for some RPC parsing functions
Add 'fallthrough' comment to a switch/case block
Improve broker.remote_id unit test
Increase broker unit test timeout intervals
* 'fatemabw/bro' of https://github.com/fatemabw/bro:
DNSSEC support in Bro
I made several changes:
- renamed event/record types
- reformatted the info added to dns.log
- removed the "addl" scripts that added extended dnssec info to dns.log
- simplifications/improvements to the internal parsing logic
Still finding it to not be foolproof enough to enable generally for all
nodes in a cluster. Specific/advanced use-cases may still consider
enabling, possibly just for specific nodes.
* origin/topic/johanna/tls-more-data:
Update NEWS for ssl changes.
SSL: test updates for record_layer version
Final touches to SSL events with record layer version.
Introduce ssl_plaintext_data event.
Add record layer version to event ssl_encrypted_data.
Add compression methods to ssl_client_hello event.
* origin/topic/jsiwek/bit-1967:
Fix a routing loop in control framework
Add Broker::forward() function
Enable implicit Broker message forwarding by default
Remove Cluster::broadcast_topic
Remove Intel Broker topics, re-use existing Cluster topics
Remove "relay" family of Broker functions
This enables explicit forwarding of events matching a given topic
prefix. Even if a receiving node has an event handler, it will not
be raised if the event was sent along a topic that matches a previous
call to Broker::forward().
