Commit graph

2261 commits

Author SHA1 Message Date
Jon Siwek
f19db92508 Merge branch 'topic/christian/inputframework-paths' of https://github.com/ckreibich/zeek
* 'topic/christian/inputframework-paths' of https://github.com/ckreibich/zeek:
  Source file path control for Input and Intel frameworks
2019-03-18 11:48:22 -07:00
Jon Siwek
01d303b480 Migrate table-based for-loops to key-value iteration 2019-03-15 19:54:44 -07:00
Christian Kreibich
3f02c0a67c Source file path control for Input and Intel frameworks
This introduces the following redefinable string constants, empty by
default:

- InputAscii::path_prefix
- InputBinary::path_prefix
- Intel::path_prefix

When using ASCII or binary reades in the Input/Intel Framework with an
input stream source that does not have an absolute path, these
constants cause Zeek to prefix the resulting paths accordingly. For
example, in the following the location on disk from which Zeek loads
the input becomes "/path/to/input/whitelist.data":

redef InputAscii::path_prefix = "/path/to/input";

event bro_init()
        {
        Input::add_table([$source="whitelist.data", ...]);
	}

These path prefixes can be absolute or relative. When an input stream
source already uses an absolute path, this path is preserved and the
new variables have no effect (i.e., we do not affect configurations
already using absolute paths).

Since the Intel framework builds upon the Input framework, the first
two paths also affect Intel file locations. If this is undesirable,
the Intel::path_prefix variable allows specifying a separate path:
when its value is absolute, the resulting source seen by the Input
framework is absolute, therefore no further changes to the paths
happen.
2019-03-15 16:43:36 -07:00
Jon Siwek
03ac32adec Merge branch 'topic/dopheide/fix-ssh-geo-data' of https://github.com/dopheide-esnet/bro
* 'topic/dopheide/fix-ssh-geo-data' of https://github.com/dopheide-esnet/bro:
  Fix geo-data to log remote_location data when auth is successful.
2019-03-15 13:03:59 -07:00
Robin Sommer
72fa3f16ad Merge remote-tracking branch 'origin/topic/jsiwek/vxlan'
* origin/topic/jsiwek/vxlan:
  GH-250: Improve/cleanup VXLAN decapsulation support
  Initial VXLAN support, need -C flag to work when running bro
2019-03-14 17:59:42 +00:00
Jon Siwek
1239a286e2 Merge branch 'patch-1' of https://github.com/neu5ron/bro
* 'patch-1' of https://github.com/neu5ron/bro:
  Update main.bro
  Update main.bro
2019-03-13 13:49:17 -07:00
Michael Dopheide
0f6f6cdb29 Fix geo-data to log remote_location data when auth is successful. 2019-03-13 14:14:38 -05:00
Robin Sommer
56da70c148 Merge remote-tracking branch 'origin/topic/jsiwek/gh-289'
* origin/topic/jsiwek/gh-289:
  GH-289: Add options to limit entries in http.log file fields
2019-03-13 18:23:06 +00:00
Robin Sommer
4cf6cb783a Merge remote-tracking branch 'origin/topic/jsiwek/gh-281'
Closes #281.

* origin/topic/jsiwek/gh-281:
  GH-281: Improve parsing of Google Pixel user agent
2019-03-13 18:09:21 +00:00
Robin Sommer
e2172018ee Merge remote-tracking branch 'origin/topic/jsiwek/gh-286'
Added note about different behaviour to NEWS.

Closes #286.

* origin/topic/jsiwek/gh-286:
  GH-286: Check for record type mismatch in ternary operator
2019-03-13 18:01:36 +00:00
Jon Siwek
09ae539ea8 GH-250: Improve/cleanup VXLAN decapsulation support
* Better parsing/error-checking of VXLAN and encapsulated packet headers

* Add/implement the "vxlan_packet" event

* Add "Tunnel::vxlan_ports" option to tune the set of VXLAN ports to
  analyze/decapsulate

* Add "Tunnel::validate_vxlan_checksums" option to allow for tuning of how
  checksums associated with the outer UDP header of a possible VXLAN
  tunnel are handled

Fixes GH-250
2019-03-12 18:15:34 -07:00
Henrik Lund Kramshoej
f4088be8a6 Initial VXLAN support, need -C flag to work when running bro 2019-03-11 14:15:39 -07:00
Jon Siwek
a36ac12e88 Improve DNS query queuing logic
Related to https://github.com/zeek/zeek/pull/276
2019-03-08 18:38:41 -08:00
Jon Siwek
011b6e10bf Merge remote-tracking branch 'origin/topic/jazoff/dns-perf'
* origin/topic/jazoff/dns-perf:
  update baselines
  Use more efficient method to grab the protocol.
  improve performance of dns policy
2019-03-08 16:23:30 -08:00
Jon Siwek
92b6656bad GH-289: Add options to limit entries in http.log file fields
The "orig_fuids", "orig_filenames", "orig_mime_types" http.log fields as
well as their "resp" counterparts are now limited to having
"HTTP::max_files_orig" or "HTTP::max_files_resp" entries, which are 15
by default.  The limit can also be ignored case-by-case via the
"HTTP::max_files_policy" hook.

Fixes GH-289
2019-02-27 14:25:15 -06:00
Nate Guagenti
3d4dd2ea67
Update main.bro
added back user agent
2019-02-26 16:56:10 -05:00
Jon Siwek
53dc1176b7 GH-281: Improve parsing of Google Pixel user agent
Fixes GH-281
2019-02-25 14:07:53 -06:00
Jon Siwek
74c225c7cb GH-286: Check for record type mismatch in ternary operator
Fixes GH-286
2019-02-25 12:55:03 -06:00
Justin Azoff
a5ad864e13 Use more efficient method to grab the protocol.
get_conn_transport_proto needs to use sessions->FindConnection and do a
hash lookup to find the connection while get_port_transport_proto just
looks at the port directly.
2019-02-17 12:39:01 -05:00
Justin Azoff
a97cf10d03 improve performance of dns policy
defer allocating various tables until at least 2 pending queries are
seen.
2019-02-17 12:39:01 -05:00
Jon Siwek
8e9dc3c789 Improve format of conn_state docs 2019-02-13 16:15:13 -08:00
Jon Siwek
0790c1c559 Merge remote-tracking branch 'origin/topic/jsiwek/gh-211'
* origin/topic/jsiwek/gh-211:
  GH-208: change invalid subnet expressions to a runtime error
  GH-211: improve consistency of how scripting errors are handled

Removed the 'allow_init_errors' option.
2019-02-07 10:33:52 -06:00
Jon Siwek
67484a90fa GH-211: improve consistency of how scripting errors are handled
Scripting errors/mistakes now consistently generate a runtime error
which have the behavior of unwinding the call stack all the way out of
the current event handler.

Before, such errors were not treated consistently and either aborted
the process entirely or emitted a message while continuing to execute
subsequent statements without well-defined behavior (possibly causing
a cascade of errors).

The previous behavior also would only unwind out of the current
function (if within a function body), not out the current event
handler, which is especially problematic for functions that return
a value: the caller is essentially left a mess with no way to deal
with it.

This also changes the behavior of the startup/initialization process
to abort if there's errors during bro_init() rather than continue one
to the main run loop.  The `allow_init_errors` option may change this
new, default behavior.
2019-01-30 11:20:09 -06:00
Johanna Amann
2e2f611df5 Merge branch 'master' of https://github.com/hosom/zeek
* 'master' of https://github.com/hosom/zeek:
  Add fuid to SSL:Invalid_Server_Cert notice
2019-01-29 14:52:34 -08:00
Stephen Hosom
e30a02e186 Add fuid to SSL:Invalid_Server_Cert notice
This is a very basid quality of life improvement. It should make it
much easier to find additional information about the certificate
in question.
2019-01-29 13:34:51 -05:00
Jon Siwek
0d685efbf5 Add Broker::peer_counts_as_iosource option
Disabling this option allows one to read pcaps, but still initiate
Broker peerings and automatically exit when done processing the pcap
file.  The default behavior would normally cause Broker::peer() to
prevent shutting the process down even after done reading the pcap.
2019-01-16 19:03:35 -06:00
Justin Azoff
cc2981ab3b improve performance of catch and release script
Defer allocation of the Entity record until it will be used.
2019-01-09 13:58:39 -05:00
Jon Siwek
e0b8b4b6b1 Replace some bro.org usages with zeek.org 2019-01-04 17:51:25 -06:00
Jon Siwek
af82e87699 Add dns_resolver option
This allows one to configure the address of the DNS server to use for
lookup operations.

Addresses GH-216
2018-11-30 16:59:45 -06:00
Jon Siwek
635e030be2 Merge remote-tracking branch 'origin/master' into dev/2.7
* origin/master:
  Fix SumStats "last" plugin in cluster mode
  Remove unnessary check
  Support appending to vector of any
  Fix coding conventions nits/typos
  Updating submodule(s).
  Revert "Switch frag_timeout from redef to option"
  Improve error handling in x509_ocsp_verify function
  Updating submodule(s).
  Updating submodule(s).
  Update baseline for plugins.hooks for redef -> option changes
  Switch frag_timeout from redef to option
  Switch GridFTP options from redef to option
  Updating submodule(s).
  Fix a unit test relying on a bash-ism
2018-11-08 12:43:22 -06:00
Jon Siwek
01e17b5ea0 Fix SumStats "last" plugin in cluster mode 2018-11-08 12:27:37 -06:00
Jon Siwek
c1f02aaa61 Revert "Switch frag_timeout from redef to option"
This reverts commit 38259afa8d.

The change to an option wasn't yet incomplete -- that would
require dynamic lookup of the value at the C++ layer.
2018-11-06 09:32:58 -06:00
Vlad Grigorescu
38259afa8d Switch frag_timeout from redef to option 2018-11-05 13:41:13 -06:00
Vlad Grigorescu
93c094fff2 Switch GridFTP options from redef to option 2018-11-05 13:41:05 -06:00
Jon Siwek
ebd8132063 Merge remote-tracking branch 'origin/master' into dev/2.7
* origin/master: (23 commits)
  Improve Travis script to show multiple core dump stacks
  Improve a weird stats unit test
  Updating submodule(s).
  Fix Travis script typo
  Add more debug output to Travis script
  Add core file search and stack trace output for Travis builds
  Updating submodule(s).
  Update license year for 2018
  GH-199: change `bro --help` exit status from 1 to 0
  Updating submodule(s).
  Add a test with an encrypted MySQL connection
  Fix parsing of MySQL NUL Strings, where we now require it to have a NUL value at the end.
  Updating submodule(s).
  Add missing record field comment
  Add missing record field comments
  Fix minor documentation mistakes
  Updating submodule(s).
  Updating submodule(s).
  Update btest baselines for fix in 46f727a6fa
  Generate ssh_auth_attempted for the 'none' authentication method.
  ...
2018-11-02 14:45:32 -05:00
Jon Siwek
0cc5e4e044 Add missing record field comment 2018-10-26 10:42:05 -05:00
Jon Siwek
8d0087154a Add missing record field comments 2018-10-26 10:24:30 -05:00
Jon Siwek
6a059a1cf7 Fix minor documentation mistakes 2018-10-25 18:56:38 -05:00
Jon Siwek
45526e6063 Merge remote-tracking branch 'origin/master' into dev/2.7
* origin/master:
  Fix typo in Sessions.h
  Update baselines for SSH capabilities fix
  Fix SSH analyzer bug where is_server in capabilities is wrong.
  Revert DNS query 255 from ANY to *
  Missing commas...
  Fix typo in dce-rpc consts
  Refactor to use consistent numeric type in dce_rpc. Add missing DNS query type codes
  Add DCE_RPC exchange_mapi operations to relevant consts.bro file
  GH-186: fix JSON formatting of timestamps before Unix epoch
  Fix test baseline for plugin skeleton update
  Fix crash when modifying a table from within its &expire_func
  Convert site::local_nets, etc. into options.
  GH-184: add `bro-config --build_type`, outputs CMake build type
  Updating submodule(s).
  Add return value checks for some RPC parsing functions
  Add 'fallthrough' comment to a switch/case block
  Improve broker.remote_id unit test
  Increase broker unit test timeout intervals
2018-10-16 16:50:27 -05:00
Jon Siwek
9a295a7009 Merge branch 'master' of https://github.com/spitfire55/bro
* 'master' of https://github.com/spitfire55/bro:
  Revert DNS query 255 from ANY to *
  Missing commas...
  Fix typo in dce-rpc consts
  Refactor to use consistent numeric type in dce_rpc. Add missing DNS query type codes

I added back in DNS constants for PTR, EDNS, and ANY to avoid breaking
code for any people that use them.

Also omitted the DNP3 function code 0x83 name change from
"AUTHENTICATE_RESP" to "AUTHENTICATE_RESPONSE", again to avoid
potentially breaking code unnecessarily: "RESP" vs. "RESPONSE" is not
wrong in any sense, just maybe a matter of clarify.
2018-10-16 15:56:37 -05:00
Dale Lakes
cf99ecf8b2 Revert DNS query 255 from ANY to * 2018-10-15 21:15:31 -04:00
Dale Lakes
c335988b0c Missing commas... 2018-10-15 20:35:17 -04:00
Dale Lakes
6399a96ea5 Fix typo in dce-rpc consts 2018-10-15 19:59:32 -04:00
Dale Lakes
3c765731b3 Refactor to use consistent numeric type in dce_rpc. Add missing DNS query type codes 2018-10-15 19:17:53 -04:00
Jon Siwek
c8637b7430 Merge branch 'master' of https://github.com/spitfire55/bro
* 'master' of https://github.com/spitfire55/bro:
  Add DCE_RPC exchange_mapi operations to relevant consts.bro file
2018-10-15 16:43:50 -05:00
Dale Lakes
a8efaf3a7e Add DCE_RPC exchange_mapi operations to relevant consts.bro file
New operations names and opnums come from the MSDN documentation
and OpenChange Project.
2018-10-15 16:47:10 -04:00
Johanna Amann
005bf5b6a4 Convert site::local_nets, etc. into options.
These are probably some of the most desired options to be dynamically
changeable; since they only are accessed in script-land there should not
be any problems with them changing on the fly.
2018-10-09 16:33:26 -07:00
Jon Siwek
1ea9c8eb40 GH-148: add priority to DNSSEC event handlers 2018-09-24 15:34:16 -05:00
Jon Siwek
71ef5c8428 Merge branch 'fatemabw/bro' of https://github.com/fatemabw/bro into dev/2.7
* 'fatemabw/bro' of https://github.com/fatemabw/bro:
  DNSSEC support in Bro

I made several changes:

  - renamed event/record types
  - reformatted the info added to dns.log
  - removed the "addl" scripts that added extended dnssec info to dns.log
  - simplifications/improvements to the internal parsing logic
2018-09-21 16:40:41 -05:00
Jon Siwek
c2c5754e28 Merge branch 'topic/jazoff/sqli-policy-hook' of https://github.com/JustinAzoff/bro
* 'topic/jazoff/sqli-policy-hook' of https://github.com/JustinAzoff/bro:
  add sqli_policy hook
2018-09-19 15:22:45 -05:00