Commit graph

1935 commits

Author SHA1 Message Date
Jon Siwek
be7110f6c0 Make Syslog analyzer accept messages that omit Priority
Essentially, it will now process/parse priority values if they are
there, or else just accept whatever remaining data/text is there as the
syslog message.  Reasoning is that there's syslog producers out there
that may have simply forgotten/neglected to send the priority value
and technically won't conform to what the standard says, though we can
infer the intent (some syslog consumers already may do similarly, but
I didn't verify).
2019-03-14 18:47:32 -07:00
Robin Sommer
72fa3f16ad Merge remote-tracking branch 'origin/topic/jsiwek/vxlan'
* origin/topic/jsiwek/vxlan:
  GH-250: Improve/cleanup VXLAN decapsulation support
  Initial VXLAN support, need -C flag to work when running bro
2019-03-14 17:59:42 +00:00
Jon Siwek
1239a286e2 Merge branch 'patch-1' of https://github.com/neu5ron/bro
* 'patch-1' of https://github.com/neu5ron/bro:
  Update main.bro
  Update main.bro
2019-03-13 13:49:17 -07:00
Robin Sommer
56da70c148 Merge remote-tracking branch 'origin/topic/jsiwek/gh-289'
* origin/topic/jsiwek/gh-289:
  GH-289: Add options to limit entries in http.log file fields
2019-03-13 18:23:06 +00:00
Robin Sommer
4cf6cb783a Merge remote-tracking branch 'origin/topic/jsiwek/gh-281'
Closes #281.

* origin/topic/jsiwek/gh-281:
  GH-281: Improve parsing of Google Pixel user agent
2019-03-13 18:09:21 +00:00
Robin Sommer
e2172018ee Merge remote-tracking branch 'origin/topic/jsiwek/gh-286'
Added note about different behaviour to NEWS.

Closes #286.

* origin/topic/jsiwek/gh-286:
  GH-286: Check for record type mismatch in ternary operator
2019-03-13 18:01:36 +00:00
Jon Siwek
09ae539ea8 GH-250: Improve/cleanup VXLAN decapsulation support
* Better parsing/error-checking of VXLAN and encapsulated packet headers

* Add/implement the "vxlan_packet" event

* Add "Tunnel::vxlan_ports" option to tune the set of VXLAN ports to
  analyze/decapsulate

* Add "Tunnel::validate_vxlan_checksums" option to allow for tuning of how
  checksums associated with the outer UDP header of a possible VXLAN
  tunnel are handled

Fixes GH-250
2019-03-12 18:15:34 -07:00
Henrik Lund Kramshoej
f4088be8a6 Initial VXLAN support, need -C flag to work when running bro 2019-03-11 14:15:39 -07:00
Jon Siwek
a36ac12e88 Improve DNS query queuing logic
Related to https://github.com/zeek/zeek/pull/276
2019-03-08 18:38:41 -08:00
Jon Siwek
011b6e10bf Merge remote-tracking branch 'origin/topic/jazoff/dns-perf'
* origin/topic/jazoff/dns-perf:
  update baselines
  Use more efficient method to grab the protocol.
  improve performance of dns policy
2019-03-08 16:23:30 -08:00
Jon Siwek
92b6656bad GH-289: Add options to limit entries in http.log file fields
The "orig_fuids", "orig_filenames", "orig_mime_types" http.log fields as
well as their "resp" counterparts are now limited to having
"HTTP::max_files_orig" or "HTTP::max_files_resp" entries, which are 15
by default.  The limit can also be ignored case-by-case via the
"HTTP::max_files_policy" hook.

Fixes GH-289
2019-02-27 14:25:15 -06:00
Nate Guagenti
3d4dd2ea67
Update main.bro
added back user agent
2019-02-26 16:56:10 -05:00
Jon Siwek
53dc1176b7 GH-281: Improve parsing of Google Pixel user agent
Fixes GH-281
2019-02-25 14:07:53 -06:00
Jon Siwek
74c225c7cb GH-286: Check for record type mismatch in ternary operator
Fixes GH-286
2019-02-25 12:55:03 -06:00
Justin Azoff
a5ad864e13 Use more efficient method to grab the protocol.
get_conn_transport_proto needs to use sessions->FindConnection and do a
hash lookup to find the connection while get_port_transport_proto just
looks at the port directly.
2019-02-17 12:39:01 -05:00
Justin Azoff
a97cf10d03 improve performance of dns policy
defer allocating various tables until at least 2 pending queries are
seen.
2019-02-17 12:39:01 -05:00
Jon Siwek
8e9dc3c789 Improve format of conn_state docs 2019-02-13 16:15:13 -08:00
Jon Siwek
0790c1c559 Merge remote-tracking branch 'origin/topic/jsiwek/gh-211'
* origin/topic/jsiwek/gh-211:
  GH-208: change invalid subnet expressions to a runtime error
  GH-211: improve consistency of how scripting errors are handled

Removed the 'allow_init_errors' option.
2019-02-07 10:33:52 -06:00
Jon Siwek
67484a90fa GH-211: improve consistency of how scripting errors are handled
Scripting errors/mistakes now consistently generate a runtime error
which have the behavior of unwinding the call stack all the way out of
the current event handler.

Before, such errors were not treated consistently and either aborted
the process entirely or emitted a message while continuing to execute
subsequent statements without well-defined behavior (possibly causing
a cascade of errors).

The previous behavior also would only unwind out of the current
function (if within a function body), not out the current event
handler, which is especially problematic for functions that return
a value: the caller is essentially left a mess with no way to deal
with it.

This also changes the behavior of the startup/initialization process
to abort if there's errors during bro_init() rather than continue one
to the main run loop.  The `allow_init_errors` option may change this
new, default behavior.
2019-01-30 11:20:09 -06:00
Jon Siwek
0d685efbf5 Add Broker::peer_counts_as_iosource option
Disabling this option allows one to read pcaps, but still initiate
Broker peerings and automatically exit when done processing the pcap
file.  The default behavior would normally cause Broker::peer() to
prevent shutting the process down even after done reading the pcap.
2019-01-16 19:03:35 -06:00
Justin Azoff
cc2981ab3b improve performance of catch and release script
Defer allocation of the Entity record until it will be used.
2019-01-09 13:58:39 -05:00
Jon Siwek
e0b8b4b6b1 Replace some bro.org usages with zeek.org 2019-01-04 17:51:25 -06:00
Jon Siwek
af82e87699 Add dns_resolver option
This allows one to configure the address of the DNS server to use for
lookup operations.

Addresses GH-216
2018-11-30 16:59:45 -06:00
Jon Siwek
635e030be2 Merge remote-tracking branch 'origin/master' into dev/2.7
* origin/master:
  Fix SumStats "last" plugin in cluster mode
  Remove unnessary check
  Support appending to vector of any
  Fix coding conventions nits/typos
  Updating submodule(s).
  Revert "Switch frag_timeout from redef to option"
  Improve error handling in x509_ocsp_verify function
  Updating submodule(s).
  Updating submodule(s).
  Update baseline for plugins.hooks for redef -> option changes
  Switch frag_timeout from redef to option
  Switch GridFTP options from redef to option
  Updating submodule(s).
  Fix a unit test relying on a bash-ism
2018-11-08 12:43:22 -06:00
Jon Siwek
01e17b5ea0 Fix SumStats "last" plugin in cluster mode 2018-11-08 12:27:37 -06:00
Jon Siwek
c1f02aaa61 Revert "Switch frag_timeout from redef to option"
This reverts commit 38259afa8d.

The change to an option wasn't yet incomplete -- that would
require dynamic lookup of the value at the C++ layer.
2018-11-06 09:32:58 -06:00
Vlad Grigorescu
38259afa8d Switch frag_timeout from redef to option 2018-11-05 13:41:13 -06:00
Vlad Grigorescu
93c094fff2 Switch GridFTP options from redef to option 2018-11-05 13:41:05 -06:00
Jon Siwek
ebd8132063 Merge remote-tracking branch 'origin/master' into dev/2.7
* origin/master: (23 commits)
  Improve Travis script to show multiple core dump stacks
  Improve a weird stats unit test
  Updating submodule(s).
  Fix Travis script typo
  Add more debug output to Travis script
  Add core file search and stack trace output for Travis builds
  Updating submodule(s).
  Update license year for 2018
  GH-199: change `bro --help` exit status from 1 to 0
  Updating submodule(s).
  Add a test with an encrypted MySQL connection
  Fix parsing of MySQL NUL Strings, where we now require it to have a NUL value at the end.
  Updating submodule(s).
  Add missing record field comment
  Add missing record field comments
  Fix minor documentation mistakes
  Updating submodule(s).
  Updating submodule(s).
  Update btest baselines for fix in 46f727a6fa
  Generate ssh_auth_attempted for the 'none' authentication method.
  ...
2018-11-02 14:45:32 -05:00
Jon Siwek
8d0087154a Add missing record field comments 2018-10-26 10:24:30 -05:00
Jon Siwek
6a059a1cf7 Fix minor documentation mistakes 2018-10-25 18:56:38 -05:00
Jon Siwek
45526e6063 Merge remote-tracking branch 'origin/master' into dev/2.7
* origin/master:
  Fix typo in Sessions.h
  Update baselines for SSH capabilities fix
  Fix SSH analyzer bug where is_server in capabilities is wrong.
  Revert DNS query 255 from ANY to *
  Missing commas...
  Fix typo in dce-rpc consts
  Refactor to use consistent numeric type in dce_rpc. Add missing DNS query type codes
  Add DCE_RPC exchange_mapi operations to relevant consts.bro file
  GH-186: fix JSON formatting of timestamps before Unix epoch
  Fix test baseline for plugin skeleton update
  Fix crash when modifying a table from within its &expire_func
  Convert site::local_nets, etc. into options.
  GH-184: add `bro-config --build_type`, outputs CMake build type
  Updating submodule(s).
  Add return value checks for some RPC parsing functions
  Add 'fallthrough' comment to a switch/case block
  Improve broker.remote_id unit test
  Increase broker unit test timeout intervals
2018-10-16 16:50:27 -05:00
Jon Siwek
9a295a7009 Merge branch 'master' of https://github.com/spitfire55/bro
* 'master' of https://github.com/spitfire55/bro:
  Revert DNS query 255 from ANY to *
  Missing commas...
  Fix typo in dce-rpc consts
  Refactor to use consistent numeric type in dce_rpc. Add missing DNS query type codes

I added back in DNS constants for PTR, EDNS, and ANY to avoid breaking
code for any people that use them.

Also omitted the DNP3 function code 0x83 name change from
"AUTHENTICATE_RESP" to "AUTHENTICATE_RESPONSE", again to avoid
potentially breaking code unnecessarily: "RESP" vs. "RESPONSE" is not
wrong in any sense, just maybe a matter of clarify.
2018-10-16 15:56:37 -05:00
Dale Lakes
cf99ecf8b2 Revert DNS query 255 from ANY to * 2018-10-15 21:15:31 -04:00
Dale Lakes
c335988b0c Missing commas... 2018-10-15 20:35:17 -04:00
Dale Lakes
6399a96ea5 Fix typo in dce-rpc consts 2018-10-15 19:59:32 -04:00
Dale Lakes
3c765731b3 Refactor to use consistent numeric type in dce_rpc. Add missing DNS query type codes 2018-10-15 19:17:53 -04:00
Jon Siwek
c8637b7430 Merge branch 'master' of https://github.com/spitfire55/bro
* 'master' of https://github.com/spitfire55/bro:
  Add DCE_RPC exchange_mapi operations to relevant consts.bro file
2018-10-15 16:43:50 -05:00
Dale Lakes
a8efaf3a7e Add DCE_RPC exchange_mapi operations to relevant consts.bro file
New operations names and opnums come from the MSDN documentation
and OpenChange Project.
2018-10-15 16:47:10 -04:00
Johanna Amann
005bf5b6a4 Convert site::local_nets, etc. into options.
These are probably some of the most desired options to be dynamically
changeable; since they only are accessed in script-land there should not
be any problems with them changing on the fly.
2018-10-09 16:33:26 -07:00
Jon Siwek
1ea9c8eb40 GH-148: add priority to DNSSEC event handlers 2018-09-24 15:34:16 -05:00
Jon Siwek
71ef5c8428 Merge branch 'fatemabw/bro' of https://github.com/fatemabw/bro into dev/2.7
* 'fatemabw/bro' of https://github.com/fatemabw/bro:
  DNSSEC support in Bro

I made several changes:

  - renamed event/record types
  - reformatted the info added to dns.log
  - removed the "addl" scripts that added extended dnssec info to dns.log
  - simplifications/improvements to the internal parsing logic
2018-09-21 16:40:41 -05:00
Jon Siwek
161aae828a Merge remote-tracking branch 'origin/topic/seth/fix-raw-reader-subprocess-exit'
* origin/topic/seth/fix-raw-reader-subprocess-exit:
  Fix an issue with raw reader culling streams for dead processes.

Updated the 'exec' utility to no longer remove input streams for
processes that are finished as the core C++ code will take care of that
(and trying to remove a stream multiple times emits a warning message).
2018-09-11 13:05:40 -05:00
Jon Siwek
34d0cf886c Fix potential memory leak in Kerberos scripts
Reported by Maksim Shudrak.
2018-09-10 18:06:07 -05:00
Jon Siwek
4bd6da7186 Update default Broker/CAF thread tuning 2018-09-07 17:50:28 -05:00
Jon Siwek
73c8cf733a Give Cluster::rr_topic "key" argument a default value 2018-09-07 09:58:57 -05:00
Jon Siwek
c73bb8fdc4 Disable broker message forwarding by default
Still finding it to not be foolproof enough to enable generally for all
nodes in a cluster.  Specific/advanced use-cases may still consider
enabling, possibly just for specific nodes.
2018-09-06 18:32:22 -05:00
Daniel Thayer
d1ed09bddb Added a documentation comment for the Input::Event type 2018-09-05 22:27:15 -05:00
Jon Siwek
b99be6458b Merge remote-tracking branch 'origin/topic/johanna/weird-options'
* origin/topic/johanna/weird-options:
  Update test baselines (weird options)
  Weird settings: make constants into options.
  Permit weird sampling rate of 0.
2018-09-05 16:57:08 -05:00
Jon Siwek
f00e2167a7 BIT-1208: remove unused weirds from Weird::actions table 2018-09-05 15:13:38 -05:00