Commit graph

994 commits

Author SHA1 Message Date
Jon Siwek
b5050437fa GH-379: move catch-and-release and unified2 scripts to policy/
These are no longer loaded by default due to the performance impact they
cause simply by being loaded (they have event handlers for commonly
generated events) and they aren't generally useful enough to justify it.
2019-06-05 13:33:45 -07:00
Jon Siwek
1ce0fcce49 GH-387: update Broker topic names to use "zeek/" prefix 2019-05-29 15:56:37 -07:00
Jon Siwek
7f0fb49612 Add an internal getenv wrapper function: zeekenv
It maps newer environment variable names starting with ZEEK to the
legacy names starting with BRO.
2019-05-23 20:42:42 -07:00
Daniel Thayer
1a74516db1 Rename all BRO-prefixed environment variables
For backward compatibility when reading values, we first check
the ZEEK-prefixed value, and if not set, then check the corresponding
BRO-prefixed value.
2019-05-22 00:12:31 -05:00
Daniel Thayer
be182aac83 More bro-to-zeek renaming in scripts and other files 2019-05-16 02:36:41 -05:00
Jon Siwek
a8c0cd7dee Fix potential race in openflow broker plugin
Broker::subscribe() after Broker::peer() may result in losing messages,
always best to do the reverse order.

Also possibly improved chance of unstable unit test output order.
2019-05-15 10:12:32 -07:00
Jon Siwek
6ad7099f7e Merge remote-tracking branch 'origin/topic/robin/gh-239'
* origin/topic/robin/gh-239:
  Undo a change to btest.cfg from a recent commit
  Updating submodule.
  Fix zeek-wrapper
  Update for renaming BroControl to ZeekControl.
  Updating submodule.
  GH-239: Rename bro to zeek, bro-config to zeek-config, and bro-path-dev to zeek-path-dev.
2019-05-14 13:27:40 -07:00
Jon Siwek
cb6b9a1f1a Allow tuning Broker log batching via scripts
Via redefining "Broker::log_batch_size" or "Broker::log_batch_interval"
2019-05-08 12:44:55 -07:00
Robin Sommer
89b8d6e7ba Update for renaming BroControl to ZeekControl. 2019-05-06 21:22:19 +00:00
Johanna Amann
dcd6454530 Remove RemoteSerializer and related code/types.
Also removes broccoli from the source tree.
2019-05-03 15:00:13 -07:00
Jon Siwek
aebcb1415d GH-234: rename Broxygen to Zeexygen along with roles/directives
* All "Broxygen" usages have been replaced in
  code, documentation, filenames, etc.

* Sphinx roles/directives like ":bro:see" are now ":zeek:see"

* The "--broxygen" command-line option is now "--zeexygen"
2019-04-22 19:45:50 -07:00
Jon Siwek
a994be9eeb Merge remote-tracking branch 'origin/topic/seth/zeek_init'
* origin/topic/seth/zeek_init:
  Some more testing fixes.
  Update docs and tests for bro_(init|done) -> zeek_(init|done)
  Implement the zeek_init handler.
2019-04-19 11:24:29 -07:00
Seth Hall
8cefb9be42 Implement the zeek_init handler.
Implements the change and a test.
2019-04-14 08:37:35 -04:00
Daniel Thayer
9cbeb44986 Merge remote-tracking branch 'upstream/master' 2019-04-14 01:35:50 -05:00
Daniel Thayer
18bd74454b Rename all scripts to have ".zeek" file extension 2019-04-11 21:12:40 -05:00
Jon Siwek
995368e68c Remove variable content from weird names
This changes many weird names to move non-static content from the
weird name into the "addl" field to help ensure the total number of
weird names is reasonably bounded.  Note the net_weird and flow_weird
events do not have an "addl" parameter, so information may no longer
be available in those cases -- to make it available again we'd need
to either (1) define new events that contain such a parameter, or
(2) change net_weird/flow_weird event signature (which is a breaking
change for user-code at the moment).

Also, the generic handling of binpac exceptions for analyzers which
to not otherwise catch and handle them has been changed from a Weird
to a ProtocolViolation.

Finally, a new "file_weird" event has been added for reporting
weirdness found during file analysis.
2019-04-01 18:30:11 -07:00
Daniel Thayer
bff8392ad4 Remove unnecessary ".bro" from @load directives
Removed ".bro" file extensions from "@load" directives because
they are not needed.
2019-03-31 02:24:47 -05:00
Jan Grashoefer
3e31f3d554 Added hook to filter intelligence items. 2019-03-24 21:31:48 +01:00
Jon Siwek
f19db92508 Merge branch 'topic/christian/inputframework-paths' of https://github.com/ckreibich/zeek
* 'topic/christian/inputframework-paths' of https://github.com/ckreibich/zeek:
  Source file path control for Input and Intel frameworks
2019-03-18 11:48:22 -07:00
Jon Siwek
01d303b480 Migrate table-based for-loops to key-value iteration 2019-03-15 19:54:44 -07:00
Christian Kreibich
3f02c0a67c Source file path control for Input and Intel frameworks
This introduces the following redefinable string constants, empty by
default:

- InputAscii::path_prefix
- InputBinary::path_prefix
- Intel::path_prefix

When using ASCII or binary reades in the Input/Intel Framework with an
input stream source that does not have an absolute path, these
constants cause Zeek to prefix the resulting paths accordingly. For
example, in the following the location on disk from which Zeek loads
the input becomes "/path/to/input/whitelist.data":

redef InputAscii::path_prefix = "/path/to/input";

event bro_init()
        {
        Input::add_table([$source="whitelist.data", ...]);
	}

These path prefixes can be absolute or relative. When an input stream
source already uses an absolute path, this path is preserved and the
new variables have no effect (i.e., we do not affect configurations
already using absolute paths).

Since the Intel framework builds upon the Input framework, the first
two paths also affect Intel file locations. If this is undesirable,
the Intel::path_prefix variable allows specifying a separate path:
when its value is absolute, the resulting source seen by the Input
framework is absolute, therefore no further changes to the paths
happen.
2019-03-15 16:43:36 -07:00
Robin Sommer
72fa3f16ad Merge remote-tracking branch 'origin/topic/jsiwek/vxlan'
* origin/topic/jsiwek/vxlan:
  GH-250: Improve/cleanup VXLAN decapsulation support
  Initial VXLAN support, need -C flag to work when running bro
2019-03-14 17:59:42 +00:00
Robin Sommer
4cf6cb783a Merge remote-tracking branch 'origin/topic/jsiwek/gh-281'
Closes #281.

* origin/topic/jsiwek/gh-281:
  GH-281: Improve parsing of Google Pixel user agent
2019-03-13 18:09:21 +00:00
Jon Siwek
09ae539ea8 GH-250: Improve/cleanup VXLAN decapsulation support
* Better parsing/error-checking of VXLAN and encapsulated packet headers

* Add/implement the "vxlan_packet" event

* Add "Tunnel::vxlan_ports" option to tune the set of VXLAN ports to
  analyze/decapsulate

* Add "Tunnel::validate_vxlan_checksums" option to allow for tuning of how
  checksums associated with the outer UDP header of a possible VXLAN
  tunnel are handled

Fixes GH-250
2019-03-12 18:15:34 -07:00
Henrik Lund Kramshoej
f4088be8a6 Initial VXLAN support, need -C flag to work when running bro 2019-03-11 14:15:39 -07:00
Jon Siwek
53dc1176b7 GH-281: Improve parsing of Google Pixel user agent
Fixes GH-281
2019-02-25 14:07:53 -06:00
Jon Siwek
74c225c7cb GH-286: Check for record type mismatch in ternary operator
Fixes GH-286
2019-02-25 12:55:03 -06:00
Jon Siwek
0d685efbf5 Add Broker::peer_counts_as_iosource option
Disabling this option allows one to read pcaps, but still initiate
Broker peerings and automatically exit when done processing the pcap
file.  The default behavior would normally cause Broker::peer() to
prevent shutting the process down even after done reading the pcap.
2019-01-16 19:03:35 -06:00
Justin Azoff
cc2981ab3b improve performance of catch and release script
Defer allocation of the Entity record until it will be used.
2019-01-09 13:58:39 -05:00
Jon Siwek
e0b8b4b6b1 Replace some bro.org usages with zeek.org 2019-01-04 17:51:25 -06:00
Jon Siwek
01e17b5ea0 Fix SumStats "last" plugin in cluster mode 2018-11-08 12:27:37 -06:00
Jon Siwek
4bd6da7186 Update default Broker/CAF thread tuning 2018-09-07 17:50:28 -05:00
Jon Siwek
73c8cf733a Give Cluster::rr_topic "key" argument a default value 2018-09-07 09:58:57 -05:00
Jon Siwek
c73bb8fdc4 Disable broker message forwarding by default
Still finding it to not be foolproof enough to enable generally for all
nodes in a cluster.  Specific/advanced use-cases may still consider
enabling, possibly just for specific nodes.
2018-09-06 18:32:22 -05:00
Daniel Thayer
d1ed09bddb Added a documentation comment for the Input::Event type 2018-09-05 22:27:15 -05:00
Jon Siwek
b99be6458b Merge remote-tracking branch 'origin/topic/johanna/weird-options'
* origin/topic/johanna/weird-options:
  Update test baselines (weird options)
  Weird settings: make constants into options.
  Permit weird sampling rate of 0.
2018-09-05 16:57:08 -05:00
Jon Siwek
f00e2167a7 BIT-1208: remove unused weirds from Weird::actions table 2018-09-05 15:13:38 -05:00
Johanna Amann
bec98b98f3 Weird settings: make constants into options.
The new weird settings are now all updateable during runtime.
2018-09-05 13:12:23 -07:00
Jon Siwek
4bbdd63054 BIT-1779: use BRO_LOG_SUFFIX env var in ascii log rotation function 2018-09-05 15:09:57 -05:00
Robin Sommer
6de436f3f6 Merge remote-tracking branch 'origin/topic/jsiwek/bit-1967'
* origin/topic/jsiwek/bit-1967:
  Fix a routing loop in control framework
  Add Broker::forward() function
  Enable implicit Broker message forwarding by default
  Remove Cluster::broadcast_topic
  Remove Intel Broker topics, re-use existing Cluster topics
  Remove "relay" family of Broker functions
2018-08-30 00:08:12 +00:00
Jon Siwek
1dcead93bf Add Broker::forward() function
This enables explicit forwarding of events matching a given topic
prefix.  Even if a receiving node has an event handler, it will not
be raised if the event was sent along a topic that matches a previous
call to Broker::forward().
2018-08-28 19:42:22 -05:00
Jon Siwek
850030822d Enable implicit Broker message forwarding by default 2018-08-28 16:50:41 -05:00
Jon Siwek
8db042a8c2 Remove Cluster::broadcast_topic
As enabling Broker forwarding would cause routing loops with messages
sent to such a topic (one subscribed to on all nodes).
2018-08-28 16:40:48 -05:00
Jon Siwek
2f1e81059b Remove Intel Broker topics, re-use existing Cluster topics
And update broker docs to reflect best-practice/convention for
declaring new topics.
2018-08-28 15:43:34 -05:00
Daniel Thayer
9bfc01b705 Convert more redef-able constants to runtime options 2018-08-27 19:38:47 -05:00
Jon Siwek
1a75ef2abd Remove "relay" family of Broker functions
Namely these are now removed:

    - Broker::relay
    - Broker::publish_and_relay
    - Cluster::relay_rr
    - Cluster::relay_hrw

The idea being that Broker may eventually implement the necessary
routing (plus load balancing) functionality.  For now, code that used
these should "manually" handle and re-publish events as needed.
2018-08-27 16:49:35 -05:00
Daniel Thayer
fd1a23ea66 Add a missing initializer to a runtime option 2018-08-24 17:03:02 -05:00
Daniel Thayer
01a899255e Convert more redef-able constants to runtime options 2018-08-24 16:05:44 -05:00
Jon Siwek
6595b21e2e Merge remote-tracking branch 'origin/topic/dnthayer/ticket1963'
* origin/topic/dnthayer/ticket1963:
  Remove unused redef-able constants
  Convert some redef-able constants to runtime options
2018-08-20 12:44:58 -05:00
Daniel Thayer
1a4629b0dc Merge remote-tracking branch 'origin/master' into topic/dnthayer/ticket1963 2018-08-17 14:11:47 -05:00