Commit graph

698 commits

Author SHA1 Message Date
Jon Siwek
8db042a8c2 Remove Cluster::broadcast_topic
As enabling Broker forwarding would cause routing loops with messages
sent to such a topic (one subscribed to on all nodes).
2018-08-28 16:40:48 -05:00
Jon Siwek
2f1e81059b Remove Intel Broker topics, re-use existing Cluster topics
And update broker docs to reflect best-practice/convention for
declaring new topics.
2018-08-28 15:43:34 -05:00
Jon Siwek
1a75ef2abd Remove "relay" family of Broker functions
Namely these are now removed:

    - Broker::relay
    - Broker::publish_and_relay
    - Cluster::relay_rr
    - Cluster::relay_hrw

The idea being that Broker may eventually implement the necessary
routing (plus load balancing) functionality.  For now, code that used
these should "manually" handle and re-publish events as needed.
2018-08-27 16:49:35 -05:00
Jon Siwek
bcf97f70ea Merge remote-tracking branch 'origin/topic/jsiwek/empty-lines'
* origin/topic/jsiwek/empty-lines:
  Add 'smtp_excessive_pending_cmds' weird
  Fix SMTP command string comparisons
  Improve handling of empty lines in several text protocol analyzers
  Add rate-limiting sampling mechanism for weird events
  Teach timestamp canonifier about timestamps before ~2001
2018-08-20 15:35:16 -05:00
Jon Siwek
1671244a64 Merge remote-tracking branch 'origin/topic/dnthayer/doc-fixes-for-2.6'
* origin/topic/dnthayer/doc-fixes-for-2.6:
  Fix some typos and improve formatting in NEWS
  Update the operators documentation
  Replace references to libgeoip in the documentation
  Update install instructions for python-ipaddress
  Update documentation of "option" and "redef" declarations
  Improvements to the config framework documentation
  Rearrange some lines on the "Log Files" documentation page
  Improve install/setup instructions for libmaxminddb
  Update NEWS for config framework clusterization changes
  Update config framework doc for clusterization changes
  Fix typos and formatting issues in config framework docs
2018-08-17 17:10:34 -05:00
Daniel Thayer
8fe300a47c Update the operators documentation
Added documentation for some new operators and improve documentation of
the "in" operator.  Also corrected a few typos in the docs.
2018-08-17 11:30:39 -05:00
Daniel Thayer
c941c565a6 Replace references to libgeoip in the documentation
Replace references to the old libgeoip library with "libmaxminddb" or
"GeoIP support".
2018-08-16 15:45:58 -05:00
Daniel Thayer
f40e317c0d Update install instructions for python-ipaddress 2018-08-16 15:41:18 -05:00
Daniel Thayer
4613347a95 Update documentation of "option" and "redef" declarations
Add documentation of using "redef" on a runtime option.  Also mention
how to change an option's value at runtime.
2018-08-16 14:23:25 -05:00
Daniel Thayer
6ef98cdb77 Improvements to the config framework documentation
Add documentation of using redef to redefine initial value of options.
Mention caveats for changing the value of specific data types.
Show an example of how to use the Config::set_value() function.
Other small improvements to the examples and text.
2018-08-16 13:32:46 -05:00
Daniel Thayer
ccfca956e9 Rearrange some lines on the "Log Files" documentation page
Moved config.log out of the "Network Protocols" section.  Moved broker.log
so that it appears in alphabetical order.
2018-08-16 11:31:26 -05:00
Daniel Thayer
9291fef6d2 Merge remote-tracking branch 'origin/master' into topic/dnthayer/doc-fixes-for-2.6 2018-08-15 15:27:44 -05:00
Jon Siwek
2c9dbdd055 Fix cluster layout graphic and doc warnings 2018-08-08 13:03:45 -05:00
Jon Siwek
599af26496 Merge remote-tracking branch 'origin/topic/vern/vec-append'
* origin/topic/vern/vec-append:
  d'oh, still have a (deprecated) string_array rather than string_vector
  forgot to update test suite results for v += e
  reap the fruits of v += e
  test case for v += e
  documentation of v += e
  v += e implemented

Fixed a mistake in find_ip_addresses()
2018-08-03 11:38:40 -05:00
Jon Siwek
06c6e1188a Merge remote-tracking branch 'origin/topic/vern/set-ops2'
* origin/topic/vern/set-ops2:
  documentation, test suite update
  implemented set relationals
  bug fix for set intersection
  set intersection implemented
  mirroring previous topic/vern/set-ops to get branch up to date, since I'm a n00b

Fixed a couple memory leaks and added a leak test
2018-08-02 10:43:41 -05:00
Jon Siwek
5804c940f1 Merge remote-tracking branch 'origin/topic/vern/perf-history'
* origin/topic/vern/perf-history:
  only generate history threshold events for > 1 instance mention those events in NEWS
  a different sort of history update
  'W' for zero window implemented; logarithmic 'T'/'C'/'W' history repetitions

I reverted a change that made TCP window tracking unconditional (possibly
accepting out-of-order packets) until further verification of test suite
changes.
2018-08-01 16:37:10 -05:00
Daniel Thayer
41a5ffa132 Improve install/setup instructions for libmaxminddb
Improved install instructions for libmaxminddb and location database.
Improved the explanation of how Bro finds the location database files,
fixed some reST formatting, and fixed a broken link.
2018-08-01 16:07:45 -05:00
Daniel Thayer
021115a6fa Merge remote-tracking branch 'origin/master' into topic/dnthayer/doc-fixes-for-2.6 2018-08-01 12:19:17 -05:00
Jon Siwek
a71d84c968 Merge branch 'topic/feature/upstream/geoip' of https://github.com/corelight/bro
* 'topic/feature/upstream/geoip' of https://github.com/corelight/bro:
  Replace GeoIP Legacy DB support with MaxMind DB support
2018-07-31 16:50:15 -05:00
Johanna Amann
a251b32d85 Merge remote-tracking branch 'origin/topic/jsiwek/openssl-1.1'
* origin/topic/jsiwek/openssl-1.1:
  Update install instructions for OpenSSL 1.1 compat
  Remove requestorName parameter of ocsp_request event
  Adjust x509 unit tests to work around OpenSSL 1.0 vs. 1.1 differences
  Fixes for OpenSSL 1.1 support
2018-07-31 13:13:06 +02:00
Jonathan Perkins
1ae7d3b349 Replace GeoIP Legacy DB support with MaxMind DB support
This updates the "lookup_location" and "lookup_asn" BIFs to use
libmaxminddb.  The motivation for this is that MaxMind is discontinuing
GeoLite Legacy databases: no updates after April 1, 2018, no downloads
after January 2, 2019.  It's also noted that all GeoIP Legacy databases
may be discontinued as they are superseded by GeoIP2.
2018-07-30 16:04:58 -05:00
Daniel Thayer
407d6461f5 Update config framework doc for clusterization changes 2018-07-30 14:27:47 -05:00
Jon Siwek
35827eeb31 Add rate-limiting sampling mechanism for weird events
The generation of weird events, by default, are now rate-limited
according to these tunable options:

  - Weird::sampling_whitelist
  - Weird::sampling_threshold
  - Weird::sampling_rate
  - Weird::sampling_duration

The new get_reporter_stats() BIF also allows one to query the
total number of weirds generated (pre-sampling) which the new
policy/misc/weird-stats.bro script uses periodically to populate
a weird_stats.log.

There's also new reporter BIFs to allow generating weirds from the
script-layer such that they go through the same, internal
rate-limiting/sampling mechanisms:

  - Reporter::conn_weird
  - Reporter::flow_weird
  - Reporter::net_weird

Some of the code was adapted from previous work by Johanna Amann.
2018-07-26 19:57:36 -05:00
Vern Paxson
88fd7510c6 reap the fruits of v += e 2018-07-26 12:51:36 -07:00
Vern Paxson
016a164bb6 documentation of v += e 2018-07-26 12:29:50 -07:00
Vern Paxson
86cd484759 documentation, test suite update 2018-07-20 08:57:37 -07:00
Jon Siwek
463e540c9b Merge remote-tracking branch 'origin/topic/vern/case-insensitive-patterns'
* origin/topic/vern/case-insensitive-patterns:
  use PCRE syntax instead of the beautiful new (?i ...) syntax
  nitlet in NEWS entry
  test suite update for case-insensitive patterns
  document use of double quotes to escape case-insensitivity
  bug fix for recent memory leak patch
  documentation updates for case-insensitive patterns
  d'oh there's isalpha.  I looked earlier for isletter :-P
  fix for handling [:(lower|upper):] in case-insensitive patterns
  implemented /re/i for case-insensitive patterns
2018-07-16 16:04:38 -05:00
Vern Paxson
187757f377 a different sort of history update 2018-07-09 13:05:50 -07:00
Jon Siwek
a66364fee0 Update install instructions for OpenSSL 1.1 compat 2018-07-02 14:04:55 -05:00
Vern Paxson
85c4b0d285 use PCRE syntax instead of the beautiful new (?i ...) syntax 2018-06-29 13:01:05 -07:00
Vern Paxson
a02d9e7f4a document use of double quotes to escape case-insensitivity 2018-06-29 11:35:22 -07:00
Vern Paxson
cfe45e0af0 documentation updates for case-insensitive patterns 2018-06-29 11:30:00 -07:00
Jon Siwek
e2d5ca5f95 Merge remote-tracking branch 'origin/topic/dnthayer/ticket1947'
* origin/topic/dnthayer/ticket1947:
  Fix some typos and formatting in NEWS
  Add pattern operators to the documentation of operators
  Fix minor typos in broker reference documentation
  Fix a broken link and some typos in broker documentation
  Fix reST formatting in documentation of "count" type
  Add documentation for some new Bro features
2018-06-27 20:27:34 -05:00
Daniel Thayer
cfe8e8f04b Merge remote-tracking branch 'origin/master' into topic/dnthayer/ticket1947 2018-06-27 14:17:44 -05:00
Daniel Thayer
bd74b4525b Add pattern operators to the documentation of operators 2018-06-27 14:00:56 -05:00
Daniel Thayer
57128af3ab Fix a broken link and some typos in broker documentation 2018-06-26 15:34:57 -05:00
Daniel Thayer
884d3d2abd Fix reST formatting in documentation of "count" type 2018-06-26 14:43:41 -05:00
Daniel Thayer
fef351b9c1 Add documentation for some new Bro features
Add documentation for the type-based "switch" statement,
the "as" operator, the "is" operator, and bitwise operators.
2018-06-26 14:38:24 -05:00
Vern Paxson
f340707e2c documentation for &/| for patterns 2018-06-22 15:23:06 -07:00
Jon Siwek
e5275b0c92 Merge branch 'topic/vern/bit-ops' of https://github.com/bro/bro
* 'topic/vern/bit-ops' of https://github.com/bro/bro:
  documentation clarification for "p1 | p2"
  documentation for bitwise operators
  document the '|' operator for patterns
  test suite for bitwise operators brief NEWS blurb allow for "counter" operands (does anyone still use these?) for one (but not both) of the bitwise operands
  bitwise operations for "count" types implemented
  Starting branch for supporting bit operations on count's.
2018-06-21 16:05:07 -05:00
Vern Paxson
344382ee7b documentation clarification for "p1 | p2" 2018-06-19 11:50:38 -07:00
Jon Siwek
c11039cb73 Make old comm. system usages an error unless old_comm_usage_is_ok is set 2018-06-15 17:15:46 -05:00
Jon Siwek
b5b688b5f5 Update make doc: don't copy broker docs 2018-06-05 15:20:49 -05:00
Jon Siwek
f9e5777e6f BIT-1635: fix make doc warnings
References to Input::Reader and Log::Writer enum types no longer emit
warnings as they are now hardcoded to be documented as part of their
associated framework scripts and so links to them now work.
2018-06-01 10:08:44 -05:00
Jon Siwek
9511daf5d2 Merge branch 'topic/dopheide/broker-docs' of https://github.com/dopheide-esnet/bro
* 'topic/dopheide/broker-docs' of https://github.com/dopheide-esnet/bro:
  A suggestion for reminding folks about calling events in Module namespaces.

I've changed this to include more accurate examples
2018-05-29 10:05:56 -05:00
Michael Dopheide
bbd65bcc74 A suggestion for reminding folks about calling events in Module namespaces. 2018-05-27 20:49:15 -05:00
Jon Siwek
b2923f5528 Documentation improvements/fixes 2018-05-23 16:50:31 -05:00
Robin Sommer
d6cddffe32 Merge remote-tracking branch 'origin/master'
* origin/master:
  Update link to flex pattern docs
2018-05-21 21:38:19 +00:00
Robin Sommer
87552390e5 Bring Broccoli back for the time being.
It's deprecated and now disabled by default, but can be reenabled by
configuring with --enable-broccoli.
2018-05-21 18:38:25 +00:00
Jon Siwek
ed7b0b3503 Update link to flex pattern docs 2018-05-21 13:38:04 -05:00