Commit graph

2220 commits

Author SHA1 Message Date
Daniel Thayer
bb313cb660 Merge remote-tracking branch 'origin/master' into topic/dnthayer/ticket1963 2018-08-27 19:39:45 -05:00
Daniel Thayer
9bfc01b705 Convert more redef-able constants to runtime options 2018-08-27 19:38:47 -05:00
Jon Siwek
1a75ef2abd Remove "relay" family of Broker functions
Namely these are now removed:

    - Broker::relay
    - Broker::publish_and_relay
    - Cluster::relay_rr
    - Cluster::relay_hrw

The idea being that Broker may eventually implement the necessary
routing (plus load balancing) functionality.  For now, code that used
these should "manually" handle and re-publish events as needed.
2018-08-27 16:49:35 -05:00
Johanna Amann
27d47314f7 Merge remote-tracking branch 'origin/master' into topic/johanna/tls-more-data 2018-08-27 09:25:40 -07:00
Johanna Amann
e055f9b36b Merge remote-tracking branch 'origin/topic/dnthayer/ticket1963'
* origin/topic/dnthayer/ticket1963:
  Add a missing initializer to a runtime option
  Convert more redef-able constants to runtime options
2018-08-24 18:05:34 -07:00
Johanna Amann
82cefd23c4 Fix base/misc/version.bro version parsing
Turns out that base/misc/version.bro did not parse Bro versions
correctly in case the version is just 2.5-12 or similar. This commit
fixes this oversight and adds a few more small testcases.
2018-08-24 17:25:16 -07:00
Johanna Amann
4b40b6ebe4 Merge remote-tracking branch 'origin/master' into topic/johanna/tls-more-data 2018-08-24 15:36:34 -07:00
Daniel Thayer
fd1a23ea66 Add a missing initializer to a runtime option 2018-08-24 17:03:02 -05:00
Daniel Thayer
8b0b7d3304 Merge remote-tracking branch 'origin/master' into topic/dnthayer/ticket1963 2018-08-24 16:06:05 -05:00
Daniel Thayer
01a899255e Convert more redef-able constants to runtime options 2018-08-24 16:05:44 -05:00
Johanna Amann
b2a0418dc5 Final touches to SSL events with record layer version. 2018-08-23 14:18:38 -07:00
Jon Siwek
620cd671ba Fix tracking of DCE-RPC context identifier mappings
This adds previously-missing support for "Alter Context"
request/response PDUs (initial patch contributed by Mark Fernandez).

Also, context ID arguments were added to dce_rpc_bind, dce_rpc_request,
and dce_rpc_response in order to properly track what endpoint/operation
a given opnum maps to.
2018-08-23 15:11:38 -05:00
Jon Siwek
b679a51376 Merge remote-tracking branch 'origin/topic/jazoff/fix-snaplen'
* origin/topic/jazoff/fix-snaplen:
  problem: default snaplen is too small for jumbo frames
2018-08-21 14:54:55 -05:00
Johanna Amann
aa2488fb69 Merge remote-tracking branch 'origin/master' into topic/johanna/tls-more-data 2018-08-20 16:10:21 -07:00
Jon Siwek
bcf97f70ea Merge remote-tracking branch 'origin/topic/jsiwek/empty-lines'
* origin/topic/jsiwek/empty-lines:
  Add 'smtp_excessive_pending_cmds' weird
  Fix SMTP command string comparisons
  Improve handling of empty lines in several text protocol analyzers
  Add rate-limiting sampling mechanism for weird events
  Teach timestamp canonifier about timestamps before ~2001
2018-08-20 15:35:16 -05:00
Jon Siwek
6595b21e2e Merge remote-tracking branch 'origin/topic/dnthayer/ticket1963'
* origin/topic/dnthayer/ticket1963:
  Remove unused redef-able constants
  Convert some redef-able constants to runtime options
2018-08-20 12:44:58 -05:00
Daniel Thayer
1a4629b0dc Merge remote-tracking branch 'origin/master' into topic/dnthayer/ticket1963 2018-08-17 14:11:47 -05:00
Johanna Amann
b1dbd757a6 Merge remote-tracking branch 'origin/master' into topic/johanna/tls-more-data 2018-08-17 11:52:00 -07:00
Daniel Thayer
a71ed6f781 Merge remote-tracking branch 'origin/master' into topic/dnthayer/doc-fixes-for-2.6 2018-08-17 11:34:16 -05:00
Jon Siwek
fcabd72b92 BIT-1815: move SMB::write_cmd_log functionality into policy/ script
The option is removed, but same functionality is now enabled simply
by loading policy/protocols/smb/log-cmds.bro
2018-08-17 11:15:18 -05:00
Jon Siwek
fc7d3cd981 Fix possible race in netcontrol acld/broker plugins
Best to subscribe before connecting
2018-08-17 10:31:31 -05:00
Jon Siwek
a04c76c035 Enable SMB by default by moving scripts from policy/ to base/ 2018-08-16 17:23:28 -05:00
Jon Siwek
7fdf621a1d BIT-1924: add DHCP port to software.log for completeness 2018-08-16 16:08:29 -05:00
Daniel Thayer
c941c565a6 Replace references to libgeoip in the documentation
Replace references to the old libgeoip library with "libmaxminddb" or
"GeoIP support".
2018-08-16 15:45:58 -05:00
Jon Siwek
81a8961f16 BIT-1858: fix logged-names for DNS RR types 44 and 45 2018-08-16 14:13:31 -05:00
Jon Siwek
da9f91fc19 Add env. variables to override Broker listen/connect retry intervals
And use them to default retries to 1sec for all unit tests.
2018-08-16 12:16:03 -05:00
Daniel Thayer
9291fef6d2 Merge remote-tracking branch 'origin/master' into topic/dnthayer/doc-fixes-for-2.6 2018-08-15 15:27:44 -05:00
Daniel Thayer
d7be90c3ca Remove unused redef-able constants 2018-08-15 10:30:09 -05:00
Daniel Thayer
dc0904a7f3 Convert some redef-able constants to runtime options 2018-08-15 10:17:14 -05:00
Jon Siwek
5821c16490 Fix SumStats::observe key normalization logic
The loop over Reducers in SumStats::observe performs a key normalization
and inadvertently modifies the key used for subsequent iterations.

Reported by Jim Mellander.
2018-08-13 17:53:26 -05:00
Jon Siwek
67524f26d5 Immediately apply broker subscriptions made during bro_init()
Otherwise that's begging for unit test failures due to races
2018-08-10 17:18:21 -05:00
Jon Siwek
083947af41 Update default broker threading configuration
Now defaults to a max of 4 threads typically indepedent of core
count (previously could go up to a hard cap of 8).  Also now allow
controlling this setting via BRO_BROKER_MAX_THREADS environment
variable.
2018-08-10 17:08:26 -05:00
Jon Siwek
2c9dbdd055 Fix cluster layout graphic and doc warnings 2018-08-08 13:03:45 -05:00
Johanna Amann
07043d6dd8 Merge branch 'topic/zhongjie/fix-sig' of https://github.com/bro/bro
* 'topic/zhongjie/fix-sig' of https://github.com/bro/bro:
  Added missing tcp-state for signature dpd_rfb_server
2018-08-08 09:34:51 -07:00
Jon Siwek
599af26496 Merge remote-tracking branch 'origin/topic/vern/vec-append'
* origin/topic/vern/vec-append:
  d'oh, still have a (deprecated) string_array rather than string_vector
  forgot to update test suite results for v += e
  reap the fruits of v += e
  test case for v += e
  documentation of v += e
  v += e implemented

Fixed a mistake in find_ip_addresses()
2018-08-03 11:38:40 -05:00
Jon Siwek
5804c940f1 Merge remote-tracking branch 'origin/topic/vern/perf-history'
* origin/topic/vern/perf-history:
  only generate history threshold events for > 1 instance mention those events in NEWS
  a different sort of history update
  'W' for zero window implemented; logarithmic 'T'/'C'/'W' history repetitions

I reverted a change that made TCP window tracking unconditional (possibly
accepting out-of-order packets) until further verification of test suite
changes.
2018-08-01 16:37:10 -05:00
Daniel Thayer
021115a6fa Merge remote-tracking branch 'origin/master' into topic/dnthayer/doc-fixes-for-2.6 2018-08-01 12:19:17 -05:00
Jonathan Perkins
1ae7d3b349 Replace GeoIP Legacy DB support with MaxMind DB support
This updates the "lookup_location" and "lookup_asn" BIFs to use
libmaxminddb.  The motivation for this is that MaxMind is discontinuing
GeoLite Legacy databases: no updates after April 1, 2018, no downloads
after January 2, 2019.  It's also noted that all GeoIP Legacy databases
may be discontinued as they are superseded by GeoIP2.
2018-07-30 16:04:58 -05:00
Daniel Thayer
0e2dbfe7c3 Fix typos and formatting issues in config framework docs 2018-07-30 14:12:25 -05:00
Jon Siwek
35827eeb31 Add rate-limiting sampling mechanism for weird events
The generation of weird events, by default, are now rate-limited
according to these tunable options:

  - Weird::sampling_whitelist
  - Weird::sampling_threshold
  - Weird::sampling_rate
  - Weird::sampling_duration

The new get_reporter_stats() BIF also allows one to query the
total number of weirds generated (pre-sampling) which the new
policy/misc/weird-stats.bro script uses periodically to populate
a weird_stats.log.

There's also new reporter BIFs to allow generating weirds from the
script-layer such that they go through the same, internal
rate-limiting/sampling mechanisms:

  - Reporter::conn_weird
  - Reporter::flow_weird
  - Reporter::net_weird

Some of the code was adapted from previous work by Johanna Amann.
2018-07-26 19:57:36 -05:00
Vern Paxson
f7358a3351 d'oh, still have a (deprecated) string_array rather than string_vector 2018-07-26 14:35:57 -07:00
Vern Paxson
88fd7510c6 reap the fruits of v += e 2018-07-26 12:51:36 -07:00
Zhongjie Wang
e31563069b Added missing tcp-state for signature dpd_rfb_server 2018-07-24 14:07:12 -07:00
Robin Sommer
f4cd567f84 Merge remote-tracking branch 'origin/topic/johanna/config-cluster'
BIT-1958 #merged

* origin/topic/johanna/config-cluster:
  Add vector to read_config_cluster test.
  Fix special-case-bug for vectors in UnaryExpr.
  Config: another cluster test-case, this time reading in a file.
  Add sending of values to nodes that dropped out.
  Continue work on config framework clusterization.
  Teach Option::set to unwrap Broker::Data values
  Start clusterizing configuration framework.
2018-07-24 01:36:59 +00:00
Jon Siwek
385350a1f3 Make Broker congestion queue size tunable and increase default 2018-07-20 12:14:33 -05:00
Vern Paxson
73349362a3 'W' for zero window implemented; logarithmic 'T'/'C'/'W' history repetitions 2018-07-09 13:05:10 -07:00
Johanna Amann
c28f1ae0ce Add sending of values to nodes that dropped out.
The only node that cannot be recovered is the manager - and the manager
should just re-read its own configuration and be ok :)
2018-06-29 13:10:00 -07:00
Johanna Amann
5f07673e25 Merge remote-tracking branch 'origin/master' into topic/johanna/config-cluster 2018-06-29 08:30:29 -07:00
Daniel Thayer
ceefb6edaf Fix minor typos in broker reference documentation 2018-06-27 13:00:09 -05:00
Johanna Amann
d6990119db Continue work on config framework clusterization.
This does not currently work.
2018-06-27 10:47:17 -07:00