Commit graph

2143 commits

Author SHA1 Message Date
Jon Siwek
81a8961f16 BIT-1858: fix logged-names for DNS RR types 44 and 45 2018-08-16 14:13:31 -05:00
Jon Siwek
da9f91fc19 Add env. variables to override Broker listen/connect retry intervals
And use them to default retries to 1sec for all unit tests.
2018-08-16 12:16:03 -05:00
Jon Siwek
5821c16490 Fix SumStats::observe key normalization logic
The loop over Reducers in SumStats::observe performs a key normalization
and inadvertently modifies the key used for subsequent iterations.

Reported by Jim Mellander.
2018-08-13 17:53:26 -05:00
Jon Siwek
a2f8d81fb6 Fix validate-certs.bro comments 2018-08-13 10:20:58 -05:00
Jon Siwek
67524f26d5 Immediately apply broker subscriptions made during bro_init()
Otherwise that's begging for unit test failures due to races
2018-08-10 17:18:21 -05:00
Jon Siwek
083947af41 Update default broker threading configuration
Now defaults to a max of 4 threads typically indepedent of core
count (previously could go up to a hard cap of 8).  Also now allow
controlling this setting via BRO_BROKER_MAX_THREADS environment
variable.
2018-08-10 17:08:26 -05:00
Jon Siwek
2c9dbdd055 Fix cluster layout graphic and doc warnings 2018-08-08 13:03:45 -05:00
Johanna Amann
07043d6dd8 Merge branch 'topic/zhongjie/fix-sig' of https://github.com/bro/bro
* 'topic/zhongjie/fix-sig' of https://github.com/bro/bro:
  Added missing tcp-state for signature dpd_rfb_server
2018-08-08 09:34:51 -07:00
Jon Siwek
e6042940dc Fix (non)suppression of proxy-bound events in known-*.bro scripts
When not using data stores, these scripts were intended to suppress
sending duplicate events to proxies by looking up the key in the local
cache.
2018-08-06 17:04:42 -05:00
Jon Siwek
599af26496 Merge remote-tracking branch 'origin/topic/vern/vec-append'
* origin/topic/vern/vec-append:
  d'oh, still have a (deprecated) string_array rather than string_vector
  forgot to update test suite results for v += e
  reap the fruits of v += e
  test case for v += e
  documentation of v += e
  v += e implemented

Fixed a mistake in find_ip_addresses()
2018-08-03 11:38:40 -05:00
Jon Siwek
5804c940f1 Merge remote-tracking branch 'origin/topic/vern/perf-history'
* origin/topic/vern/perf-history:
  only generate history threshold events for > 1 instance mention those events in NEWS
  a different sort of history update
  'W' for zero window implemented; logarithmic 'T'/'C'/'W' history repetitions

I reverted a change that made TCP window tracking unconditional (possibly
accepting out-of-order packets) until further verification of test suite
changes.
2018-08-01 16:37:10 -05:00
Jonathan Perkins
1ae7d3b349 Replace GeoIP Legacy DB support with MaxMind DB support
This updates the "lookup_location" and "lookup_asn" BIFs to use
libmaxminddb.  The motivation for this is that MaxMind is discontinuing
GeoLite Legacy databases: no updates after April 1, 2018, no downloads
after January 2, 2019.  It's also noted that all GeoIP Legacy databases
may be discontinued as they are superseded by GeoIP2.
2018-07-30 16:04:58 -05:00
Vern Paxson
f7358a3351 d'oh, still have a (deprecated) string_array rather than string_vector 2018-07-26 14:35:57 -07:00
Vern Paxson
88fd7510c6 reap the fruits of v += e 2018-07-26 12:51:36 -07:00
Zhongjie Wang
e31563069b Added missing tcp-state for signature dpd_rfb_server 2018-07-24 14:07:12 -07:00
Robin Sommer
f4cd567f84 Merge remote-tracking branch 'origin/topic/johanna/config-cluster'
BIT-1958 #merged

* origin/topic/johanna/config-cluster:
  Add vector to read_config_cluster test.
  Fix special-case-bug for vectors in UnaryExpr.
  Config: another cluster test-case, this time reading in a file.
  Add sending of values to nodes that dropped out.
  Continue work on config framework clusterization.
  Teach Option::set to unwrap Broker::Data values
  Start clusterizing configuration framework.
2018-07-24 01:36:59 +00:00
Jon Siwek
385350a1f3 Make Broker congestion queue size tunable and increase default 2018-07-20 12:14:33 -05:00
Jon Siwek
6215d45f10 Improve control framework id-update/test output 2018-07-20 11:59:40 -05:00
Johanna Amann
da58f9d4a6 Merge remote-tracking branch 'origin/master' into topic/johanna/config-cluster 2018-07-18 09:56:01 -07:00
Vern Paxson
73349362a3 'W' for zero window implemented; logarithmic 'T'/'C'/'W' history repetitions 2018-07-09 13:05:10 -07:00
Liviu Valsan
acf1c591ea Added support for making optional the extraction of DNS entries from X509 SAN as Intel::seen records. 2018-07-03 15:08:21 +02:00
Johanna Amann
c28f1ae0ce Add sending of values to nodes that dropped out.
The only node that cannot be recovered is the manager - and the manager
should just re-read its own configuration and be ok :)
2018-06-29 13:10:00 -07:00
Johanna Amann
5f07673e25 Merge remote-tracking branch 'origin/master' into topic/johanna/config-cluster 2018-06-29 08:30:29 -07:00
Daniel Thayer
ceefb6edaf Fix minor typos in broker reference documentation 2018-06-27 13:00:09 -05:00
Johanna Amann
d6990119db Continue work on config framework clusterization.
This does not currently work.
2018-06-27 10:47:17 -07:00
Johanna Amann
948bb4b9ec Merge remote-tracking branch 'origin/master' into topic/johanna/config-cluster 2018-06-22 10:43:09 -07:00
Jon Siwek
8ee92c70da Add ability for BroControl to skip cluster setup
It does this by setting the BROCTL_CHECK_CONFIG env. variable.
Related to BIT-1938.
2018-06-21 11:57:18 -05:00
Jon Siwek
c11039cb73 Make old comm. system usages an error unless old_comm_usage_is_ok is set 2018-06-15 17:15:46 -05:00
Jon Siwek
e578c1c231 Add Broker::max_threads and Broker::max_sleep tuning options
The former replaces the pcap vs. live versions of the same tuning
option.  If a user does not change these, Bro makes some internal
decisions that may help avoid performance problems on systems with high
core counts: the number of CAF threads is capped at 8 and the maximum
sleep duration for under-utilized threads is increased to 64ms (CAF's
default is 10ms).
2018-06-14 12:57:23 -05:00
Jon Siwek
c9fe9a943c Add Broker::max_live_threads and Broker::max_pcap_threads tunables
These may be used to change the number of scheduler threads that the
underlying CAF library creates.  In pcap mode, it's currently hardcoded
to the minimal 4 threads due to potentially significant overhead in CAF.
2018-06-12 13:54:43 -05:00
Robin Sommer
280e8c7dcc Merge branch 'master' of ssh://git.bro.org/bro 2018-06-08 16:41:28 +00:00
Robin Sommer
554e8cc73d Merge remote-tracking branch 'origin/topic/jsiwek/broker-listen-env'
BIT-1935 #merged

* origin/topic/jsiwek/broker-listen-env:
  Allow BRO_DEFAULT_LISTEN_ADDRESS to control broker listen address
2018-06-08 16:38:31 +00:00
Jon Siwek
b51e6f39dd Correct conn history field documentation 2018-06-08 10:43:32 -05:00
Jon Siwek
a43c7d0691 Merge remote-tracking branch 'origin/topic/jsiwek/ncp-fixes'
* origin/topic/jsiwek/ncp-fixes:
  Add NCP::max_frame_size tuning option
  Migrate NCP analyzer to use latest analyzer API
2018-06-05 11:32:38 -05:00
Seth Hall
cd18d96205 Removed a few more discovered UTF-8 characters in Bro scripts. 2018-06-02 04:57:48 -04:00
Seth Hall
51f20136d5 Remove some UTF-8 characters that snuck into a few strings. 2018-06-02 04:37:08 -04:00
Jon Siwek
1b4e0116f4 Allow BRO_DEFAULT_LISTEN_ADDRESS to control broker listen address
This environment variable is now set to listen only on IPv4 loopback
when running unit tests (instead of using the default INADDR_ANY).

This also moves some of the @loads out from init-bare.bro into a new
init-frameworks-and-bifs.bro in order to better support calling BIFs
(like `getenv`) from variable initializations in those particular
frameworks.
2018-06-01 15:38:11 -05:00
Johanna Amann
327acf6555 KRB: do not set keytab by default.
Only enable decryption if a user purposefully sets a keytab.
2018-06-01 12:46:26 -07:00
Johanna Amann
5d5dd65cab Merge branch 'krb_changes2' of https://github.com/jwallior/bro
I did some cleanup work.
2018-06-01 12:28:45 -07:00
Johanna Amann
b2dc7ffb26 Merge branch 'smb2-updates' of https://github.com/dtrejod/bro 2018-05-31 21:13:20 -07:00
Johanna Amann
1fec186c39 Fix SCT validation when invalid certificates are in chain.
At the moment it would try to access an unset optional in this case.
2018-05-31 14:58:46 -07:00
Johanna Amann
44dfcb7c6a Start clusterizing configuration framework.
This is not finished and currently does not work due Broker not liking
to serialize into any types.
2018-05-29 14:08:05 -07:00
Jon Siwek
fe478877c6 Change Intel framework to round-robin insertion events across proxies 2018-05-24 14:36:22 -05:00
Jon Siwek
45178f3051 Add a counter for number of alive nodes within a given cluster pool 2018-05-24 14:33:35 -05:00
Jon Siwek
95ea84e60e Fix how cluster framework tracks worker count 2018-05-24 14:32:45 -05:00
Jon Siwek
58864c358c Add NCP::max_frame_size tuning option
This helps prevent excessive allocations based on message lengths
taken from NCP headers.
2018-05-22 18:27:52 -05:00
Robin Sommer
fe7e1ee7f0 Merge topic/actor-system throug a squashed commit. 2018-05-18 22:39:23 +00:00
Johanna Amann
2b24e04ada Add non-standard experimental Google post-quantum ciphers 2018-05-17 15:54:48 -07:00
Jon Siwek
057c982b9d Merge branch 'BIT-1927' of https://github.com/hosom/bro
* 'BIT-1927' of https://github.com/hosom/bro:
  Fix for BIT-1927: notice/extend-email relocation

And updated coverage unit tests.

BIT-1927 #close
2018-05-10 11:28:08 -05:00
Julien Wallior
98a430c1eb Decrypt the krb ticket and send authentication data out. 2018-05-10 10:14:30 -04:00