Commit graph

1835 commits

Author SHA1 Message Date
Jon Siwek
81a8961f16 BIT-1858: fix logged-names for DNS RR types 44 and 45 2018-08-16 14:13:31 -05:00
Jon Siwek
da9f91fc19 Add env. variables to override Broker listen/connect retry intervals
And use them to default retries to 1sec for all unit tests.
2018-08-16 12:16:03 -05:00
Jon Siwek
5821c16490 Fix SumStats::observe key normalization logic
The loop over Reducers in SumStats::observe performs a key normalization
and inadvertently modifies the key used for subsequent iterations.

Reported by Jim Mellander.
2018-08-13 17:53:26 -05:00
Jon Siwek
67524f26d5 Immediately apply broker subscriptions made during bro_init()
Otherwise that's begging for unit test failures due to races
2018-08-10 17:18:21 -05:00
Jon Siwek
083947af41 Update default broker threading configuration
Now defaults to a max of 4 threads typically indepedent of core
count (previously could go up to a hard cap of 8).  Also now allow
controlling this setting via BRO_BROKER_MAX_THREADS environment
variable.
2018-08-10 17:08:26 -05:00
Jon Siwek
2c9dbdd055 Fix cluster layout graphic and doc warnings 2018-08-08 13:03:45 -05:00
Johanna Amann
07043d6dd8 Merge branch 'topic/zhongjie/fix-sig' of https://github.com/bro/bro
* 'topic/zhongjie/fix-sig' of https://github.com/bro/bro:
  Added missing tcp-state for signature dpd_rfb_server
2018-08-08 09:34:51 -07:00
Jon Siwek
599af26496 Merge remote-tracking branch 'origin/topic/vern/vec-append'
* origin/topic/vern/vec-append:
  d'oh, still have a (deprecated) string_array rather than string_vector
  forgot to update test suite results for v += e
  reap the fruits of v += e
  test case for v += e
  documentation of v += e
  v += e implemented

Fixed a mistake in find_ip_addresses()
2018-08-03 11:38:40 -05:00
Jon Siwek
5804c940f1 Merge remote-tracking branch 'origin/topic/vern/perf-history'
* origin/topic/vern/perf-history:
  only generate history threshold events for > 1 instance mention those events in NEWS
  a different sort of history update
  'W' for zero window implemented; logarithmic 'T'/'C'/'W' history repetitions

I reverted a change that made TCP window tracking unconditional (possibly
accepting out-of-order packets) until further verification of test suite
changes.
2018-08-01 16:37:10 -05:00
Jonathan Perkins
1ae7d3b349 Replace GeoIP Legacy DB support with MaxMind DB support
This updates the "lookup_location" and "lookup_asn" BIFs to use
libmaxminddb.  The motivation for this is that MaxMind is discontinuing
GeoLite Legacy databases: no updates after April 1, 2018, no downloads
after January 2, 2019.  It's also noted that all GeoIP Legacy databases
may be discontinued as they are superseded by GeoIP2.
2018-07-30 16:04:58 -05:00
Vern Paxson
f7358a3351 d'oh, still have a (deprecated) string_array rather than string_vector 2018-07-26 14:35:57 -07:00
Vern Paxson
88fd7510c6 reap the fruits of v += e 2018-07-26 12:51:36 -07:00
Zhongjie Wang
e31563069b Added missing tcp-state for signature dpd_rfb_server 2018-07-24 14:07:12 -07:00
Robin Sommer
f4cd567f84 Merge remote-tracking branch 'origin/topic/johanna/config-cluster'
BIT-1958 #merged

* origin/topic/johanna/config-cluster:
  Add vector to read_config_cluster test.
  Fix special-case-bug for vectors in UnaryExpr.
  Config: another cluster test-case, this time reading in a file.
  Add sending of values to nodes that dropped out.
  Continue work on config framework clusterization.
  Teach Option::set to unwrap Broker::Data values
  Start clusterizing configuration framework.
2018-07-24 01:36:59 +00:00
Jon Siwek
385350a1f3 Make Broker congestion queue size tunable and increase default 2018-07-20 12:14:33 -05:00
Vern Paxson
73349362a3 'W' for zero window implemented; logarithmic 'T'/'C'/'W' history repetitions 2018-07-09 13:05:10 -07:00
Johanna Amann
c28f1ae0ce Add sending of values to nodes that dropped out.
The only node that cannot be recovered is the manager - and the manager
should just re-read its own configuration and be ok :)
2018-06-29 13:10:00 -07:00
Johanna Amann
5f07673e25 Merge remote-tracking branch 'origin/master' into topic/johanna/config-cluster 2018-06-29 08:30:29 -07:00
Daniel Thayer
ceefb6edaf Fix minor typos in broker reference documentation 2018-06-27 13:00:09 -05:00
Johanna Amann
d6990119db Continue work on config framework clusterization.
This does not currently work.
2018-06-27 10:47:17 -07:00
Johanna Amann
948bb4b9ec Merge remote-tracking branch 'origin/master' into topic/johanna/config-cluster 2018-06-22 10:43:09 -07:00
Jon Siwek
8ee92c70da Add ability for BroControl to skip cluster setup
It does this by setting the BROCTL_CHECK_CONFIG env. variable.
Related to BIT-1938.
2018-06-21 11:57:18 -05:00
Jon Siwek
c11039cb73 Make old comm. system usages an error unless old_comm_usage_is_ok is set 2018-06-15 17:15:46 -05:00
Jon Siwek
e578c1c231 Add Broker::max_threads and Broker::max_sleep tuning options
The former replaces the pcap vs. live versions of the same tuning
option.  If a user does not change these, Bro makes some internal
decisions that may help avoid performance problems on systems with high
core counts: the number of CAF threads is capped at 8 and the maximum
sleep duration for under-utilized threads is increased to 64ms (CAF's
default is 10ms).
2018-06-14 12:57:23 -05:00
Jon Siwek
c9fe9a943c Add Broker::max_live_threads and Broker::max_pcap_threads tunables
These may be used to change the number of scheduler threads that the
underlying CAF library creates.  In pcap mode, it's currently hardcoded
to the minimal 4 threads due to potentially significant overhead in CAF.
2018-06-12 13:54:43 -05:00
Robin Sommer
280e8c7dcc Merge branch 'master' of ssh://git.bro.org/bro 2018-06-08 16:41:28 +00:00
Robin Sommer
554e8cc73d Merge remote-tracking branch 'origin/topic/jsiwek/broker-listen-env'
BIT-1935 #merged

* origin/topic/jsiwek/broker-listen-env:
  Allow BRO_DEFAULT_LISTEN_ADDRESS to control broker listen address
2018-06-08 16:38:31 +00:00
Jon Siwek
b51e6f39dd Correct conn history field documentation 2018-06-08 10:43:32 -05:00
Jon Siwek
a43c7d0691 Merge remote-tracking branch 'origin/topic/jsiwek/ncp-fixes'
* origin/topic/jsiwek/ncp-fixes:
  Add NCP::max_frame_size tuning option
  Migrate NCP analyzer to use latest analyzer API
2018-06-05 11:32:38 -05:00
Seth Hall
cd18d96205 Removed a few more discovered UTF-8 characters in Bro scripts. 2018-06-02 04:57:48 -04:00
Seth Hall
51f20136d5 Remove some UTF-8 characters that snuck into a few strings. 2018-06-02 04:37:08 -04:00
Jon Siwek
1b4e0116f4 Allow BRO_DEFAULT_LISTEN_ADDRESS to control broker listen address
This environment variable is now set to listen only on IPv4 loopback
when running unit tests (instead of using the default INADDR_ANY).

This also moves some of the @loads out from init-bare.bro into a new
init-frameworks-and-bifs.bro in order to better support calling BIFs
(like `getenv`) from variable initializations in those particular
frameworks.
2018-06-01 15:38:11 -05:00
Johanna Amann
327acf6555 KRB: do not set keytab by default.
Only enable decryption if a user purposefully sets a keytab.
2018-06-01 12:46:26 -07:00
Johanna Amann
5d5dd65cab Merge branch 'krb_changes2' of https://github.com/jwallior/bro
I did some cleanup work.
2018-06-01 12:28:45 -07:00
Johanna Amann
44dfcb7c6a Start clusterizing configuration framework.
This is not finished and currently does not work due Broker not liking
to serialize into any types.
2018-05-29 14:08:05 -07:00
Jon Siwek
fe478877c6 Change Intel framework to round-robin insertion events across proxies 2018-05-24 14:36:22 -05:00
Jon Siwek
45178f3051 Add a counter for number of alive nodes within a given cluster pool 2018-05-24 14:33:35 -05:00
Jon Siwek
95ea84e60e Fix how cluster framework tracks worker count 2018-05-24 14:32:45 -05:00
Jon Siwek
58864c358c Add NCP::max_frame_size tuning option
This helps prevent excessive allocations based on message lengths
taken from NCP headers.
2018-05-22 18:27:52 -05:00
Robin Sommer
fe7e1ee7f0 Merge topic/actor-system throug a squashed commit. 2018-05-18 22:39:23 +00:00
Johanna Amann
2b24e04ada Add non-standard experimental Google post-quantum ciphers 2018-05-17 15:54:48 -07:00
Julien Wallior
98a430c1eb Decrypt the krb ticket and send authentication data out. 2018-05-10 10:14:30 -04:00
Stephen Hosom
6611d28920 Fix for BIT-1927: notice/extend-email relocation
This is a fairly straightforward change. Previously, users had no
control over whether this script was loaded. By relocating it to
policy, users can now choose whether or not this is necessary
functionality without modifying core Bro scripts.
2018-05-10 09:13:52 -04:00
Jon Siwek
81133f3116 Merge remote-tracking branch 'origin/topic/seth/dhcp-update'
* origin/topic/seth/dhcp-update:
  Rework to the DHCP analyzer.
  First step of DHCP analyzer rearchitecture.
  Add .btest scripts for dhck_ack and dhcp_discover messages verifying that new options are correctly reported in dhcp.log records.
  Extend DHCP protocol analyzer with new options.

BIT-1924 #merged

Additional changes:

* Removed known-hosts.bro as the only thing populating its table was
  the already-removed known-hosts-and-devices.bro.  So a
  known_devices.log will no longer be generated.

* In dhcp-options.pac, the process_relay_agent_inf_option had a memleak
  and also process_auto_proxy_config_option looked like it accessed one
  byte past the end of the available bytestring, so fixed those.
2018-05-01 18:06:41 -05:00
Seth Hall
e5e1037e91 Fix subnet expiration in the intel framework. 2018-04-27 16:42:19 -04:00
Luciano Mammino
ff84e4faa3
Update consts.bro
Removed duplicated entries (`000001a0-0000-0000-c000-000000000046`, with `i=3, i=4`) and related comments line

This change should make this file more readable and it will be easier to convert it to other formats (personally I found it out while I was trying to convert this to a JavaScript structure).
2018-04-25 16:25:34 +01:00
Seth Hall
a80131c06e Updating the defined SMB2 dialects to match Microsofts current docs. 2018-04-05 17:13:10 -04:00
Seth Hall
68d0085d35 Merge branch 'smb_changes' of https://github.com/jwallior/bro into jwallior-smb_changes 2018-04-02 17:06:30 -04:00
Johanna Amann
f39efd0317 Recognize TLS 1.3 negotiation correctly.
The way in which TLS 1.3 is negotiated was changed slightly in later
revisions of the standard. The final version is only sent in an
extension - while the version field in the server hello still shows TLS
1.2.

This patch makes ssl.log show the correct version again.
2018-03-27 14:58:06 -07:00
Daniel Thayer
f3e42874b8 Improve config framework documentation comments
Fixed typos and formatting.
2018-03-15 14:16:00 -05:00