Commit graph

2026 commits

Author SHA1 Message Date
Jon Siwek
385350a1f3 Make Broker congestion queue size tunable and increase default 2018-07-20 12:14:33 -05:00
Vern Paxson
73349362a3 'W' for zero window implemented; logarithmic 'T'/'C'/'W' history repetitions 2018-07-09 13:05:10 -07:00
Johanna Amann
c28f1ae0ce Add sending of values to nodes that dropped out.
The only node that cannot be recovered is the manager - and the manager
should just re-read its own configuration and be ok :)
2018-06-29 13:10:00 -07:00
Johanna Amann
5f07673e25 Merge remote-tracking branch 'origin/master' into topic/johanna/config-cluster 2018-06-29 08:30:29 -07:00
Daniel Thayer
ceefb6edaf Fix minor typos in broker reference documentation 2018-06-27 13:00:09 -05:00
Johanna Amann
d6990119db Continue work on config framework clusterization.
This does not currently work.
2018-06-27 10:47:17 -07:00
Johanna Amann
948bb4b9ec Merge remote-tracking branch 'origin/master' into topic/johanna/config-cluster 2018-06-22 10:43:09 -07:00
Jon Siwek
8ee92c70da Add ability for BroControl to skip cluster setup
It does this by setting the BROCTL_CHECK_CONFIG env. variable.
Related to BIT-1938.
2018-06-21 11:57:18 -05:00
Jon Siwek
c11039cb73 Make old comm. system usages an error unless old_comm_usage_is_ok is set 2018-06-15 17:15:46 -05:00
Jon Siwek
e578c1c231 Add Broker::max_threads and Broker::max_sleep tuning options
The former replaces the pcap vs. live versions of the same tuning
option.  If a user does not change these, Bro makes some internal
decisions that may help avoid performance problems on systems with high
core counts: the number of CAF threads is capped at 8 and the maximum
sleep duration for under-utilized threads is increased to 64ms (CAF's
default is 10ms).
2018-06-14 12:57:23 -05:00
Jon Siwek
c9fe9a943c Add Broker::max_live_threads and Broker::max_pcap_threads tunables
These may be used to change the number of scheduler threads that the
underlying CAF library creates.  In pcap mode, it's currently hardcoded
to the minimal 4 threads due to potentially significant overhead in CAF.
2018-06-12 13:54:43 -05:00
Robin Sommer
280e8c7dcc Merge branch 'master' of ssh://git.bro.org/bro 2018-06-08 16:41:28 +00:00
Robin Sommer
554e8cc73d Merge remote-tracking branch 'origin/topic/jsiwek/broker-listen-env'
BIT-1935 #merged

* origin/topic/jsiwek/broker-listen-env:
  Allow BRO_DEFAULT_LISTEN_ADDRESS to control broker listen address
2018-06-08 16:38:31 +00:00
Jon Siwek
b51e6f39dd Correct conn history field documentation 2018-06-08 10:43:32 -05:00
Jon Siwek
a43c7d0691 Merge remote-tracking branch 'origin/topic/jsiwek/ncp-fixes'
* origin/topic/jsiwek/ncp-fixes:
  Add NCP::max_frame_size tuning option
  Migrate NCP analyzer to use latest analyzer API
2018-06-05 11:32:38 -05:00
Seth Hall
cd18d96205 Removed a few more discovered UTF-8 characters in Bro scripts. 2018-06-02 04:57:48 -04:00
Seth Hall
51f20136d5 Remove some UTF-8 characters that snuck into a few strings. 2018-06-02 04:37:08 -04:00
Jon Siwek
1b4e0116f4 Allow BRO_DEFAULT_LISTEN_ADDRESS to control broker listen address
This environment variable is now set to listen only on IPv4 loopback
when running unit tests (instead of using the default INADDR_ANY).

This also moves some of the @loads out from init-bare.bro into a new
init-frameworks-and-bifs.bro in order to better support calling BIFs
(like `getenv`) from variable initializations in those particular
frameworks.
2018-06-01 15:38:11 -05:00
Johanna Amann
327acf6555 KRB: do not set keytab by default.
Only enable decryption if a user purposefully sets a keytab.
2018-06-01 12:46:26 -07:00
Johanna Amann
5d5dd65cab Merge branch 'krb_changes2' of https://github.com/jwallior/bro
I did some cleanup work.
2018-06-01 12:28:45 -07:00
Johanna Amann
44dfcb7c6a Start clusterizing configuration framework.
This is not finished and currently does not work due Broker not liking
to serialize into any types.
2018-05-29 14:08:05 -07:00
Jon Siwek
fe478877c6 Change Intel framework to round-robin insertion events across proxies 2018-05-24 14:36:22 -05:00
Jon Siwek
45178f3051 Add a counter for number of alive nodes within a given cluster pool 2018-05-24 14:33:35 -05:00
Jon Siwek
95ea84e60e Fix how cluster framework tracks worker count 2018-05-24 14:32:45 -05:00
Jon Siwek
58864c358c Add NCP::max_frame_size tuning option
This helps prevent excessive allocations based on message lengths
taken from NCP headers.
2018-05-22 18:27:52 -05:00
Robin Sommer
fe7e1ee7f0 Merge topic/actor-system throug a squashed commit. 2018-05-18 22:39:23 +00:00
Johanna Amann
2b24e04ada Add non-standard experimental Google post-quantum ciphers 2018-05-17 15:54:48 -07:00
Julien Wallior
98a430c1eb Decrypt the krb ticket and send authentication data out. 2018-05-10 10:14:30 -04:00
Stephen Hosom
6611d28920 Fix for BIT-1927: notice/extend-email relocation
This is a fairly straightforward change. Previously, users had no
control over whether this script was loaded. By relocating it to
policy, users can now choose whether or not this is necessary
functionality without modifying core Bro scripts.
2018-05-10 09:13:52 -04:00
Jon Siwek
81133f3116 Merge remote-tracking branch 'origin/topic/seth/dhcp-update'
* origin/topic/seth/dhcp-update:
  Rework to the DHCP analyzer.
  First step of DHCP analyzer rearchitecture.
  Add .btest scripts for dhck_ack and dhcp_discover messages verifying that new options are correctly reported in dhcp.log records.
  Extend DHCP protocol analyzer with new options.

BIT-1924 #merged

Additional changes:

* Removed known-hosts.bro as the only thing populating its table was
  the already-removed known-hosts-and-devices.bro.  So a
  known_devices.log will no longer be generated.

* In dhcp-options.pac, the process_relay_agent_inf_option had a memleak
  and also process_auto_proxy_config_option looked like it accessed one
  byte past the end of the available bytestring, so fixed those.
2018-05-01 18:06:41 -05:00
Seth Hall
e5e1037e91 Fix subnet expiration in the intel framework. 2018-04-27 16:42:19 -04:00
Luciano Mammino
ff84e4faa3
Update consts.bro
Removed duplicated entries (`000001a0-0000-0000-c000-000000000046`, with `i=3, i=4`) and related comments line

This change should make this file more readable and it will be easier to convert it to other formats (personally I found it out while I was trying to convert this to a JavaScript structure).
2018-04-25 16:25:34 +01:00
Seth Hall
be9f97d8b4 Updates to NTLM script handling.
- This separates NTLM handling away from SMB.
 - It logs more accurately when logins are succeed or fail
   or even if the resulting status of an authentication is
   unknown.
 - Adds some new fields where the server is indicating information
   about itself (server_nb_computer_name, server_dns_computer_name,
   and server_tree_name)
2018-04-06 13:09:24 -04:00
Seth Hall
a80131c06e Updating the defined SMB2 dialects to match Microsofts current docs. 2018-04-05 17:13:10 -04:00
Seth Hall
68d0085d35 Merge branch 'smb_changes' of https://github.com/jwallior/bro into jwallior-smb_changes 2018-04-02 17:06:30 -04:00
Johanna Amann
f39efd0317 Recognize TLS 1.3 negotiation correctly.
The way in which TLS 1.3 is negotiated was changed slightly in later
revisions of the standard. The final version is only sent in an
extension - while the version field in the server hello still shows TLS
1.2.

This patch makes ssl.log show the correct version again.
2018-03-27 14:58:06 -07:00
Daniel Thayer
f3e42874b8 Improve config framework documentation comments
Fixed typos and formatting.
2018-03-15 14:16:00 -05:00
Seth Hall
e76b56ce53 Rework to the DHCP analyzer.
Highlights:
 - Reduced all DHCP events into a single dhcp_message event. (removed legacy events since they weren't widely used anyway)
 - Support many more DHCP options.
 - DHCP log is completely reworked and now represents DHCP sessions
   based on the transaction ID (and works on clusters).
 - Removed the known-devices-and-hostnames script since it's generally
   less relevant now with the updated log.
2018-03-01 08:36:32 -08:00
Johanna Amann
8ea7de9380 Update Mozilla CA list to state of NSS 3.35. 2018-02-16 10:53:51 -08:00
Johanna Amann
981c889352 Update list of Certificate Transparency logs. 2018-02-16 10:44:29 -08:00
Robin Sommer
fff4db5145 Merge remote-tracking branch 'origin/topic/johanna/config'
Closes BIT-1900.

* origin/topic/johanna/config:
  Use port_mgr->Get() in the input framework config changes.
  Allow the empty field separator to be empty; use in config framework.
  Fix small bug in config reader.
  Fix segmentation fault when parsing sets containing invalid elements.
  Add config framework.
2018-02-07 14:10:48 -08:00
Robin Sommer
9f4cef23ad Merge remote-tracking branch 'origin/topic/jsiwek/sumstats-non-cluster-fixes'
Closes BIT-1898.

* origin/topic/jsiwek/sumstats-non-cluster-fixes:
  Fix problems with SumStats non-cluster.bro script
2018-02-05 15:14:40 -08:00
Jon Siwek
4cd9f0edf9 Merge branch 'smb-transaction-messages' of https://github.com/jbencteux/bro
* 'smb-transaction-messages' of https://github.com/jbencteux/bro:
  add test for smb1_com_transaction_response event changes
  add test for smb1_com_transaction2_secondary_request event changes
  add test for smb1_com_transaction2_request event changes
  add test for smb1_com_transaction_secondary_request event changes
  add test for smb1_com_transaction_request event changes
  fix setup field handling in smb1_com_transaction_request messages
  fix smb1_com_transaction* messages
  add smb1_transaction2_secondary_request event
  add smb1_transaction_secondary_request event
  add parameters and data to smb1_transaction_request/response messages
  add SMB_Parameters.Words to smb1_transaction2_request event
2018-02-05 16:33:25 -06:00
Jon Siwek
9603deeb64 Merge branch 'nfs-updates' of https://github.com/dtrejod/bro
* 'nfs-updates' of https://github.com/dtrejod/bro:
  Format print nfs units tests to improve output readability. Add unit tests for new NFS events -- nfs_proc_symlink, nfs_proc_link, nfs_proc_sattr.
  Bug fix: nfs3_writeargs didn't properly return filehandle.
  Add nfs_proc_symlink, nfs_proc_link, nfs_proc_sattr.
2018-02-05 14:15:18 -06:00
Jon Siwek
35fa1261f3 Merge branch 'mount-protocol' of https://github.com/dtrejod/bro
* 'mount-protocol' of https://github.com/dtrejod/bro:
  Add unit tests for new MOUNT events -- mount_proc_mnt, mount_proc_umnt, mount_proc_umnt_all, mount_proc_not_implemented.
  Add mount_proc_null, mount_proc_mnt, mount_proc_umnt, mount_proc_umnt_all, mount_proc_not_implemented, mount_reply_status.
2018-02-05 13:32:36 -06:00
Seth Hall
c2f35920fd First step of DHCP analyzer rearchitecture.
Old event prototypes have changed and the events are broken right
now and may be removed in favor of the new generic "dhcp_message"
event.

DHCP option parsing is abstracted from the main code base of the
protocol parser and are all now located in their own file.

Documentation, tests, and final code cleanup are still pending.
2018-02-02 10:14:15 -05:00
Seth Hall
ba49ab8201 Merge branch 'feature/new_dhcp_data' of https://github.com/Mr-Click/bro into topic/seth/merge-121-dhcp-extensions 2018-02-01 10:06:35 -05:00
Johanna Amann
ac9fd000e0 Merge remote-tracking branch 'origin/master' into topic/johanna/config 2018-01-29 14:43:20 -08:00
Johanna Amann
196994a48d Allow the empty field separator to be empty; use in config framework.
This small change allows the empty field separator to be empty. This
means that we can represent an empty list by a empty input string, which
was not possible before.

Before, an empty empty field separator meant that there is no empty
field - to get back to this behavior one now has to set the empty field
separator to a string that is guaranteed to not be part of the input
data. Note that we did not use "empty" empty field separators anywhere
and I am not aware of this being used by anyone - the new behavior seems
like it is much more useful in practice.

This also changes the config framework to interpret empty lists as...
empty, instead of interpreting them as lists that have one zero-length
element; this seems like the saner default.
2018-01-29 14:22:59 -08:00
Jon Siwek
3495b2fa9d Fix problems with SumStats non-cluster.bro script
* Add proper namespace scoping to a 'SumStats::process_epoch_result'
  scheduled event.

* Fix iterator invalidation within 'SumStats::process_epoch_result'

* Give 'SumStats::process_epoch_result' a copy of the result table so
  that the SumStats framework can clear the original and move on to the
  next epoch immediately.

* The previous baseline of the basic sumstats unit test did look wrong
  to me and probably was actually indicative of the iterator invalidation
  problem.

Thanks to Jim Mellander for reporting the issues.
2018-01-18 11:14:39 -06:00