Commit graph

498 commits

Author SHA1 Message Date
Daniel Thayer
021115a6fa Merge remote-tracking branch 'origin/master' into topic/dnthayer/doc-fixes-for-2.6 2018-08-01 12:19:17 -05:00
Jon Siwek
a71d84c968 Merge branch 'topic/feature/upstream/geoip' of https://github.com/corelight/bro
* 'topic/feature/upstream/geoip' of https://github.com/corelight/bro:
  Replace GeoIP Legacy DB support with MaxMind DB support
2018-07-31 16:50:15 -05:00
Daniel Thayer
8928189878 Update NEWS for config framework clusterization changes 2018-07-31 14:53:02 -05:00
Johanna Amann
a251b32d85 Merge remote-tracking branch 'origin/topic/jsiwek/openssl-1.1'
* origin/topic/jsiwek/openssl-1.1:
  Update install instructions for OpenSSL 1.1 compat
  Remove requestorName parameter of ocsp_request event
  Adjust x509 unit tests to work around OpenSSL 1.0 vs. 1.1 differences
  Fixes for OpenSSL 1.1 support
2018-07-31 13:13:06 +02:00
Jonathan Perkins
1ae7d3b349 Replace GeoIP Legacy DB support with MaxMind DB support
This updates the "lookup_location" and "lookup_asn" BIFs to use
libmaxminddb.  The motivation for this is that MaxMind is discontinuing
GeoLite Legacy databases: no updates after April 1, 2018, no downloads
after January 2, 2019.  It's also noted that all GeoIP Legacy databases
may be discontinued as they are superseded by GeoIP2.
2018-07-30 16:04:58 -05:00
Jon Siwek
35827eeb31 Add rate-limiting sampling mechanism for weird events
The generation of weird events, by default, are now rate-limited
according to these tunable options:

  - Weird::sampling_whitelist
  - Weird::sampling_threshold
  - Weird::sampling_rate
  - Weird::sampling_duration

The new get_reporter_stats() BIF also allows one to query the
total number of weirds generated (pre-sampling) which the new
policy/misc/weird-stats.bro script uses periodically to populate
a weird_stats.log.

There's also new reporter BIFs to allow generating weirds from the
script-layer such that they go through the same, internal
rate-limiting/sampling mechanisms:

  - Reporter::conn_weird
  - Reporter::flow_weird
  - Reporter::net_weird

Some of the code was adapted from previous work by Johanna Amann.
2018-07-26 19:57:36 -05:00
Vern Paxson
016a164bb6 documentation of v += e 2018-07-26 12:29:50 -07:00
Robin Sommer
8ac17d99a1 Merge remote-tracking branch 'origin/topic/jsiwek/bit-1950'
BIT-1950 #merged

* origin/topic/jsiwek/bit-1950:
  BIT-1950: support PPPoE over QinQ
2018-07-24 15:05:48 +00:00
Robin Sommer
d62079b59a Updating submodule(s).
[nomail]
2018-07-24 02:43:06 +00:00
Vern Paxson
86cd484759 documentation, test suite update 2018-07-20 08:57:37 -07:00
Vern Paxson
f4728bd603 only generate history threshold events for > 1 instance
mention those events in NEWS
2018-07-17 10:25:45 -07:00
Jon Siwek
463e540c9b Merge remote-tracking branch 'origin/topic/vern/case-insensitive-patterns'
* origin/topic/vern/case-insensitive-patterns:
  use PCRE syntax instead of the beautiful new (?i ...) syntax
  nitlet in NEWS entry
  test suite update for case-insensitive patterns
  document use of double quotes to escape case-insensitivity
  bug fix for recent memory leak patch
  documentation updates for case-insensitive patterns
  d'oh there's isalpha.  I looked earlier for isletter :-P
  fix for handling [:(lower|upper):] in case-insensitive patterns
  implemented /re/i for case-insensitive patterns
2018-07-16 16:04:38 -05:00
Jon Siwek
a45039de00 Merge remote-tracking branch 'origin/topic/vern/runtime-pattern-bifs'
* origin/topic/vern/runtime-pattern-bifs:
  de-restrict pattern-oriented BiFs to no longer require only running at init
2018-07-16 11:03:27 -05:00
Vern Paxson
73349362a3 'W' for zero window implemented; logarithmic 'T'/'C'/'W' history repetitions 2018-07-09 13:05:10 -07:00
Vern Paxson
b9a5d9ccbe de-restrict pattern-oriented BiFs to no longer require only running at init 2018-07-05 10:13:20 -07:00
Jon Siwek
bb55f82809 Remove requestorName parameter of ocsp_request event
This field isn't publicly available via the OpenSSL 1.1 API, not used
in the base scripts, and has no example in the test suit, so removing
it is simpler than trying to support manually parsing it out of the
raw data.
2018-06-29 16:15:34 -05:00
Vern Paxson
85c4b0d285 use PCRE syntax instead of the beautiful new (?i ...) syntax 2018-06-29 13:01:05 -07:00
Vern Paxson
726424f371 nitlet in NEWS entry 2018-06-29 12:00:37 -07:00
Vern Paxson
a02d9e7f4a document use of double quotes to escape case-insensitivity 2018-06-29 11:35:22 -07:00
Vern Paxson
cfe45e0af0 documentation updates for case-insensitive patterns 2018-06-29 11:30:00 -07:00
Daniel Thayer
8849e214ca Fix some typos and formatting in NEWS 2018-06-27 14:33:07 -05:00
Jon Siwek
e33a3a9c02 Fix typo in NEWS 2018-06-26 18:10:07 -05:00
Vern Paxson
80b3b82b54 implemented /re/i for case-insensitive patterns 2018-06-26 15:59:41 -07:00
Vern Paxson
f340707e2c documentation for &/| for patterns 2018-06-22 15:23:06 -07:00
Vern Paxson
9e2c70b90b deprecate merge_patterns() 2018-06-22 14:17:15 -07:00
Vern Paxson
cff68b4371 deprecate mixing scalars and vectors 2018-06-22 10:03:13 -07:00
Vern Paxson
b864772e8a fixed typos in NEWS 2018-06-21 15:15:05 -07:00
Jon Siwek
e5275b0c92 Merge branch 'topic/vern/bit-ops' of https://github.com/bro/bro
* 'topic/vern/bit-ops' of https://github.com/bro/bro:
  documentation clarification for "p1 | p2"
  documentation for bitwise operators
  document the '|' operator for patterns
  test suite for bitwise operators brief NEWS blurb allow for "counter" operands (does anyone still use these?) for one (but not both) of the bitwise operands
  bitwise operations for "count" types implemented
  Starting branch for supporting bit operations on count's.
2018-06-21 16:05:07 -05:00
Johanna Amann
3a9575bf8a Make kerberos initialization static.
There does not seem to be any reason to initialize the members more than
once globally.
2018-06-01 13:49:08 -07:00
Jon Siwek
84c1d9c808 Fix NEWS file formatting 2018-05-29 10:17:04 -05:00
Robin Sommer
ad1978f698 Updating NEWS and CHANGES. 2018-05-21 20:31:29 +00:00
Jon Siwek
81133f3116 Merge remote-tracking branch 'origin/topic/seth/dhcp-update'
* origin/topic/seth/dhcp-update:
  Rework to the DHCP analyzer.
  First step of DHCP analyzer rearchitecture.
  Add .btest scripts for dhck_ack and dhcp_discover messages verifying that new options are correctly reported in dhcp.log records.
  Extend DHCP protocol analyzer with new options.

BIT-1924 #merged

Additional changes:

* Removed known-hosts.bro as the only thing populating its table was
  the already-removed known-hosts-and-devices.bro.  So a
  known_devices.log will no longer be generated.

* In dhcp-options.pac, the process_relay_agent_inf_option had a memleak
  and also process_auto_proxy_config_option looked like it accessed one
  byte past the end of the available bytestring, so fixed those.
2018-05-01 18:06:41 -05:00
Jon Siwek
4e9a8e02e0 BIT-1430: Improve cross compilation support
* Add --toolchain= configure option

* Add --with-bifcl= configure option

* Change --with-binpac= configure option to mean "path to binpac
  executable"

* Add an example of how to use the above options for cross compiling
  Bro to the docs
2018-04-27 11:04:31 -05:00
Vern Paxson
58ffd61dcc test suite for bitwise operators
brief NEWS blurb
allow for "counter" operands (does anyone still use these?) for one (but not both) of the bitwise operands
2018-04-26 13:25:04 -07:00
Jon Siwek
10b1857a9c Update &check'd fields in various protocol parsers
* The altered Modbus checks seemed overly strict -- the pcap used
  for the unit test at least had quantities/byte_count fields of
  zero, to which the server responds with an error (expected).

* Most of the altered DNP3 checks seemed overly strict and caused
  the unit tests to fail.  The one that was just wrong was the
  'start' field in header blocks.

* Removed the "start" parameter of the dnp3_header_block event
  since it's always the same value.

* The SMB check failed to compile and I don't know what it intended
  to do, so removed.
2018-04-18 18:16:34 -05:00
Daniel Thayer
c759583d11 Fix minor typos and doc build warnings in NEWS 2018-03-15 14:13:54 -05:00
Robin Sommer
fff4db5145 Merge remote-tracking branch 'origin/topic/johanna/config'
Closes BIT-1900.

* origin/topic/johanna/config:
  Use port_mgr->Get() in the input framework config changes.
  Allow the empty field separator to be empty; use in config framework.
  Fix small bug in config reader.
  Fix segmentation fault when parsing sets containing invalid elements.
  Add config framework.
2018-02-07 14:10:48 -08:00
Robin Sommer
ae51f72de0 Updating NEWS. 2018-02-06 08:54:54 -08:00
Jon Siwek
4cd9f0edf9 Merge branch 'smb-transaction-messages' of https://github.com/jbencteux/bro
* 'smb-transaction-messages' of https://github.com/jbencteux/bro:
  add test for smb1_com_transaction_response event changes
  add test for smb1_com_transaction2_secondary_request event changes
  add test for smb1_com_transaction2_request event changes
  add test for smb1_com_transaction_secondary_request event changes
  add test for smb1_com_transaction_request event changes
  fix setup field handling in smb1_com_transaction_request messages
  fix smb1_com_transaction* messages
  add smb1_transaction2_secondary_request event
  add smb1_transaction_secondary_request event
  add parameters and data to smb1_transaction_request/response messages
  add SMB_Parameters.Words to smb1_transaction2_request event
2018-02-05 16:33:25 -06:00
Jon Siwek
9603deeb64 Merge branch 'nfs-updates' of https://github.com/dtrejod/bro
* 'nfs-updates' of https://github.com/dtrejod/bro:
  Format print nfs units tests to improve output readability. Add unit tests for new NFS events -- nfs_proc_symlink, nfs_proc_link, nfs_proc_sattr.
  Bug fix: nfs3_writeargs didn't properly return filehandle.
  Add nfs_proc_symlink, nfs_proc_link, nfs_proc_sattr.
2018-02-05 14:15:18 -06:00
Jon Siwek
35fa1261f3 Merge branch 'mount-protocol' of https://github.com/dtrejod/bro
* 'mount-protocol' of https://github.com/dtrejod/bro:
  Add unit tests for new MOUNT events -- mount_proc_mnt, mount_proc_umnt, mount_proc_umnt_all, mount_proc_not_implemented.
  Add mount_proc_null, mount_proc_mnt, mount_proc_umnt, mount_proc_umnt_all, mount_proc_not_implemented, mount_reply_status.
2018-02-05 13:32:36 -06:00
Jon Siwek
5ed5587b5c Merge remote-tracking branch 'origin/topic/feature/logging-filter-list'
* origin/topic/feature/logging-filter-list:
  Logging: implement get_filter_names and small fixes.

Removed some superfluous existence checks before deleting table indices.

BIT-1890 #merged
2018-01-17 16:17:43 -06:00
Jon Siwek
fc6e3b4e36 Merge remote-tracking branch 'origin/topic/johanna/recognize-websockets'
* origin/topic/johanna/recognize-websockets:
  HTTP: Recognize and skip upgrade/websocket connections.

BIT-1888 #Merged
2018-01-16 15:45:29 -06:00
Jon Siwek
09438d2d06 Merge remote-tracking branch 'origin/topic/johanna/conn-tunnels-empty'
* origin/topic/johanna/conn-tunnels-empty:
  Make tunnel_parents in conn.log optional.

BIT-1889 #merged
2018-01-16 14:56:15 -06:00
Johanna Amann
cc738544eb Merge remote-tracking branch 'remotes/origin/topic/jsiwek/prealloc-ports'
* remotes/origin/topic/jsiwek/prealloc-ports:
  Clean up PortManager class, mark PortVal ctors deprecated.
  Add BRO_DEPRECATED macro.
  Preallocate all possible PortVals.

BIT-1881 #merged
2018-01-12 10:06:30 -08:00
Seth Hall
f236dcdaa3 Updated news. 2017-12-13 15:09:52 -05:00
Jon Siwek
9d2c41a4ff Merge remote-tracking branch 'origin/topic/johanna/socks-password'
* origin/topic/johanna/socks-password:
  Do not log SOCKS passwords by default.

BIT-1791 #merged
2017-12-02 09:53:56 -06:00
Johanna Amann
8ba5c03538 Do not log SOCKS passwords by default.
This introduces a new option, SOCKS::default_capture_password which can
be used to specify if Socks passwords are logged by default

Like fot FTP/HTTP, this option is set to false by default.

Addresses BIT-1791
2017-12-01 14:36:57 -08:00
Johanna Amann
577357d509 Small fixes and changes for the recent ssl-keyexchange-event merge.
This commit fixes a few small issues.

* server key exchange parameters are only parsed when a named curve is
  given.

* I removed the ssl-verbose.bro and moved the functionality into the
  testcase.

  The information that we get with these events is likely irrelevant to
  the majority of Bro users; I do not think that we have to ship a
  script that uses them by default. A script like this would be
  something to publish via the Bro package manager instead; this is the
  approach that we have taken with a number of the recent SSL addition.

* I marked the ssl_server_curve event as deprecated. More information is
  contained in the new ssl_ecdh_server_params event.

  This is an events that is probably seldomly (or never) directly used
  by anyone; I plan to completely remove it right after the 2.6 release.
2017-11-28 12:33:27 -08:00
Robin Sommer
6ba710fbf7 Updating CHANGES, NEWS, and VERSION. 2017-08-04 15:09:51 -07:00