Commit graph

1785 commits

Author SHA1 Message Date
Johanna Amann
6779325520 NetControl: find_rules_subnet works in cluster mode
This introduces two new events, NetControl::rule_new and
NetControl::rule_destroyed, which are raised when rules are first added
and then deleted from the internal state tracking.
2016-05-20 11:11:44 -07:00
Johanna Amann
52d694f3bd Merge remote-tracking branch 'origin/master' into topic/johanna/netcontrol-improvements 2016-05-19 16:17:07 -07:00
Robin Sommer
e2dee49fc1 Fixing control frameworks net_stats and peer_status commands.
Turns out the code implementing them went missing with the recent
merge of the stats changes.

For peer status, I removed most of the numbers that the code used to
return because (1) we don't have access to that data anymore, and (2)
even in 2.4 the numbers returned already didn't make sense (because
they were global values, not per peer). We could consider just
removing the peer_status command at all.
2016-05-17 16:11:22 -07:00
Jan Grashoefer
cb33028702 Added hook to allow extending the intel log.
The extension mechanism is basically the one that Seth introduced with
his intel extensions. The main difference lies in using a hook instead
of an event. An example policy implements whitelisting.
2016-05-11 23:59:46 +02:00
Jan Grashoefer
859eb5eac7 Merge branch 'master' into topic/jgras/intel-update 2016-05-11 18:59:58 +02:00
Daniel Thayer
d91dd8d9a8 Fix Bro and unit tests when broker is not enabled
When Bro was compiled with broker disabled, then some Bro scripts
were referencing functions and types that were not defined.  Fixed
by adding @ifdefs to several scripts.  Removed one @ifdef because
it was causing several unit tests to fail.

Also fixed the @TEST-REQUIRES check in tests that rely on broker so
that such tests are skipped when broker is disabled.
2016-05-10 06:24:35 -05:00
Robin Sommer
00d94f1bbc Merge remote-tracking branch 'origin/topic/seth/stats-improvement'
(Cleaned up some code a little bit.)

* origin/topic/seth/stats-improvement:
  Fixing tests for stats improvements
  Rename the reporting interval variable for stats.
  Removing more broken functionality due to changed stats apis.
  Removing some references to resource_usage()
  Removing Broker stats, it was broken and incomplete.
  Fixing default stats collection interval to every 5 minutes.
  Add DNS stats to the stats.log
  Small stats script tweaks and beginning broker stats.
  Continued stats cleanup and extension.
  More stats collection extensions.
  More stats improvements
  Slight change to Mach API for collecting memory usage.
  Fixing some small mistakes.
  Updating the cmake submodule for the stats updates.
  Fix memory usage collection on Mac OS X.
  Cleaned up stats collection.

BIT-1581 #merged
2016-05-07 11:51:35 -07:00
Robin Sommer
34c7a3ef68 Merge remote-tracking branch 'origin/topic/dnthayer/doc-improvements'
* origin/topic/dnthayer/doc-improvements:
  Fix some scripting tutorial examples
  Add some missing Bro script documentation
  Fix more "make doc" warnings
  Fix some "make doc" warnings
  Improve documentation of the "for" statement
  Add more documentation to sumstats framework scripts
  Update traffic per core estimate in the cluster doc
  Update documentation for DNS "Z" field
  Improve documentation of Bro script statements

BIT-1520 #merged
2016-05-06 16:59:07 -07:00
Robin Sommer
abebd036bc Merge remote-tracking branch 'origin/topic/johanna/xmpp-starttls'
* origin/topic/johanna/xmpp-starttls:
  XMPP: Add StartTLS event and update tests
  Add xmpp dpd sig and fix a few parsing problems for connections that do not upgrade to TLS.
  Add simple XMPP StartTLS analyzer.

BIT-1579 #merged
2016-05-06 16:56:06 -07:00
Daniel Thayer
2d9127888f Add some missing Bro script documentation
Also fixed a few reST formatting issues.
2016-05-05 16:35:31 -05:00
Daniel Thayer
28125e367e Fix more "make doc" warnings 2016-05-05 15:18:50 -05:00
Daniel Thayer
75e69d8c09 Fix some "make doc" warnings 2016-05-05 12:47:21 -05:00
Daniel Thayer
da6611e3c2 Merge remote-tracking branch 'origin/master' into topic/dnthayer/doc-improvements 2016-05-05 11:33:57 -05:00
Robin Sommer
209c8936d1 Merge remote-tracking branch 'origin/fastpath'
* origin/fastpath:
  Fix a few incorrect type tags in Bro broker source code
  Update docs and tests of the fmt() function
  Revert "Fix RFB analyzer to build on FreeBSD"
  Fix RFB analyzer to build on FreeBSD
2016-05-03 11:36:52 -07:00
Seth Hall
f8f5998328 Fixing tests for stats improvements 2016-05-02 16:43:08 -04:00
Seth Hall
da014e1eca Rename the reporting interval variable for stats. 2016-05-02 16:20:53 -04:00
Seth Hall
d9d579c52c Merge remote-tracking branch 'origin/master' into topic/seth/stats-improvement 2016-05-02 14:34:29 -04:00
Johanna Amann
86836c0bb8 Merge remote-tracking branch 'origin/master' into topic/johanna/netcontrol-improvements 2016-05-02 11:23:36 -07:00
Johanna Amann
6905984ee7 Merge remote-tracking branch 'origin/master' into topic/johanna/xmpp-starttls 2016-04-29 12:56:12 -07:00
Robin Sommer
befad8f825 Merge remote-tracking branch 'origin/topic/dnthayer/ticket1449'
BIT-1449 #merged

* origin/topic/dnthayer/ticket1449:
  Rename broker BIF wrapper functions in a few more places
  Sync the core/leaks/broker/data.bro test with broker/data.bro
  Add missing tests for broker data BIFs
  Code cleanup for some broker tests
  Add script wrapper functions for broker data BIFs
  Add script wrapper functions for broker BIFs
2016-04-28 10:24:39 -07:00
Robin Sommer
abb42a5bd9 Merge remote-tracking branch 'origin/topic/johanna/intel-uid-fuid'
BIT-1572 #merged

* origin/topic/johanna/intel-uid-fuid:
  Intel: Allow to provide uid/fuid instead of conn/f.
2016-04-28 10:20:50 -07:00
Daniel Thayer
4df948f3c8 Add script wrapper functions for broker data BIFs 2016-04-26 22:01:09 -05:00
Daniel Thayer
f44bb4d9b8 Add script wrapper functions for broker BIFs
Also renamed the "print" function to "send_print" and the "event"
function to "send_event" because Bro shows a syntax error when a
Bro script function is named "event" or "print".
2016-04-26 18:10:30 -05:00
Johanna Amann
3669b6aa9c Merge remote-tracking branch 'origin/master' into topic/johanna/imap-starttls 2016-04-26 10:52:16 -07:00
Johanna Amann
41606e18fb Intel: Allow to provide uid/fuid instead of conn/f.
This patch allows users to provide the fuid or the connection id
directly, in case they do not have access to either in the event that
they handle.

An example for this is the handling of certificates in SSL, where the
fa_file record cannot be retained because this would create a cyclic
data structure.

This patch also provides file IDs for hostname matches in certificates,
which was not possible with the previous API.
2016-04-25 16:54:47 -07:00
Johanna Amann
cfa4fb4946 Merge branch 'dns-caa-decode' of https://github.com/marktayl/bro into merge
* 'dns-caa-decode' of https://github.com/marktayl/bro:
  Add DNS "CAA" RR type and event.
2016-04-25 15:36:18 -07:00
Johanna Amann
e7ec537ed5 Merge remote-tracking branch 'origin/topic/dnthayer/broker-namespace'
* origin/topic/dnthayer/broker-namespace:
  Split the broker main.bro into two scripts
  Rename the BrokerStore namespace to Broker
  Rename the BrokerComm namespace to Broker

BIT-1563 #merged
2016-04-22 16:45:09 -07:00
Johanna Amann
176bb6ef34 Merge branch 'topic/jgras/packet-header' of https://github.com/J-Gras/bro
In the merge, I changed IP.cc to use icmp6_hdr for icmpv6 instead of the
icmp* that was used in the patch. While it does not make a difference
for this case, it seems cleaner.

BIT-1570 #merged
2016-04-22 16:07:01 -07:00
Mark Taylor
a14de582a2 Add DNS "CAA" RR type and event. 2016-04-22 15:26:34 -04:00
Robin Sommer
efde4a74b0 Merge remote-tracking branch 'origin/topic/johanna/intel-cert-hash'
BIT-1567 #merged

* origin/topic/johanna/intel-cert-hash:
  Intel: CERT_HASH indicator type was never checked
2016-04-22 08:37:14 -07:00
Jan Grashoefer
c0bf1b3c67 Added get_current_packet_header bif. 2016-04-20 00:00:47 +02:00
Seth Hall
9aa9618473 Additional mime types for file identification and a few fixes.
Some of the existing mime types received extended matchers
to fix problems with UTF-16 BOMs.

New file mime types:
 - .ini files
 - MS Registry policy files
 - MS Registry files
 - MS Registry format files (e.g. DESKTOP.DAT)
 - MS Outlook PST files
 - Apple AFPInfo files

Mime type fixes:
 - MP3 files with ID3 tags.
 - JSON and XML matchers were extended
2016-04-14 10:06:58 -04:00
Seth Hall
2bde8ac052 Merge remote-tracking branch 'origin/topic/seth/file-entropy' into topic/seth/file-entropy
# Conflicts:
#	testing/btest/Baseline/coverage.bare-load-baseline/canonified_loaded_scripts.log
2016-04-12 23:40:07 -04:00
Seth Hall
9d0899325a Merge remote-tracking branch 'martin/topic/fox/rfb'
* martin/topic/fox/rfb:
  Fixed issue in state machine
  Some styling tweaks
  Implement protocol confirmation
  Analyzer and bro script for RFB protocol (VNC)

* <seth> I also applied a bit of clean up to the base
  script to make it match other scripts better and
  updated tests.
2016-04-12 17:00:06 -04:00
Johanna Amann
00e759b44c Intel: CERT_HASH indicator type was never checked
Hence, when people specify data of type CERT_HASH in their intel source
files, it will never trigger an alert.
2016-04-11 15:50:55 +02:00
Martin van Hensbergen
034f725f3f Some styling tweaks
- used transient declarations where appropriate
- fixed brackets
- cleaned up some comments
2016-04-11 11:35:36 +02:00
Martin van Hensbergen
04dd65cbaf Implement protocol confirmation
Do not set the service field in the bro script but
use the protocol confirmation paradigm.

Protocol is considered confirmed if both a
succesful client and server banner have been
parsed.
2016-04-11 11:28:22 +02:00
Martin van Hensbergen
849875e8be Analyzer and bro script for RFB protocol (VNC)
This analyzer parses the Remote Frame Buffer
protocol, usually referred to as the 'VNC protocol'.

It supports several dialects (3.3, 3.7, 3.8) and
also handles the Apple Remote Desktop variant.

It will log such facts as client/server versions,
authentication method used, authentication result,
height, width and name of the shared screen.

It also includes two testcases.

Todo: Apple Remote Desktop seems to have some
bytes prepended to the screen name. This is
not interepreted correctly.
2016-04-11 10:35:00 +02:00
Daniel Thayer
cca9a6616e Split the broker main.bro into two scripts
Separate the former BrokerComm and BrokerStore portions of the script
because these files will be much larger when script wrappers for BIFs are
written.
2016-03-30 20:32:36 -05:00
Daniel Thayer
f46dfac63a Rename the BrokerStore namespace to Broker 2016-03-30 16:39:19 -05:00
Daniel Thayer
9f5c820c7b Rename the BrokerComm namespace to Broker 2016-03-30 14:31:25 -05:00
Jan Grashoefer
2ebac70782 Added remove function to intel-framework. 2016-03-30 20:03:07 +02:00
Johanna Amann
ee132b6da4 NetControl: fix acld whitelist command 2016-03-24 15:21:36 -07:00
Johanna Amann
ee4abb5db2 Merge remote-tracking branch 'origin/master' into topic/johanna/netcontrol-improvements 2016-03-24 15:07:38 -07:00
Johanna Amann
34ad4cf638 NetControl: add rule exists as state besides added and failure.
Rules that are already existing will, by default, not automatically be
timed out by NetControl.
2016-03-24 15:06:07 -07:00
Johanna Amann
8650841bf5 Only load openflow/netcontrol if compiled with broker. 2016-03-24 13:39:34 -07:00
Jan Grashoefer
cafae5351b Added support for subnets to intel-framework.
The intel-framework now supports the new indicator type Intel::SUBNET.
As subnets are matched against seen addresses, the field matched was
introduced to indicate which indicator types caused the hit. A testcase
for subents was added and the old ones have been updated accordingly.
2016-03-22 19:16:51 +01:00
Jan Grashoefer
06faee2cc8 Merge branch 'master' into topic/jgras/intel-update 2016-03-22 18:06:09 +01:00
Seth Hall
89b4d79f93 Merge remote-tracking branch 'origin/master' into topic/seth/file-entropy
# Conflicts:
#	scripts/test-all-policy.bro
#	testing/btest/Baseline/coverage.default-load-baseline/canonified_loaded_scripts.log
2016-03-21 11:39:15 -04:00
Jan Grashoefer
0146e85c41 Refactoring of meta data handling for intel.
To simplify meta data handling inside the intel framework and avoid
duplicate insertion of meta data on update, meta data is stored in a
table indexed by meta data source.
2016-03-19 17:12:06 +01:00