Commit graph

789 commits

Author SHA1 Message Date
Seth Hall
d70f895be3 Merge branch 'J-Gras-topic/jgras/bit-1679'
* J-Gras-topic/jgras/bit-1679:
  Handle removing non-existent intel items.
  Separated file and default info added to matches.
2016-10-02 14:44:43 -04:00
Seth Hall
20fc6398be Merge branch 'topic/jgras/bit-1679' of https://github.com/J-Gras/bro into J-Gras-topic/jgras/bit-1679 2016-10-02 14:21:19 -04:00
Johanna Amann
f66ef7b1e6 Slight naming changes for Bro version information.
Main API now looks like this:

@if ( Version::number >= 20500 )
or
@if ( Version::at_least("2.5") )
2016-09-29 13:36:47 -07:00
Johanna Amann
35465aaf30 Add convenient way to access version information to Bro.
With the introduction of the package manager, it will get more common
that applications are able to get information about the currently
running version of Bro. With this commit, scripts can easily compare
which version of Bro they are running.

Commonly, this probably will either look like this (both lines check if
the current Bro version is greater or equal to 2.5)

@if ( Version::num >= 20500 )
or
@if ( Version::greater_equal("2.5") )

Version::info contains detailed information about the running version of
Bro, including beta flags, etc.
2016-09-29 12:45:48 -07:00
Johanna Amann
038dfa6273 Actually check if the number of fields in a write are equal to the
number of fields required.

Addresses BIT-1683

I do not think this quite fixes the underlying issue of BIT-1683 - it
should not be possible to get to this state in normal operations.

Also fixes a small memory leak for disabled writers.
2016-09-22 16:43:37 -07:00
Jan Grashoefer
8c024ca094 Handle removing non-existent intel items.
The intel framework raises a reporter info on removing non-existent
intel items. An according test case has been added.

Fixes #1679.
2016-09-21 00:37:38 +02:00
Seth Hall
8015e35747 Fix a crash when a user disables DCE_RPC and enabled SMB.
I wasn't accounting for analyzers being disabled and not actually
instantiating when requested.  This includes a test which
verifies there is no crash or problem when a user disables DCE_RPC.
2016-08-28 21:28:57 -04:00
Johanna Amann
1889f409e9 Change timings of intel expire-item test.
Seems to be stable on slow systems with this.
2016-08-12 13:10:04 -07:00
Johanna Amann
33c85895b8 Make netcontrol cluster test stable.
It now consistently works for me.
2016-08-12 11:39:34 -07:00
Johanna Amann
0e44b91cd9 Merge remote-tracking branch 'origin/topic/seth/log-framework-ext'
* origin/topic/seth/log-framework-ext:
  Log extensions: series of small fixes and new tests.
  Change the function for log extension to take a path only and update tests.
  Final changes to log framework ext code.
  Add logging framework metadata mechanism.
  Add unrolling separator & field name map to logging framework.
2016-08-11 07:47:13 -07:00
Johanna Amann
5ec2108bc7 Make invalidtext.bro test stable. 2016-08-10 18:28:38 -07:00
Johanna Amann
6e769db23b Log extensions: series of small fixes and new tests.
The extensions now work with optional types, as well with complex types
(like subrecords). Not returning a record in the ext_func no longer
crashes bro.

The default_ext_func was switched to return void in
cases where no extension revord is defined (was bool).

I also got rid of the offsets in the indices - with the rest of the
implementation, that was not really necessary and made the code more
complex.
2016-08-10 14:14:03 -07:00
Seth Hall
79e49e1ab5 Change the function for log extension to take a path only and update tests.
We were having stack overflow issues creep in with the self-referential
field for the log extension function.
2016-08-10 13:53:27 -04:00
Seth Hall
5f6565d62c Final changes to log framework ext code.
The "metadata" functionality has been renamed to "ext" to
represent that the logs are being extended.  The function that
returns the record which is used to extend the log now receives
a log filter as it's single argument.

The field name "unrolling" is now renamed to "scope" so the variables
names now look like this: "Log::default_scope_sep"
2016-08-10 12:43:32 -04:00
Seth Hall
a60ce35103 Merge remote-tracking branch 'origin/master' into topic/seth/log-framework-ext 2016-08-10 10:28:04 -04:00
Seth Hall
520ac8d92c Merge remote-tracking branch 'origin/master' into topic/seth/smb
# Conflicts:
#	scripts/base/protocols/dce-rpc/main.bro
#	scripts/base/protocols/ntlm/main.bro
#	scripts/policy/protocols/smb/smb1-main.bro
#	src/analyzer/protocol/smb/smb-common.pac
#	src/analyzer/protocol/smb/smb-strings.pac
#	src/analyzer/protocol/smb/smb1-com-locking-andx.pac
#	src/analyzer/protocol/smb/smb1-com-logoff-andx.pac
#	src/analyzer/protocol/smb/smb1-com-nt-create-andx.pac
#	src/analyzer/protocol/smb/smb1-com-open-andx.pac
#	src/analyzer/protocol/smb/smb1-com-read-andx.pac
#	src/analyzer/protocol/smb/smb1-com-session-setup-andx.pac
#	src/analyzer/protocol/smb/smb1-com-transaction-secondary.pac
#	src/analyzer/protocol/smb/smb1-com-transaction.pac
#	src/analyzer/protocol/smb/smb1-com-tree-connect-andx.pac
#	src/analyzer/protocol/smb/smb1-com-write-andx.pac
#	src/analyzer/protocol/smb/smb1-protocol.pac
2016-08-08 15:46:49 -04:00
Robin Sommer
84b372a772 Merge remote-tracking branch 'origin/topic/johanna/rule-reasons'
BIT-1655 #merged

* origin/topic/johanna/rule-reasons:
  NetControl: allow reasons in remove_rule calls.
2016-08-05 18:48:19 -07:00
Robin Sommer
2a854acd2b Revert "Merge branch 'topic/seth/smb'"
This reverts commit 9d9c7bafd3, reversing
changes made to eeb8c0cbb0.

The SMB analyzer wasn't ready yet.
2016-08-05 15:04:49 -07:00
Johanna Amann
4f1a2c7b62 NetControl: allow reasons in remove_rule calls.
This adds the capability for the user to attach a reason when removing
or destroying a rule. The message will both be logged in netcontrol.log
and forwarded to the responsible plugins.

Addresses BIT-1655
2016-08-05 10:49:09 -07:00
Seth Hall
f0bce636cf Merge remote-tracking branch 'origin/topic/dnthayer/ticket1627'
* origin/topic/dnthayer/ticket1627:
  Add a test for starting a cluster with a logger node
  Update broctl submodule
  Update broctl submodule to branch topic/dnthayer/ticket1627
  Change how logger node is detected in cluster framework
  Update test baselines for the new logger node type
  Update docs for the new logger node type
  Add a new node type for logging
2016-08-05 12:59:05 -04:00
Seth Hall
7b3ec047d0 Merge remote-tracking branch 'origin/master' into topic/seth/smb 2016-08-05 11:46:13 -04:00
Seth Hall
eeb8c0cbb0 Merge branch 'topic/seth/intel-update-merge'
Closes github's #77 and closes BIT-1606

* topic/seth/intel-update-merge:
  Tiny scoping updates and test baseline updates for Intel framework.
  Minor documentation cleanups.
  Fixed insertion of nested subnets.
  Refactored FAF integration of intel framework.
  Added expiration for intelligence items.
  Improved intel notices.
  Added hook to allow extending the intel log.
  Added remove function to intel-framework.
  Added support for subnets to intel-framework.
  Refactoring of meta data handling for intel.
  Added testcase for intel updates.
2016-08-05 09:36:52 -04:00
Johanna Amann
bac1bd5bdf Merge remote-tracking branch 'origin/topic/robin/bit-1641'
* origin/topic/robin/bit-1641:
  Fixing duplicate SSH authentication failure events.

I changed the test slightly; the output of uniq is not stable between
operating systems (on OS-X, it emits a space, on Linux it apparently
emits a tab). I removed the call to uniq - sort by itself is enough to
create a difference if there are duplicate entries.

Addresses BIT-1641
2016-08-02 15:28:31 -07:00
Seth Hall
d6a7322a75 Merge branch 'topic/jgras/intel-update' of https://github.com/J-Gras/bro into topic/seth/intel-update-merge
# Conflicts:
#	testing/btest/Baseline/coverage.default-load-baseline/canonified_loaded_scripts.log
#	testing/btest/Baseline/scripts.policy.frameworks.intel.seen.certs/intel-all.log
2016-08-02 15:50:43 -04:00
Robin Sommer
176d9f23be Fixing duplicate SSH authentication failure events.
We now do not raise more than one failure event per connection.

Addresses BIT-1641.
2016-08-01 12:42:03 -07:00
Robin Sommer
3513cfc219 Merge remote-tracking branch 'origin/topic/johanna/bit-1644'
BIT-1644 #merged

* origin/topic/johanna/bit-1644:
  Correct endianness of IP addresses in SNMP.
2016-08-01 08:14:54 -07:00
Robin Sommer
dd06f8b290 Merge remote-tracking branch 'origin/topic/johanna/bit-1651'
BIT-1651 #merged

* origin/topic/johanna/bit-1651:
  Fix behavior of connection_pending event
2016-08-01 08:13:57 -07:00
Seth Hall
da7ec8064b Merge remote-tracking branch 'origin/master' into topic/seth/smb
# Conflicts:
#	testing/btest/Baseline/coverage.bare-load-baseline/canonified_loaded_scripts.log
#	testing/btest/Baseline/coverage.default-load-baseline/canonified_loaded_scripts.log
#	testing/btest/Baseline/plugins.hooks/output
#	testing/btest/Baseline/scripts.policy.misc.dump-events/all-events.log
#	testing/btest/Baseline/scripts.policy.misc.dump-events/smtp-events.log
2016-07-30 01:58:23 -04:00
Johanna Amann
4ad5d9073a NetControl: add catch and release event when IPs are forgotten.
This adds an event that is raised once Catch & Release ceases the
block management for an IP address because the IP has not been seen in
traffic during the watch interval.

This allows users who use their own logic on the top of catch and
release know when they will have to start re-blocking the IP if it
occurs in traffic again.
2016-07-28 16:28:07 -04:00
Johanna Amann
93db6cd876 Fix behavior of connection_pending event
It is now really only raised when Bro is terminating. Also adds a
test-case that raises the event.
2016-07-26 15:49:51 -07:00
Johanna Amann
7603567782 Correct endianness of IP addresses in SNMP.
Addresses BIT-1644
2016-07-26 15:02:11 -07:00
Robin Sommer
5b73345b73 Merge remote-tracking branch 'origin/topic/johanna/bit-1181'
BIT-1181 #merged

* origin/topic/johanna/bit-1181:
  Input: Further small changes to error handling
  Add error events to input framework.
2016-07-26 14:52:27 -07:00
Johanna Amann
c8e1a39758 Input: Further small changes to error handling
Calling Error() in an input reader now automatically will disable the
reader and return a failure in the Update/Heartbeat calls.

Also adds more tests.

Addresses BIT-1181
2016-07-25 15:35:46 -07:00
Johanna Amann
6b9abe85a7 Add error events to input framework.
This change introduces error events for Table and Event readers. Users
can now specify an event that is called when an info, warning, or error
is emitted by their input reader. This can, e.g., be used to raise
notices in case errors occur when reading an important input stream.

Example:

event error_event(desc: Input::TableDescription, msg: string, level: Reporter::Level)
	{
	...
	}

event bro_init()
	{
	Input::add_table([$source="a", $error_ev=error_event, ...]);
	}

For the moment, this converts all errors in the Asciiformatter into
warnings (to show that they are non-fatal) - the Reader itself also has
to throw an Error to show that a fatal error occurred and processing
will be abort.

It might be nicer to change this and require readers to mark fatal
errors as such when throwing them.

Addresses BIT-1181
2016-07-22 19:45:28 -07:00
Johanna Amann
4968a5c654 Enable SQLite shared cache mode.
This allows all threads accessing the same database to share sqlite
objects. This, for example, fixes the issue with several threads
simultaneously writing to the same database file.

See https://www.sqlite.org/sharedcache.html

Addresses BIT-1325
2016-07-21 12:10:20 -07:00
Daniel Thayer
59f0ea1a0c Add a test for starting a cluster with a logger node
Also modified cluster framework scripts so that any node can raise an
event to be handled on the logger node.
2016-07-15 15:23:49 -05:00
Johanna Amann
11ec4903ee SSL: add support for signature_algorithms extension. 2016-07-11 15:01:59 -07:00
Robin Sommer
7e7002329c Merge remote-tracking branch 'origin/topic/seth/smb' into topic/robin/smb-merge 2016-07-07 12:07:53 -07:00
Seth Hall
d1b1a560c0 Add a DCE-RPC test. 2016-07-07 14:17:25 -04:00
Robin Sommer
781fdfa5ba Merge remote-tracking branch 'origin/topic/seth/smb' into topic/robin/smb-merge 2016-07-07 10:41:31 -07:00
Robin Sommer
721693425f Escape the empty indicator in logs if it occurs literally as a field's
actual content.

Addresses BIT-931.
2016-07-05 16:34:24 -07:00
Seth Hall
514dfc3479 Merge remote-tracking branch 'origin/master' into topic/seth/smb
# Conflicts:
#	testing/btest/Baseline/plugins.hooks/output
#	testing/btest/Baseline/scripts.policy.misc.dump-events/all-events.log
#	testing/btest/Baseline/scripts.policy.misc.dump-events/smtp-events.log
2016-06-29 09:43:31 -04:00
Jan Grashoefer
df5d9adfb4 Fixed insertion of nested subnets.
When inserting, existance of the given subnet is checked using exact
matching instead of longest prefix matching. Before, inserting a subnet
would have updated the subnet item, which is the longest prefix of the
inserted subnet, if present.
2016-06-22 21:14:06 +02:00
Johanna Amann
6361a0d658 Merge remote-tracking branch 'origin/master' into topic/johanna/netcontrol-improvements 2016-06-21 12:42:06 -07:00
Seth Hall
8eb67a0a6c Update tests to match move of smb base scripts into policy/ 2016-06-20 16:22:57 -04:00
Johanna Amann
efab728876 Merge branch 'topic/jgras/input-default' of https://github.com/J-Gras/bro
* 'topic/jgras/input-default' of https://github.com/J-Gras/bro:
  Fixed &default values ignored by input framework.

BIT-1623 #merged
2016-06-15 17:32:50 -07:00
Jan Grashoefer
8d853872a4 Fixed &default values ignored by input framework.
Reading into a record, the input framework will no longer reset a
&default value in case there is no value to read.
2016-06-16 01:02:51 +02:00
Jan Grashoefer
5d340e669c Added expiration for intelligence items.
Expiration of intelligence items can be configured using
Intel::item_expiration. Expiration can be handled using the
Intel::item_expired hook.
2016-06-15 19:29:48 +02:00
Jan Grashoefer
7e0b5b4ba4 Merge branch 'master' into topic/jgras/intel-update 2016-06-15 19:26:26 +02:00
Seth Hall
90399db32d Additional test specifically for the HTTP filename handling. 2016-06-15 01:56:07 -04:00