Commit graph

9176 commits

Author SHA1 Message Date
Jan Grashoefer
0f86aaff4b Added test for intel item filtering. 2019-03-24 21:32:45 +01:00
Jan Grashoefer
3e31f3d554 Added hook to filter intelligence items. 2019-03-24 21:31:48 +01:00
Jon Siwek
b2930a6ec3 Updating submodule(s).
[nomail]
2019-03-22 16:35:28 -07:00
Jon Siwek
2b79c7af16 Updating submodule(s).
[nomail]
2019-03-22 13:07:52 -07:00
Jon Siwek
1b76d92e97 Merge branch 'smb3-negotiate-response' of https://github.com/mauropalumbo75/zeek
* 'smb3-negotiate-response' of https://github.com/mauropalumbo75/zeek:
  added test and pcap files for smb 3.1.1 negotiate-response
  smb3.1.1 additions to negotiate-response command

I made several modifications:

  - Code format, style, naming changes

  - For completeness/correctness, I added parsing support for the remaining
    context type structures.

  - Moved the optional padding before the NegotiateContextList field to
    also require the 0x0311 dialect version (some failures in
    pre-existing unit tests pointed this out as an issue)
2019-03-21 14:13:21 -07:00
Jon Siwek
9eb5449ecb Merge branch 'smb3-transform-header' of https://github.com/mauropalumbo75/zeek
* 'smb3-transform-header' of https://github.com/mauropalumbo75/zeek:
  clean up, test and pcap for transform_header added
  added smb2-com-transform-header for smb3.x
2019-03-20 19:30:32 -07:00
Jon Siwek
2d8acab664 Merge branch 'smb2-fix' of https://github.com/mauropalumbo75/zeek
* 'smb2-fix' of https://github.com/mauropalumbo75/zeek:
  added test and pcap files for smb_files.log fix
  fixing some missing log lines in smb_files.log
2019-03-20 18:01:35 -07:00
Jon Siwek
a262ed8d9c Merge branch 'master' of https://github.com/rdenniston/zeek
* 'master' of https://github.com/rdenniston/zeek:
  Add linux netfilter NFLOG capture functionality initial commit

I made modifications:

  - Formatting / code style

  - More error handling and validity checks

  - The Type and Length value of TLVs is technically host order

  - Changed / fixed the Length value padding check: it's generally
    32-bit alignment, not just aligning any TLV less than 8 bytes.
2019-03-19 19:19:02 -07:00
Jon Siwek
eb9a05cd25 GH-307: Build binpac as a shared lib, not static by default 2019-03-18 14:00:54 -07:00
Jon Siwek
f19db92508 Merge branch 'topic/christian/inputframework-paths' of https://github.com/ckreibich/zeek
* 'topic/christian/inputframework-paths' of https://github.com/ckreibich/zeek:
  Source file path control for Input and Intel frameworks
2019-03-18 11:48:22 -07:00
Jon Siwek
01d303b480 Migrate table-based for-loops to key-value iteration 2019-03-15 19:54:44 -07:00
Jon Siwek
41c7b229d3 Merge branch 'master' of https://github.com/ZekeMedley/zeek
* 'master' of https://github.com/ZekeMedley/zeek:
  Add key-value for loop

Fixes GH-154
2019-03-15 19:45:07 -07:00
Christian Kreibich
3f02c0a67c Source file path control for Input and Intel frameworks
This introduces the following redefinable string constants, empty by
default:

- InputAscii::path_prefix
- InputBinary::path_prefix
- Intel::path_prefix

When using ASCII or binary reades in the Input/Intel Framework with an
input stream source that does not have an absolute path, these
constants cause Zeek to prefix the resulting paths accordingly. For
example, in the following the location on disk from which Zeek loads
the input becomes "/path/to/input/whitelist.data":

redef InputAscii::path_prefix = "/path/to/input";

event bro_init()
        {
        Input::add_table([$source="whitelist.data", ...]);
	}

These path prefixes can be absolute or relative. When an input stream
source already uses an absolute path, this path is preserved and the
new variables have no effect (i.e., we do not affect configurations
already using absolute paths).

Since the Intel framework builds upon the Input framework, the first
two paths also affect Intel file locations. If this is undesirable,
the Intel::path_prefix variable allows specifying a separate path:
when its value is absolute, the resulting source seen by the Input
framework is absolute, therefore no further changes to the paths
happen.
2019-03-15 16:43:36 -07:00
Jon Siwek
03ac32adec Merge branch 'topic/dopheide/fix-ssh-geo-data' of https://github.com/dopheide-esnet/bro
* 'topic/dopheide/fix-ssh-geo-data' of https://github.com/dopheide-esnet/bro:
  Fix geo-data to log remote_location data when auth is successful.
2019-03-15 13:03:59 -07:00
Jon Siwek
be7110f6c0 Make Syslog analyzer accept messages that omit Priority
Essentially, it will now process/parse priority values if they are
there, or else just accept whatever remaining data/text is there as the
syslog message.  Reasoning is that there's syslog producers out there
that may have simply forgotten/neglected to send the priority value
and technically won't conform to what the standard says, though we can
infer the intent (some syslog consumers already may do similarly, but
I didn't verify).
2019-03-14 18:47:32 -07:00
Jon Siwek
158313875c Move NEWS file back into main repo from zeek-docs 2019-03-14 16:40:27 -07:00
Jon Siwek
a9771bc0eb Fix signed/unsigned comparison compiler warning 2019-03-14 16:24:27 -07:00
Robin Sommer
72fa3f16ad Merge remote-tracking branch 'origin/topic/jsiwek/vxlan'
* origin/topic/jsiwek/vxlan:
  GH-250: Improve/cleanup VXLAN decapsulation support
  Initial VXLAN support, need -C flag to work when running bro
2019-03-14 17:59:42 +00:00
ZekeMedley
1f7924754e Add key-value for loop 2019-03-14 09:46:16 -07:00
Jon Siwek
389fe2bc2a Merge remote-tracking branch 'origin/topic/jazoff/datastructures-defer-init'
* origin/topic/jazoff/datastructures-defer-init:
  Defer initialization of lists and dicts until an item is added.
2019-03-13 17:34:10 -07:00
Jon Siwek
1239a286e2 Merge branch 'patch-1' of https://github.com/neu5ron/bro
* 'patch-1' of https://github.com/neu5ron/bro:
  Update main.bro
  Update main.bro
2019-03-13 13:49:17 -07:00
Michael Dopheide
0f6f6cdb29 Fix geo-data to log remote_location data when auth is successful. 2019-03-13 14:14:38 -05:00
Robin Sommer
56da70c148 Merge remote-tracking branch 'origin/topic/jsiwek/gh-289'
* origin/topic/jsiwek/gh-289:
  GH-289: Add options to limit entries in http.log file fields
2019-03-13 18:23:06 +00:00
Robin Sommer
aab2d186f4 Merge remote-tracking branch 'origin/topic/jsiwek/gh-282'
Closes #282.

* origin/topic/jsiwek/gh-282:
  GH-282: Remove JSON formatter's range restriction on numbers
2019-03-13 18:14:19 +00:00
Robin Sommer
4cf6cb783a Merge remote-tracking branch 'origin/topic/jsiwek/gh-281'
Closes #281.

* origin/topic/jsiwek/gh-281:
  GH-281: Improve parsing of Google Pixel user agent
2019-03-13 18:09:21 +00:00
Robin Sommer
e2172018ee Merge remote-tracking branch 'origin/topic/jsiwek/gh-286'
Added note about different behaviour to NEWS.

Closes #286.

* origin/topic/jsiwek/gh-286:
  GH-286: Check for record type mismatch in ternary operator
2019-03-13 18:01:36 +00:00
Jon Siwek
09ae539ea8 GH-250: Improve/cleanup VXLAN decapsulation support
* Better parsing/error-checking of VXLAN and encapsulated packet headers

* Add/implement the "vxlan_packet" event

* Add "Tunnel::vxlan_ports" option to tune the set of VXLAN ports to
  analyze/decapsulate

* Add "Tunnel::validate_vxlan_checksums" option to allow for tuning of how
  checksums associated with the outer UDP header of a possible VXLAN
  tunnel are handled

Fixes GH-250
2019-03-12 18:15:34 -07:00
Henrik Lund Kramshoej
f4088be8a6 Initial VXLAN support, need -C flag to work when running bro 2019-03-11 14:15:39 -07:00
Jon Siwek
a36ac12e88 Improve DNS query queuing logic
Related to https://github.com/zeek/zeek/pull/276
2019-03-08 18:38:41 -08:00
Jon Siwek
011b6e10bf Merge remote-tracking branch 'origin/topic/jazoff/dns-perf'
* origin/topic/jazoff/dns-perf:
  update baselines
  Use more efficient method to grab the protocol.
  improve performance of dns policy
2019-03-08 16:23:30 -08:00
Jon Siwek
8e2741544d Updating submodule(s).
[nomail]
2019-03-07 18:49:30 -08:00
Jon Siwek
e2ca695c31 Merge branch 'patch-1' of https://github.com/g0nzu1/zeek
* 'patch-1' of https://github.com/g0nzu1/zeek:
  Minor fixes Line 76 and Line
2019-03-07 13:14:53 -08:00
Jon Siwek
e0a099fdfa Updating submodule(s).
[nomail]
2019-03-07 12:20:18 -08:00
Jon Siwek
628a46d8fd GH-219: revert previous change to |x| operator for interval/time
The result of the |x| operator for interval and time types historically
returned a value of type double.  This was changed as part of
3256ac7c49 to return interval/time, but
this now reverts to returning a double again to avoid introducing a
change that may break user code.

Fixes GH-219
2019-03-06 15:36:31 -08:00
g0nzu1
3fc2325416
Minor fixes Line 76 and Line
Line 76: Replaced  ' for ;
Line 120: Replaced ojbects to objects

I tested it on my cloned repo and everything seems to compile without errors (Ubuntu Bionic Beaver) regarding the DNP3 Protocol
2019-03-02 14:54:47 -05:00
Jon Siwek
92b6656bad GH-289: Add options to limit entries in http.log file fields
The "orig_fuids", "orig_filenames", "orig_mime_types" http.log fields as
well as their "resp" counterparts are now limited to having
"HTTP::max_files_orig" or "HTTP::max_files_resp" entries, which are 15
by default.  The limit can also be ignored case-by-case via the
"HTTP::max_files_policy" hook.

Fixes GH-289
2019-02-27 14:25:15 -06:00
Jon Siwek
b20cd599a0 GH-282: Remove JSON formatter's range restriction on numbers
It now produces numbers as large as is required to match the data it
needs to represent.  It's up to the consumer to decide how to
appropriately handle values that may be outside their supported
range/precision.

Fixes GH-282
2019-02-27 11:22:29 -06:00
Stephen Hosom
1d5eac4ee1 Normalize the intel seen filename for smb. 2019-02-27 09:24:52 -05:00
Stephen Hosom
2d3a21968e load smb-filenames in scripts/policy/frameworks/intel/seen/__load__.bro 2019-02-27 08:56:28 -05:00
Stephen Hosom
8ce6d67acc Add SMB::IN_FILE_NAME to Intel::Where enum
This should reduce the ambiguity of where precisely the indicator was
seen so that it isn't confused with the normal File::IN_NAME hit.
2019-02-27 08:53:52 -05:00
Stephen Hosom
4ae92161e9 Support filenamess for SMB files
Hook file_new to observe filenames in SMB traffic and fire into Intel::seen
2019-02-27 08:47:53 -05:00
Nate Guagenti
3d4dd2ea67
Update main.bro
added back user agent
2019-02-26 16:56:10 -05:00
Jon Siwek
8b9d525097 Update external test suite commit hash 2019-02-26 15:09:32 -06:00
mauro
b6aa0b9d72 added more options to SMB2 set-info command 2019-02-26 15:11:59 +01:00
Jon Siwek
53dc1176b7 GH-281: Improve parsing of Google Pixel user agent
Fixes GH-281
2019-02-25 14:07:53 -06:00
Jon Siwek
74c225c7cb GH-286: Check for record type mismatch in ternary operator
Fixes GH-286
2019-02-25 12:55:03 -06:00
Justin Azoff
8fc83f5fee Remove redundant buffering in contentline
The contentline analyzer has two code paths that buffer data:
 * right at the top of DeliverStream
 * later in DoDeliverOnce

However, contentline can be in plain delivery mode, and if so, the
buffer resize in DeliverStream does not need to be done just because
DeliverStream was passed an 8K data chunk.

This was causing contentline to resize it's buffer to fit chunks of HTTP
response data.  Additionally, the buffer was sized to be 3/2 of the
chunk, so an 8K chunk would result in a 12K allocation.
2019-02-24 11:17:33 -05:00
Justin Azoff
bc1f2a0a63 Avoid buffering all http headers
Only buffer all http headers if the http_all_headers event is in use.
2019-02-23 21:02:51 -05:00
Jon Siwek
a342090f18 Merge remote-tracking branch 'origin/topic/johanna/input-whitespace'
* origin/topic/johanna/input-whitespace:
  Be more liberal with whitespaces for ip-addresses, subnets, etc.
2019-02-22 14:58:43 -06:00
Jon Siwek
9345ff346a Add missing libkrb5 include dir to CMake config 2019-02-22 14:35:54 -06:00