Commit graph

8717 commits

Author SHA1 Message Date
Jon Siwek
2d8acab664 Merge branch 'smb2-fix' of https://github.com/mauropalumbo75/zeek
* 'smb2-fix' of https://github.com/mauropalumbo75/zeek:
  added test and pcap files for smb_files.log fix
  fixing some missing log lines in smb_files.log
2019-03-20 18:01:35 -07:00
Jon Siwek
a262ed8d9c Merge branch 'master' of https://github.com/rdenniston/zeek
* 'master' of https://github.com/rdenniston/zeek:
  Add linux netfilter NFLOG capture functionality initial commit

I made modifications:

  - Formatting / code style

  - More error handling and validity checks

  - The Type and Length value of TLVs is technically host order

  - Changed / fixed the Length value padding check: it's generally
    32-bit alignment, not just aligning any TLV less than 8 bytes.
2019-03-19 19:19:02 -07:00
Jon Siwek
eb9a05cd25 GH-307: Build binpac as a shared lib, not static by default 2019-03-18 14:00:54 -07:00
Jon Siwek
f19db92508 Merge branch 'topic/christian/inputframework-paths' of https://github.com/ckreibich/zeek
* 'topic/christian/inputframework-paths' of https://github.com/ckreibich/zeek:
  Source file path control for Input and Intel frameworks
2019-03-18 11:48:22 -07:00
Jon Siwek
01d303b480 Migrate table-based for-loops to key-value iteration 2019-03-15 19:54:44 -07:00
Jon Siwek
41c7b229d3 Merge branch 'master' of https://github.com/ZekeMedley/zeek
* 'master' of https://github.com/ZekeMedley/zeek:
  Add key-value for loop

Fixes GH-154
2019-03-15 19:45:07 -07:00
Christian Kreibich
3f02c0a67c Source file path control for Input and Intel frameworks
This introduces the following redefinable string constants, empty by
default:

- InputAscii::path_prefix
- InputBinary::path_prefix
- Intel::path_prefix

When using ASCII or binary reades in the Input/Intel Framework with an
input stream source that does not have an absolute path, these
constants cause Zeek to prefix the resulting paths accordingly. For
example, in the following the location on disk from which Zeek loads
the input becomes "/path/to/input/whitelist.data":

redef InputAscii::path_prefix = "/path/to/input";

event bro_init()
        {
        Input::add_table([$source="whitelist.data", ...]);
	}

These path prefixes can be absolute or relative. When an input stream
source already uses an absolute path, this path is preserved and the
new variables have no effect (i.e., we do not affect configurations
already using absolute paths).

Since the Intel framework builds upon the Input framework, the first
two paths also affect Intel file locations. If this is undesirable,
the Intel::path_prefix variable allows specifying a separate path:
when its value is absolute, the resulting source seen by the Input
framework is absolute, therefore no further changes to the paths
happen.
2019-03-15 16:43:36 -07:00
Jon Siwek
03ac32adec Merge branch 'topic/dopheide/fix-ssh-geo-data' of https://github.com/dopheide-esnet/bro
* 'topic/dopheide/fix-ssh-geo-data' of https://github.com/dopheide-esnet/bro:
  Fix geo-data to log remote_location data when auth is successful.
2019-03-15 13:03:59 -07:00
Jon Siwek
be7110f6c0 Make Syslog analyzer accept messages that omit Priority
Essentially, it will now process/parse priority values if they are
there, or else just accept whatever remaining data/text is there as the
syslog message.  Reasoning is that there's syslog producers out there
that may have simply forgotten/neglected to send the priority value
and technically won't conform to what the standard says, though we can
infer the intent (some syslog consumers already may do similarly, but
I didn't verify).
2019-03-14 18:47:32 -07:00
Jon Siwek
158313875c Move NEWS file back into main repo from zeek-docs 2019-03-14 16:40:27 -07:00
Jon Siwek
a9771bc0eb Fix signed/unsigned comparison compiler warning 2019-03-14 16:24:27 -07:00
Robin Sommer
72fa3f16ad Merge remote-tracking branch 'origin/topic/jsiwek/vxlan'
* origin/topic/jsiwek/vxlan:
  GH-250: Improve/cleanup VXLAN decapsulation support
  Initial VXLAN support, need -C flag to work when running bro
2019-03-14 17:59:42 +00:00
ZekeMedley
1f7924754e Add key-value for loop 2019-03-14 09:46:16 -07:00
Jon Siwek
389fe2bc2a Merge remote-tracking branch 'origin/topic/jazoff/datastructures-defer-init'
* origin/topic/jazoff/datastructures-defer-init:
  Defer initialization of lists and dicts until an item is added.
2019-03-13 17:34:10 -07:00
Jon Siwek
1239a286e2 Merge branch 'patch-1' of https://github.com/neu5ron/bro
* 'patch-1' of https://github.com/neu5ron/bro:
  Update main.bro
  Update main.bro
2019-03-13 13:49:17 -07:00
Michael Dopheide
0f6f6cdb29 Fix geo-data to log remote_location data when auth is successful. 2019-03-13 14:14:38 -05:00
Robin Sommer
56da70c148 Merge remote-tracking branch 'origin/topic/jsiwek/gh-289'
* origin/topic/jsiwek/gh-289:
  GH-289: Add options to limit entries in http.log file fields
2019-03-13 18:23:06 +00:00
Robin Sommer
aab2d186f4 Merge remote-tracking branch 'origin/topic/jsiwek/gh-282'
Closes #282.

* origin/topic/jsiwek/gh-282:
  GH-282: Remove JSON formatter's range restriction on numbers
2019-03-13 18:14:19 +00:00
Robin Sommer
4cf6cb783a Merge remote-tracking branch 'origin/topic/jsiwek/gh-281'
Closes #281.

* origin/topic/jsiwek/gh-281:
  GH-281: Improve parsing of Google Pixel user agent
2019-03-13 18:09:21 +00:00
Robin Sommer
e2172018ee Merge remote-tracking branch 'origin/topic/jsiwek/gh-286'
Added note about different behaviour to NEWS.

Closes #286.

* origin/topic/jsiwek/gh-286:
  GH-286: Check for record type mismatch in ternary operator
2019-03-13 18:01:36 +00:00
Jon Siwek
09ae539ea8 GH-250: Improve/cleanup VXLAN decapsulation support
* Better parsing/error-checking of VXLAN and encapsulated packet headers

* Add/implement the "vxlan_packet" event

* Add "Tunnel::vxlan_ports" option to tune the set of VXLAN ports to
  analyze/decapsulate

* Add "Tunnel::validate_vxlan_checksums" option to allow for tuning of how
  checksums associated with the outer UDP header of a possible VXLAN
  tunnel are handled

Fixes GH-250
2019-03-12 18:15:34 -07:00
Henrik Lund Kramshoej
f4088be8a6 Initial VXLAN support, need -C flag to work when running bro 2019-03-11 14:15:39 -07:00
Jon Siwek
a36ac12e88 Improve DNS query queuing logic
Related to https://github.com/zeek/zeek/pull/276
2019-03-08 18:38:41 -08:00
Jon Siwek
011b6e10bf Merge remote-tracking branch 'origin/topic/jazoff/dns-perf'
* origin/topic/jazoff/dns-perf:
  update baselines
  Use more efficient method to grab the protocol.
  improve performance of dns policy
2019-03-08 16:23:30 -08:00
Jon Siwek
8e2741544d Updating submodule(s).
[nomail]
2019-03-07 18:49:30 -08:00
Jon Siwek
e2ca695c31 Merge branch 'patch-1' of https://github.com/g0nzu1/zeek
* 'patch-1' of https://github.com/g0nzu1/zeek:
  Minor fixes Line 76 and Line
2019-03-07 13:14:53 -08:00
Jon Siwek
e0a099fdfa Updating submodule(s).
[nomail]
2019-03-07 12:20:18 -08:00
Jon Siwek
628a46d8fd GH-219: revert previous change to |x| operator for interval/time
The result of the |x| operator for interval and time types historically
returned a value of type double.  This was changed as part of
3256ac7c49 to return interval/time, but
this now reverts to returning a double again to avoid introducing a
change that may break user code.

Fixes GH-219
2019-03-06 15:36:31 -08:00
g0nzu1
3fc2325416
Minor fixes Line 76 and Line
Line 76: Replaced  ' for ;
Line 120: Replaced ojbects to objects

I tested it on my cloned repo and everything seems to compile without errors (Ubuntu Bionic Beaver) regarding the DNP3 Protocol
2019-03-02 14:54:47 -05:00
Jon Siwek
92b6656bad GH-289: Add options to limit entries in http.log file fields
The "orig_fuids", "orig_filenames", "orig_mime_types" http.log fields as
well as their "resp" counterparts are now limited to having
"HTTP::max_files_orig" or "HTTP::max_files_resp" entries, which are 15
by default.  The limit can also be ignored case-by-case via the
"HTTP::max_files_policy" hook.

Fixes GH-289
2019-02-27 14:25:15 -06:00
Jon Siwek
b20cd599a0 GH-282: Remove JSON formatter's range restriction on numbers
It now produces numbers as large as is required to match the data it
needs to represent.  It's up to the consumer to decide how to
appropriately handle values that may be outside their supported
range/precision.

Fixes GH-282
2019-02-27 11:22:29 -06:00
Stephen Hosom
1d5eac4ee1 Normalize the intel seen filename for smb. 2019-02-27 09:24:52 -05:00
Stephen Hosom
2d3a21968e load smb-filenames in scripts/policy/frameworks/intel/seen/__load__.bro 2019-02-27 08:56:28 -05:00
Stephen Hosom
8ce6d67acc Add SMB::IN_FILE_NAME to Intel::Where enum
This should reduce the ambiguity of where precisely the indicator was
seen so that it isn't confused with the normal File::IN_NAME hit.
2019-02-27 08:53:52 -05:00
Stephen Hosom
4ae92161e9 Support filenamess for SMB files
Hook file_new to observe filenames in SMB traffic and fire into Intel::seen
2019-02-27 08:47:53 -05:00
Nate Guagenti
3d4dd2ea67
Update main.bro
added back user agent
2019-02-26 16:56:10 -05:00
Jon Siwek
8b9d525097 Update external test suite commit hash 2019-02-26 15:09:32 -06:00
Jon Siwek
53dc1176b7 GH-281: Improve parsing of Google Pixel user agent
Fixes GH-281
2019-02-25 14:07:53 -06:00
Jon Siwek
74c225c7cb GH-286: Check for record type mismatch in ternary operator
Fixes GH-286
2019-02-25 12:55:03 -06:00
Jon Siwek
a342090f18 Merge remote-tracking branch 'origin/topic/johanna/input-whitespace'
* origin/topic/johanna/input-whitespace:
  Be more liberal with whitespaces for ip-addresses, subnets, etc.
2019-02-22 14:58:43 -06:00
Jon Siwek
9345ff346a Add missing libkrb5 include dir to CMake config 2019-02-22 14:35:54 -06:00
mauro
a346b01a85 clean up, test and pcap for transform_header added 2019-02-21 12:01:02 +01:00
mauro
f1cdae2829 added smb2-com-transform-header for smb3.x 2019-02-21 09:53:32 +01:00
Johanna Amann
cb47b37215 Be more liberal with whitespaces for ip-addresses, subnets, etc.
This change ignores leading/trailing whitespaces for a couple of
data-types (bool, port, subnet, addr) and just parses them as if the
whitespace was not present.
2019-02-19 15:06:06 -08:00
Justin Azoff
52dbaef6da Defer initialization of lists and dicts until an item is added.
Instead of pre-allocating every list with space for 10 items, don't
initialize it at all until the first Insert.

Instead of pre-allocating every dictionary with 17 buckets,
don't initialize it at all until the first Insert.
2019-02-18 12:50:22 -05:00
Justin Azoff
ea167846fa update baselines 2019-02-17 12:48:24 -05:00
Justin Azoff
a5ad864e13 Use more efficient method to grab the protocol.
get_conn_transport_proto needs to use sessions->FindConnection and do a
hash lookup to find the connection while get_port_transport_proto just
looks at the port directly.
2019-02-17 12:39:01 -05:00
Justin Azoff
a97cf10d03 improve performance of dns policy
defer allocating various tables until at least 2 pending queries are
seen.
2019-02-17 12:39:01 -05:00
Jon Siwek
aff3f4b3fd Skip autogenerated doc coverage test for Travis pull requests 2019-02-15 17:53:53 -06:00
Jon Siwek
e6e328efb4 Merge branch 'master' of https://github.com/ZekeMedley/zeek
* 'master' of https://github.com/ZekeMedley/zeek:
  lstrip test output cleanup
  implemented rstrip
  add rstrip tests
  cleanup of lstrip function
  added implementation of lstrip
  added tests for lstrip function
2019-02-15 17:51:29 -06:00