Commit graph

8390 commits

Author SHA1 Message Date
Jon Siwek
23c244d448 Change next version number in NEWS 2019-04-05 17:06:03 -07:00
Jon Siwek
f7ee433a55 Merge remote-tracking branch 'origin/topic/johanna/dtls-errors'
* origin/topic/johanna/dtls-errors:
  Update DTLS error handling
2019-04-04 16:38:04 -07:00
Johanna Amann
7c48aad582 Update DTLS error handling
DTLS now only outputs protocol violations once it saw something that
looked like a DTLS connection (at least a client hello). Before the
danger that it misinterprets something is too high.

It has a configurable number of invalid packets that it can skip over
(because other protocols might be interleaved with the connection) and a
maximum amount of Protocol violations that it outputs because of wrong
packet versions.
2019-04-04 12:27:42 -07:00
Jon Siwek
3f7bbf2784 Update external test commit pointers 2019-04-03 13:26:51 -07:00
Johanna Amann
99c89d55d6 Merge remote-tracking branch 'origin/topic/jsiwek/bounded-weirds'
* origin/topic/jsiwek/bounded-weirds:
  Fix reporter net_weird API usage for unknown_mobility_type
  Remove variable content from weird names
2019-04-03 10:16:39 -07:00
Johanna Amann
210a4361fa Merge remote-tracking branch 'origin/topic/jsiwek/syslog-missing-pri'
* origin/topic/jsiwek/syslog-missing-pri:
  Make Syslog analyzer accept messages that omit Priority
2019-04-03 09:25:52 -07:00
Jon Siwek
faa0f43558 Fix reporter net_weird API usage for unknown_mobility_type
(This code path is pre-processed out by default)
2019-04-02 10:41:32 -07:00
Jon Siwek
995368e68c Remove variable content from weird names
This changes many weird names to move non-static content from the
weird name into the "addl" field to help ensure the total number of
weird names is reasonably bounded.  Note the net_weird and flow_weird
events do not have an "addl" parameter, so information may no longer
be available in those cases -- to make it available again we'd need
to either (1) define new events that contain such a parameter, or
(2) change net_weird/flow_weird event signature (which is a breaking
change for user-code at the moment).

Also, the generic handling of binpac exceptions for analyzers which
to not otherwise catch and handle them has been changed from a Weird
to a ProtocolViolation.

Finally, a new "file_weird" event has been added for reporting
weirdness found during file analysis.
2019-04-01 18:30:11 -07:00
Jon Siwek
956674745b Merge remote-tracking branch 'origin/topic/jazoff/weird-stats-perf'
* origin/topic/jazoff/weird-stats-perf:
  Reduce weird-stats overhead
2019-03-27 12:37:00 -07:00
Jon Siwek
ee00483ee6 Update now-broken Broker API usages
Related to the changes from https://github.com/zeek/broker/pull/38
2019-03-27 10:55:31 -07:00
Justin Azoff
73954bca27 Reduce weird-stats overhead
observe_weird_stats only needs to be called when cluster_ss_request is
called for the weirds.statistics stat, not for all of them.
2019-03-27 11:06:39 -04:00
Jon Siwek
dbf5d5fc95 Deprecate str_shell_escape, add safe_shell_quote replacement 2019-03-25 17:51:04 -07:00
Jon Siwek
8b29df96cc Merge branch 'master' of https://github.com/hosom/zeek
* 'master' of https://github.com/hosom/zeek:
  Normalize the intel seen filename for smb.
  load smb-filenames in scripts/policy/frameworks/intel/seen/__load__.bro
  Add SMB::IN_FILE_NAME to Intel::Where enum
  Support filenamess for SMB files

I added a test case
2019-03-25 16:45:59 -07:00
Jon Siwek
fe2f465023 Merge branch 'topic/jgras/intel-filter' of https://github.com/J-Gras/zeek
* 'topic/jgras/intel-filter' of https://github.com/J-Gras/zeek:
  Added new intel policy script to policy test.
  Added test for intel removal policy script.
  Added policy script for intel removal.
  Added test for intel item filtering.
  Added hook to filter intelligence items.
2019-03-25 09:43:38 -07:00
Jan Grashoefer
a7b9c1d258 Added new intel policy script to policy test. 2019-03-24 22:45:24 +01:00
Jan Grashoefer
2d9b90cddc Added test for intel removal policy script. 2019-03-24 22:24:12 +01:00
Jan Grashoefer
c301e1c9b4 Added policy script for intel removal. 2019-03-24 22:16:13 +01:00
Jan Grashoefer
0f86aaff4b Added test for intel item filtering. 2019-03-24 21:32:45 +01:00
Jan Grashoefer
3e31f3d554 Added hook to filter intelligence items. 2019-03-24 21:31:48 +01:00
Jon Siwek
b2930a6ec3 Updating submodule(s).
[nomail]
2019-03-22 16:35:28 -07:00
Jon Siwek
2b79c7af16 Updating submodule(s).
[nomail]
2019-03-22 13:07:52 -07:00
Jon Siwek
1b76d92e97 Merge branch 'smb3-negotiate-response' of https://github.com/mauropalumbo75/zeek
* 'smb3-negotiate-response' of https://github.com/mauropalumbo75/zeek:
  added test and pcap files for smb 3.1.1 negotiate-response
  smb3.1.1 additions to negotiate-response command

I made several modifications:

  - Code format, style, naming changes

  - For completeness/correctness, I added parsing support for the remaining
    context type structures.

  - Moved the optional padding before the NegotiateContextList field to
    also require the 0x0311 dialect version (some failures in
    pre-existing unit tests pointed this out as an issue)
2019-03-21 14:13:21 -07:00
Jon Siwek
9eb5449ecb Merge branch 'smb3-transform-header' of https://github.com/mauropalumbo75/zeek
* 'smb3-transform-header' of https://github.com/mauropalumbo75/zeek:
  clean up, test and pcap for transform_header added
  added smb2-com-transform-header for smb3.x
2019-03-20 19:30:32 -07:00
Jon Siwek
2d8acab664 Merge branch 'smb2-fix' of https://github.com/mauropalumbo75/zeek
* 'smb2-fix' of https://github.com/mauropalumbo75/zeek:
  added test and pcap files for smb_files.log fix
  fixing some missing log lines in smb_files.log
2019-03-20 18:01:35 -07:00
Jon Siwek
a262ed8d9c Merge branch 'master' of https://github.com/rdenniston/zeek
* 'master' of https://github.com/rdenniston/zeek:
  Add linux netfilter NFLOG capture functionality initial commit

I made modifications:

  - Formatting / code style

  - More error handling and validity checks

  - The Type and Length value of TLVs is technically host order

  - Changed / fixed the Length value padding check: it's generally
    32-bit alignment, not just aligning any TLV less than 8 bytes.
2019-03-19 19:19:02 -07:00
Jon Siwek
eb9a05cd25 GH-307: Build binpac as a shared lib, not static by default 2019-03-18 14:00:54 -07:00
Jon Siwek
f19db92508 Merge branch 'topic/christian/inputframework-paths' of https://github.com/ckreibich/zeek
* 'topic/christian/inputframework-paths' of https://github.com/ckreibich/zeek:
  Source file path control for Input and Intel frameworks
2019-03-18 11:48:22 -07:00
Jon Siwek
01d303b480 Migrate table-based for-loops to key-value iteration 2019-03-15 19:54:44 -07:00
Jon Siwek
41c7b229d3 Merge branch 'master' of https://github.com/ZekeMedley/zeek
* 'master' of https://github.com/ZekeMedley/zeek:
  Add key-value for loop

Fixes GH-154
2019-03-15 19:45:07 -07:00
Christian Kreibich
3f02c0a67c Source file path control for Input and Intel frameworks
This introduces the following redefinable string constants, empty by
default:

- InputAscii::path_prefix
- InputBinary::path_prefix
- Intel::path_prefix

When using ASCII or binary reades in the Input/Intel Framework with an
input stream source that does not have an absolute path, these
constants cause Zeek to prefix the resulting paths accordingly. For
example, in the following the location on disk from which Zeek loads
the input becomes "/path/to/input/whitelist.data":

redef InputAscii::path_prefix = "/path/to/input";

event bro_init()
        {
        Input::add_table([$source="whitelist.data", ...]);
	}

These path prefixes can be absolute or relative. When an input stream
source already uses an absolute path, this path is preserved and the
new variables have no effect (i.e., we do not affect configurations
already using absolute paths).

Since the Intel framework builds upon the Input framework, the first
two paths also affect Intel file locations. If this is undesirable,
the Intel::path_prefix variable allows specifying a separate path:
when its value is absolute, the resulting source seen by the Input
framework is absolute, therefore no further changes to the paths
happen.
2019-03-15 16:43:36 -07:00
Jon Siwek
03ac32adec Merge branch 'topic/dopheide/fix-ssh-geo-data' of https://github.com/dopheide-esnet/bro
* 'topic/dopheide/fix-ssh-geo-data' of https://github.com/dopheide-esnet/bro:
  Fix geo-data to log remote_location data when auth is successful.
2019-03-15 13:03:59 -07:00
Jon Siwek
be7110f6c0 Make Syslog analyzer accept messages that omit Priority
Essentially, it will now process/parse priority values if they are
there, or else just accept whatever remaining data/text is there as the
syslog message.  Reasoning is that there's syslog producers out there
that may have simply forgotten/neglected to send the priority value
and technically won't conform to what the standard says, though we can
infer the intent (some syslog consumers already may do similarly, but
I didn't verify).
2019-03-14 18:47:32 -07:00
Jon Siwek
158313875c Move NEWS file back into main repo from zeek-docs 2019-03-14 16:40:27 -07:00
Jon Siwek
a9771bc0eb Fix signed/unsigned comparison compiler warning 2019-03-14 16:24:27 -07:00
Robin Sommer
72fa3f16ad Merge remote-tracking branch 'origin/topic/jsiwek/vxlan'
* origin/topic/jsiwek/vxlan:
  GH-250: Improve/cleanup VXLAN decapsulation support
  Initial VXLAN support, need -C flag to work when running bro
2019-03-14 17:59:42 +00:00
ZekeMedley
1f7924754e Add key-value for loop 2019-03-14 09:46:16 -07:00
Jon Siwek
389fe2bc2a Merge remote-tracking branch 'origin/topic/jazoff/datastructures-defer-init'
* origin/topic/jazoff/datastructures-defer-init:
  Defer initialization of lists and dicts until an item is added.
2019-03-13 17:34:10 -07:00
Jon Siwek
1239a286e2 Merge branch 'patch-1' of https://github.com/neu5ron/bro
* 'patch-1' of https://github.com/neu5ron/bro:
  Update main.bro
  Update main.bro
2019-03-13 13:49:17 -07:00
Michael Dopheide
0f6f6cdb29 Fix geo-data to log remote_location data when auth is successful. 2019-03-13 14:14:38 -05:00
Robin Sommer
56da70c148 Merge remote-tracking branch 'origin/topic/jsiwek/gh-289'
* origin/topic/jsiwek/gh-289:
  GH-289: Add options to limit entries in http.log file fields
2019-03-13 18:23:06 +00:00
Robin Sommer
aab2d186f4 Merge remote-tracking branch 'origin/topic/jsiwek/gh-282'
Closes #282.

* origin/topic/jsiwek/gh-282:
  GH-282: Remove JSON formatter's range restriction on numbers
2019-03-13 18:14:19 +00:00
Robin Sommer
4cf6cb783a Merge remote-tracking branch 'origin/topic/jsiwek/gh-281'
Closes #281.

* origin/topic/jsiwek/gh-281:
  GH-281: Improve parsing of Google Pixel user agent
2019-03-13 18:09:21 +00:00
Robin Sommer
e2172018ee Merge remote-tracking branch 'origin/topic/jsiwek/gh-286'
Added note about different behaviour to NEWS.

Closes #286.

* origin/topic/jsiwek/gh-286:
  GH-286: Check for record type mismatch in ternary operator
2019-03-13 18:01:36 +00:00
Jon Siwek
09ae539ea8 GH-250: Improve/cleanup VXLAN decapsulation support
* Better parsing/error-checking of VXLAN and encapsulated packet headers

* Add/implement the "vxlan_packet" event

* Add "Tunnel::vxlan_ports" option to tune the set of VXLAN ports to
  analyze/decapsulate

* Add "Tunnel::validate_vxlan_checksums" option to allow for tuning of how
  checksums associated with the outer UDP header of a possible VXLAN
  tunnel are handled

Fixes GH-250
2019-03-12 18:15:34 -07:00
Henrik Lund Kramshoej
f4088be8a6 Initial VXLAN support, need -C flag to work when running bro 2019-03-11 14:15:39 -07:00
Jon Siwek
a36ac12e88 Improve DNS query queuing logic
Related to https://github.com/zeek/zeek/pull/276
2019-03-08 18:38:41 -08:00
Jon Siwek
011b6e10bf Merge remote-tracking branch 'origin/topic/jazoff/dns-perf'
* origin/topic/jazoff/dns-perf:
  update baselines
  Use more efficient method to grab the protocol.
  improve performance of dns policy
2019-03-08 16:23:30 -08:00
Jon Siwek
8e2741544d Updating submodule(s).
[nomail]
2019-03-07 18:49:30 -08:00
Jon Siwek
e2ca695c31 Merge branch 'patch-1' of https://github.com/g0nzu1/zeek
* 'patch-1' of https://github.com/g0nzu1/zeek:
  Minor fixes Line 76 and Line
2019-03-07 13:14:53 -08:00
Jon Siwek
e0a099fdfa Updating submodule(s).
[nomail]
2019-03-07 12:20:18 -08:00