Commit graph

1131 commits

Author SHA1 Message Date
Jon Siwek
64631a2d9f Merge branch 'master' into topic/vern/script-inlining 2021-01-06 22:08:29 -08:00
Jon Siwek
604fbea00d Merge remote-tracking branch 'origin/topic/johanna/1323'
* origin/topic/johanna/1323:
  TLS 1.3 changes: Address review feedback
  Add one more TLS 1.3 testcase and update NEWS
  Fix TLS 1.3 session resumption detection.
  Introduce ssl_probable_encrypted_handshake_message event
  SSL Analyzer: ignore CCS for TLS 1.3
  TLS analyzer: change logic to track TLS 1.3 connection establishment
2020-12-22 11:00:59 -08:00
Johanna Amann
22ed75c3ce Add one more TLS 1.3 testcase and update NEWS 2020-12-15 16:57:26 +00:00
Johanna Amann
3c95c9a956 Fix TLS 1.3 session resumption detection.
Now we detect TLS 1.3 session resumption by looking if both sides have
the PSK extension set, which is much more exact than the previous
approach.
2020-12-15 16:34:47 +00:00
Jon Siwek
206c674cc9 Merge EDNS ECS option parsing security/bug fixes 2020-12-15 08:03:40 -08:00
Jon Siwek
dd7bda6a1c Fix local.zeek compatibility test 2020-12-14 22:28:27 -08:00
Jon Siwek
30df2c58c0 Merge branch 'master' into topic/vern/script-inlining 2020-12-13 10:42:26 -08:00
Jon Siwek
7c27d4c1e7 Add test case to cover weird EDNS ECS parsing situations 2020-12-08 13:14:20 -08:00
Jon Siwek
0b8535b879 Merge remote-tracking branch 'origin/topic/vlad/gh-1286'
Merge adjustments:
- Rewrote the check for error response as a switch statement to
  fix compiler warning about signed/unsigned comparison and also
  to just simplify/clarify the logic.
- Changed the btest to use `zeek -b`.

* origin/topic/vlad/gh-1286:
  Add tests for new SMB3 multichannel support
  Fix SMB2 response status parsing. Fixes #1286
2020-12-07 16:08:04 -08:00
Vlad Grigorescu
f30ca69402 Add tests for new SMB3 multichannel support 2020-12-07 15:35:04 -06:00
Christian Kreibich
17cd740519 Update btest-diff calls on binary files to using "--binary"
This converts Johanna's TEST_DIFF_CANONIFIER removals via "env -u" to
using the new "btest-diff --binary", and updates the affected baselines.
2020-12-06 20:19:52 -08:00
Johanna Amann
442cb40db8 Fix binary baseline & line-end problem
By default all baslines are run through diff-remove-timestamp. On a BSD
sed implementation, this means that a newline is added to the end of the
file, if no newline was there originally. This behavior differs from GNU
sed, which does not add a newline.

In this commit we unify this behavior by always adding a newline, even
when using GNU sed. This commit also disables the canonifier for a bunch
of binary baselines, so we do not have to change them.
2020-12-06 20:19:52 -08:00
Christian Kreibich
c0bd02e18f Canonifier improvements for the scripts.base.frameworks.logging.ascii-double test
This now more surgically applies canonifiers so that the double-format
numerical output isn't itself canonified.
2020-12-06 20:19:52 -08:00
Jon Siwek
ff7d9e3d72 Merge branch 'master' into topic/vern/script-inlining 2020-12-04 16:18:14 -08:00
Johanna Amann
47ceac2491 Merge remote-tracking branch 'origin/topic/johanna/GH-348'
* origin/topic/johanna/GH-348:
  Sumstats: allow users to manage epoch manually
  Sumstats: epoch_finished was not called under certain circumstances
2020-12-04 18:49:53 +00:00
Jon Siwek
9717c623c2 Update Python invocations to explicit python3 2020-11-25 14:57:09 -08:00
Vern Paxson
c42586af2c inlining of Zeek script functions 2020-11-19 16:16:59 -08:00
Tim Wojtulewicz
01ec5ebdcd Reverts the regex change in dead3226a5.
The regex change broke some of the external tests. I added some more cases
to the regular email btest to hopefully cover all of the cases better.
2020-11-18 15:35:17 -07:00
Tim Wojtulewicz
a56fbe408f Merge remote-tracking branch 'christina23/master'
* christina23/master:
  Added unit tests for regex fix
  Improved regex for SMTP parsing
2020-11-17 15:01:21 -07:00
Jon Siwek
6ed4850a26 Merge remote-tracking branch 'origin/topic/jsiwek/gh-352-improve-sql-injection-regex' into master
* origin/topic/jsiwek/gh-352-improve-sql-injection-regex:
  GH-352: Improve HTTP::match_sql_injection_uri regex
2020-11-16 11:51:04 -08:00
Jon Siwek
02c0b33b54 Merge remote-tracking branch 'origin/topic/jsiwek/gh-1264-ssh-host-key-fingerprints' into master
* origin/topic/jsiwek/gh-1264-ssh-host-key-fingerprints:
  Simply ssh/main.zeek by using "ssh_server_host_key" for fingerprinting
  Deprecate "ssh1_server_host_key" parameters *e* and *p*
  GH-1264: Implement "ssh_server_host_key" event
2020-11-16 11:22:37 -08:00
christina23
bb7af9a823 Added unit tests for regex fix 2020-11-16 11:43:35 -05:00
Johanna Amann
7362f30c3a Sumstats: allow users to manage epoch manually
This change allows users to specify an epoch length of 0, which means
that the user manually has to finish the epochs. A new next_epoch
function is introduced to allow users to manually end epochs.

Addresses GH-348
2020-11-16 15:48:46 +00:00
Brevet Vivien
3769ed6c66 [SSH] Handle SSH version 1.99
SSH can set in its identification a version 1.99 (SSH-1.99-xxx).
That means the client/server is compatible with SSHv1 and SSHv2.
So the version choice depends of the both side.

1.99 : 1.99 => 2.0
1.99 : 1.x  => 1.x
1.99 : 2.0  => 2.O

(see "Compatibility With Old SSH Versions" in RFC 4253)
2020-11-14 15:33:34 +01:00
Jon Siwek
45449dad72 Deprecate "ssh1_server_host_key" parameters *e* and *p*
They are named such that *e* is actually the modulus, not the exponent.
The replacement parameters are named *exponent* and *modulus* for
clarity.
2020-11-13 22:58:56 -08:00
Jon Siwek
bd40a97a78 GH-1264: Implement "ssh_server_host_key" event
This event provides host key fingerprints for both SSH1 and SSH2.
2020-11-13 22:58:56 -08:00
Jon Siwek
ae923106f1 GH-352: Improve HTTP::match_sql_injection_uri regex
Changes \x00-\x37 ranges to \x00-\x1f with assumption that the former
was attempting to match ASCII control characters, but mistook an octal
range for hex.  This change reduces some false positives.
2020-11-12 16:19:35 -08:00
Robin Sommer
78fc5615a5 Merge remote-tracking branch 'origin/topic/christian/fix-extfunc-includes'
* origin/topic/christian/fix-extfunc-includes:
  Fix type clash fatal error with log filters that use $ext_func and $include/$exclude
2020-11-12 10:33:48 +00:00
Tim Wojtulewicz
a6a4b976ec Support for additional DNS RR Type: LOC[29], SSHFP[44], NSEC3PARAM[51], custom BIND9 signaling[65534] 2020-11-11 13:35:51 -07:00
Christian Kreibich
f97a33e14d Fix type clash fatal error with log filters that use $ext_func and $include/$exclude
The logging manager's Manager::TraverseRecord(), called when adding a
log filter to a stream, skipped any fields intoduced by a filter's
$ext_func when such fields weren't mentioned in a $include restriction
or mentioned in an $exclude restriction. This was inconsistent with
Manager::RecordToFilterVals, used when actually writing log entries,
which does include those values.

The result was that the record indices descent in Manager::RecordToFilterVals
expects to find only record values, when in fact only the record
provided by ext_func is present. This leads to type mismatches and
hard Zeek exits like this one:

1300475173.475401 fatal error in zeek/share/zeek//base/init-bare.zeek, line 4810: Val::CONVERTER (string/record) (zeek)

The fix makes ext_func's field additions decisive, meaning the
filter's include/exclude lists don't apply to it. If a user really
wants to override this, they can reset the filter's ext_func back to
our no-op default.

The included btest produces the above error when the fix is not present.
2020-11-10 18:13:16 -08:00
Jon Siwek
b7bd87d9cf GH-1252: rename files with colons for Windows compatibility
A short-term measure so that the Zeek source tree can simply exist on a
Windows filesystem.  For true support, the logic that decides/generates
the filename format will need to change.
2020-11-04 10:29:52 -08:00
Jon Siwek
5e164469f3 Merge remote-tracking branch 'origin/topic/vlad/gh-1225'
* origin/topic/vlad/gh-1225:
  Extract length as a uint8
  Change ICMP ND length to a uint16
2020-10-16 10:37:08 -07:00
Vlad Grigorescu
b5d11d1ace Change ICMP ND length to a uint16 2020-10-15 16:56:05 -05:00
Johanna Amann
e6805258dc dump-events: try to make baseline work on all systems 2020-10-15 15:35:51 +00:00
Johanna Amann
e95e9b8a5d Merge remote-tracking branch 'origin/master' into topic/johanna/GH-169 2020-10-15 15:31:23 +00:00
Johanna Amann
f4d9cf5f89 Introduce generate_all_events bif and add option to misc/dump-events
generate_all_events causes all events to be raised internally; this
makes it possible for dump_events to really capture all events (and not
just those that were handled).

Addresses GH-169
2020-10-12 16:21:18 +02:00
Vlad Grigorescu
c58cf69890 Add test for CaptureLoss::Too_Little_Traffic 2020-10-08 11:34:04 -05:00
Christian Kreibich
c662237499 Update testing/btest/scripts/base/frameworks/logging/hooks.zeek
Co-authored-by: Robin Sommer <robin@corelight.com>
2020-09-30 13:10:35 -07:00
Christian Kreibich
3d2c3284c7 Btests for log filter policy hooks 2020-09-30 13:10:35 -07:00
Jon Siwek
b065b6ed61 Merge remote-tracking branch 'origin/topic/robin/filtered-trace'
* origin/topic/robin/filtered-trace:
  Tweak find-filtered-trace to not flag traces if they have non-TCP traffic.
2020-09-25 15:12:04 -07:00
Jon Siwek
c1492942bb Merge branch 'add-dce-rpc-payloads' of https://github.com/ynadji/zeek
- Changed the new stub events to correctly check for existence of
  their associated handler before generating an event

- Added a test case for the new stub event

* 'add-dce-rpc-payloads' of https://github.com/ynadji/zeek:
  Add stub payload to dce_rpc_request and dce_rpc_response
2020-09-25 14:39:30 -07:00
Robin Sommer
1f6fc4415f Tweak find-filtered-trace to not flag traces if they have non-TCP
traffic.

Closes #160.
2020-09-25 11:29:44 +00:00
Peter Oettig
b2e6c9ac9a Initial implementation of Lower-Level analyzers 2020-09-23 11:13:25 -07:00
Jon Siwek
01d83c3204 Avoid race condition in control/id_value btest 2020-09-22 14:11:17 -07:00
Tim Wojtulewicz
6b93020c6f Merge remote-tracking branch 'origin/topic/jsiwek/gh-1164-fix-rst-conn-state'
* origin/topic/jsiwek/gh-1164-fix-rst-conn-state:
  GH-1164: Fix incorrect RSTOS0 conn_state determinations
2020-09-17 13:40:06 -07:00
Tim Wojtulewicz
3b641870dc Merge remote-tracking branch 'origin/topic/jsiwek/gh-1119-conn-removal-callbacks'
* origin/topic/jsiwek/gh-1119-conn-removal-callbacks:
  GH-1119: add base/protcols/conn/removal-hooks.zeek
  Avoid run-time cost of find-filtered-trace.zeek for live traffic
  Remove connection_successful and successful_connection_remove events
2020-09-17 13:16:19 -07:00
Jon Siwek
1046e45dba Improve notice/suppression-cluster btest to avoid race potential 2020-09-14 12:28:25 -07:00
Jon Siwek
a16bd47bf7 GH-1164: Fix incorrect RSTOS0 conn_state determinations
The RSTOS0 `conn_state` label is documented as "Originator sent a SYN
followed by a RST, never saw SYN-ACK from responder", but was previously
applied to cases where no originator SYN exists, like a single RST-only
packet.
2020-09-11 16:14:41 -07:00
Jon Siwek
05cf511f18 GH-1119: add base/protcols/conn/removal-hooks.zeek
This adds two new functions: `Conn::register_removal_hook()` and
`Conn::unregister_removal_hook()` for registering a hook function to be
called back during `connection_state_remove`.  The benefit of using hook
callback approach is better scalability: the overhead of unrelated
protocols having to dispatch no-op `connection_state_remove` handlers is
avoided.
2020-09-11 12:12:10 -07:00
Robin Sommer
0af57d12b2 Change HTTP's DPD signatures so that each side can trigger the analyzer on its own.
This is to avoid missing large sessions where a single side exceeds
the DPD buffer size. It comes with the trade-off that now the analyzer
can be triggered by anybody controlling one of the endpoints (instead
of both).

Test suite changes are minor, and nothing in "external".

Closes #343.
2020-09-08 07:33:36 +00:00