Commit graph

1144 commits

Author SHA1 Message Date
Jon Siwek
569830526f Increase timeout of openflow.log-cluster btest 2021-03-23 19:34:34 -07:00
Jon Siwek
737d2c390b Support explicit disabling of file analyzers 2021-02-23 15:50:18 -08:00
Johanna Amann
6c554ddde3 Ascii reader test with 0-bytes
Add a second test, which contains actual 0-bytes.
2021-02-12 14:49:33 +00:00
Johanna Amann
61290fc19c Fix buffer overread in ascii formatter
When a text with an (escaped) zero byte was passed to ParseValue, only
the part of the string up to the zero byte was copied, but the length of
the full string was passed to the input framework.

This leads to the input manager reading over the end of the buffer.

Fixes zeek/zeek#1398
2021-02-12 14:24:36 +00:00
Vern Paxson
c991c54690 &is_set => &is_assigned 2021-02-04 12:18:46 -08:00
Jon Siwek
cfe29c2488 Merge branch 'master' into topic/vern/reaching-defs 2021-02-01 17:27:37 -08:00
Jon Siwek
1ca85f0221 Merge remote-tracking branch 'origin/topic/timw/deprecation-cleanup'
Merge adjustments:
- Removed some stale str_split() references from docs
- Renumbered TypeTag enum comments
- Simplified test-case for @unload (don't need .bro files anymore)

* origin/topic/timw/deprecation-cleanup:
  Doc updates
  Fix language.init-in-anon-function btest due to changes to log filter predicates
  Remove deprecated log filter predicates for 4.1
  Remove Plugin::HookCallFunction and fix tests related to it
  Remove support for .bro script extension and BRO_ environment variables
  Remove deprecated ICMP events
  Remove some deprected methods/events from bif files
  Remove TYPE_COUNTER
  Remove all of the random single-file deprecations
  Remove all fully-deprecated files
  Update bifcl submodule to remove deprecations from generated code
2021-01-29 16:40:54 -08:00
Christian Kreibich
38d6b6a98b Skip negative integers in input framework where not type-permissible
Script-layer counts, when provided as negative integers in an input
file, got cast to unsigned values because strtoull() does not complain
about negative values. For example, input string "-1" would lead to
value 18446744073709551615 (an all-ones 64-bit int) on x86_64. This is
more likely to be an error than an intent to get very large,
platform-dependent values, so these input lines are now skipped with
according messaging in the reporter.log/stderr.

This also affected ports: -1/tcp got cast to unsigned and only thrown
out because PortVal rejects values > 65535, mapping them to 0. We now
skip such inputs as well.

Updates existing input framework tests to capture the new behavior.
2021-01-28 23:46:43 -08:00
Christian Kreibich
4ce3bf3cd2 Remove deprecated log filter predicates for 4.1
Update the logging framework tests: since hooks operate
by name, they cannot be anonymous. I'm also dropping the &optional
attribute from the status field, since here know that the values are
actually defined, and access to an optional status field should
normally be guarded by the existence test operator.

Also includes baseline update for plugins.hooks, which picks up the
fact that the pred record field is now gone.
2021-01-27 10:52:40 -07:00
Tim Wojtulewicz
44ad614094 Remove deprecated ICMP events 2021-01-27 10:52:40 -07:00
Vern Paxson
c43925b7ee updates to test suite, including new baseline for "-a usage" environment 2021-01-23 10:57:08 -08:00
Jon Oakley
25de6f28e5 GH-1352: Added flag to stop processing SMTP headers in attached
messages
2021-01-21 14:55:10 -05:00
Vern Paxson
dc38880cee canonicalization for an error message in one of the alternative test baselines 2021-01-10 14:44:46 -08:00
Jon Siwek
64631a2d9f Merge branch 'master' into topic/vern/script-inlining 2021-01-06 22:08:29 -08:00
Jon Siwek
604fbea00d Merge remote-tracking branch 'origin/topic/johanna/1323'
* origin/topic/johanna/1323:
  TLS 1.3 changes: Address review feedback
  Add one more TLS 1.3 testcase and update NEWS
  Fix TLS 1.3 session resumption detection.
  Introduce ssl_probable_encrypted_handshake_message event
  SSL Analyzer: ignore CCS for TLS 1.3
  TLS analyzer: change logic to track TLS 1.3 connection establishment
2020-12-22 11:00:59 -08:00
Johanna Amann
22ed75c3ce Add one more TLS 1.3 testcase and update NEWS 2020-12-15 16:57:26 +00:00
Johanna Amann
3c95c9a956 Fix TLS 1.3 session resumption detection.
Now we detect TLS 1.3 session resumption by looking if both sides have
the PSK extension set, which is much more exact than the previous
approach.
2020-12-15 16:34:47 +00:00
Jon Siwek
206c674cc9 Merge EDNS ECS option parsing security/bug fixes 2020-12-15 08:03:40 -08:00
Jon Siwek
dd7bda6a1c Fix local.zeek compatibility test 2020-12-14 22:28:27 -08:00
Jon Siwek
30df2c58c0 Merge branch 'master' into topic/vern/script-inlining 2020-12-13 10:42:26 -08:00
Jon Siwek
7c27d4c1e7 Add test case to cover weird EDNS ECS parsing situations 2020-12-08 13:14:20 -08:00
Jon Siwek
0b8535b879 Merge remote-tracking branch 'origin/topic/vlad/gh-1286'
Merge adjustments:
- Rewrote the check for error response as a switch statement to
  fix compiler warning about signed/unsigned comparison and also
  to just simplify/clarify the logic.
- Changed the btest to use `zeek -b`.

* origin/topic/vlad/gh-1286:
  Add tests for new SMB3 multichannel support
  Fix SMB2 response status parsing. Fixes #1286
2020-12-07 16:08:04 -08:00
Vlad Grigorescu
f30ca69402 Add tests for new SMB3 multichannel support 2020-12-07 15:35:04 -06:00
Christian Kreibich
17cd740519 Update btest-diff calls on binary files to using "--binary"
This converts Johanna's TEST_DIFF_CANONIFIER removals via "env -u" to
using the new "btest-diff --binary", and updates the affected baselines.
2020-12-06 20:19:52 -08:00
Johanna Amann
442cb40db8 Fix binary baseline & line-end problem
By default all baslines are run through diff-remove-timestamp. On a BSD
sed implementation, this means that a newline is added to the end of the
file, if no newline was there originally. This behavior differs from GNU
sed, which does not add a newline.

In this commit we unify this behavior by always adding a newline, even
when using GNU sed. This commit also disables the canonifier for a bunch
of binary baselines, so we do not have to change them.
2020-12-06 20:19:52 -08:00
Christian Kreibich
c0bd02e18f Canonifier improvements for the scripts.base.frameworks.logging.ascii-double test
This now more surgically applies canonifiers so that the double-format
numerical output isn't itself canonified.
2020-12-06 20:19:52 -08:00
Jon Siwek
ff7d9e3d72 Merge branch 'master' into topic/vern/script-inlining 2020-12-04 16:18:14 -08:00
Johanna Amann
47ceac2491 Merge remote-tracking branch 'origin/topic/johanna/GH-348'
* origin/topic/johanna/GH-348:
  Sumstats: allow users to manage epoch manually
  Sumstats: epoch_finished was not called under certain circumstances
2020-12-04 18:49:53 +00:00
Jon Siwek
9717c623c2 Update Python invocations to explicit python3 2020-11-25 14:57:09 -08:00
Vern Paxson
c42586af2c inlining of Zeek script functions 2020-11-19 16:16:59 -08:00
Tim Wojtulewicz
01ec5ebdcd Reverts the regex change in dead3226a5.
The regex change broke some of the external tests. I added some more cases
to the regular email btest to hopefully cover all of the cases better.
2020-11-18 15:35:17 -07:00
Tim Wojtulewicz
a56fbe408f Merge remote-tracking branch 'christina23/master'
* christina23/master:
  Added unit tests for regex fix
  Improved regex for SMTP parsing
2020-11-17 15:01:21 -07:00
Jon Siwek
6ed4850a26 Merge remote-tracking branch 'origin/topic/jsiwek/gh-352-improve-sql-injection-regex' into master
* origin/topic/jsiwek/gh-352-improve-sql-injection-regex:
  GH-352: Improve HTTP::match_sql_injection_uri regex
2020-11-16 11:51:04 -08:00
Jon Siwek
02c0b33b54 Merge remote-tracking branch 'origin/topic/jsiwek/gh-1264-ssh-host-key-fingerprints' into master
* origin/topic/jsiwek/gh-1264-ssh-host-key-fingerprints:
  Simply ssh/main.zeek by using "ssh_server_host_key" for fingerprinting
  Deprecate "ssh1_server_host_key" parameters *e* and *p*
  GH-1264: Implement "ssh_server_host_key" event
2020-11-16 11:22:37 -08:00
christina23
bb7af9a823 Added unit tests for regex fix 2020-11-16 11:43:35 -05:00
Johanna Amann
7362f30c3a Sumstats: allow users to manage epoch manually
This change allows users to specify an epoch length of 0, which means
that the user manually has to finish the epochs. A new next_epoch
function is introduced to allow users to manually end epochs.

Addresses GH-348
2020-11-16 15:48:46 +00:00
Brevet Vivien
3769ed6c66 [SSH] Handle SSH version 1.99
SSH can set in its identification a version 1.99 (SSH-1.99-xxx).
That means the client/server is compatible with SSHv1 and SSHv2.
So the version choice depends of the both side.

1.99 : 1.99 => 2.0
1.99 : 1.x  => 1.x
1.99 : 2.0  => 2.O

(see "Compatibility With Old SSH Versions" in RFC 4253)
2020-11-14 15:33:34 +01:00
Jon Siwek
45449dad72 Deprecate "ssh1_server_host_key" parameters *e* and *p*
They are named such that *e* is actually the modulus, not the exponent.
The replacement parameters are named *exponent* and *modulus* for
clarity.
2020-11-13 22:58:56 -08:00
Jon Siwek
bd40a97a78 GH-1264: Implement "ssh_server_host_key" event
This event provides host key fingerprints for both SSH1 and SSH2.
2020-11-13 22:58:56 -08:00
Jon Siwek
ae923106f1 GH-352: Improve HTTP::match_sql_injection_uri regex
Changes \x00-\x37 ranges to \x00-\x1f with assumption that the former
was attempting to match ASCII control characters, but mistook an octal
range for hex.  This change reduces some false positives.
2020-11-12 16:19:35 -08:00
Robin Sommer
78fc5615a5 Merge remote-tracking branch 'origin/topic/christian/fix-extfunc-includes'
* origin/topic/christian/fix-extfunc-includes:
  Fix type clash fatal error with log filters that use $ext_func and $include/$exclude
2020-11-12 10:33:48 +00:00
Tim Wojtulewicz
a6a4b976ec Support for additional DNS RR Type: LOC[29], SSHFP[44], NSEC3PARAM[51], custom BIND9 signaling[65534] 2020-11-11 13:35:51 -07:00
Christian Kreibich
f97a33e14d Fix type clash fatal error with log filters that use $ext_func and $include/$exclude
The logging manager's Manager::TraverseRecord(), called when adding a
log filter to a stream, skipped any fields intoduced by a filter's
$ext_func when such fields weren't mentioned in a $include restriction
or mentioned in an $exclude restriction. This was inconsistent with
Manager::RecordToFilterVals, used when actually writing log entries,
which does include those values.

The result was that the record indices descent in Manager::RecordToFilterVals
expects to find only record values, when in fact only the record
provided by ext_func is present. This leads to type mismatches and
hard Zeek exits like this one:

1300475173.475401 fatal error in zeek/share/zeek//base/init-bare.zeek, line 4810: Val::CONVERTER (string/record) (zeek)

The fix makes ext_func's field additions decisive, meaning the
filter's include/exclude lists don't apply to it. If a user really
wants to override this, they can reset the filter's ext_func back to
our no-op default.

The included btest produces the above error when the fix is not present.
2020-11-10 18:13:16 -08:00
Jon Siwek
b7bd87d9cf GH-1252: rename files with colons for Windows compatibility
A short-term measure so that the Zeek source tree can simply exist on a
Windows filesystem.  For true support, the logic that decides/generates
the filename format will need to change.
2020-11-04 10:29:52 -08:00
Jon Siwek
5e164469f3 Merge remote-tracking branch 'origin/topic/vlad/gh-1225'
* origin/topic/vlad/gh-1225:
  Extract length as a uint8
  Change ICMP ND length to a uint16
2020-10-16 10:37:08 -07:00
Vlad Grigorescu
b5d11d1ace Change ICMP ND length to a uint16 2020-10-15 16:56:05 -05:00
Johanna Amann
e6805258dc dump-events: try to make baseline work on all systems 2020-10-15 15:35:51 +00:00
Johanna Amann
e95e9b8a5d Merge remote-tracking branch 'origin/master' into topic/johanna/GH-169 2020-10-15 15:31:23 +00:00
Johanna Amann
f4d9cf5f89 Introduce generate_all_events bif and add option to misc/dump-events
generate_all_events causes all events to be raised internally; this
makes it possible for dump_events to really capture all events (and not
just those that were handled).

Addresses GH-169
2020-10-12 16:21:18 +02:00
Vlad Grigorescu
c58cf69890 Add test for CaptureLoss::Too_Little_Traffic 2020-10-08 11:34:04 -05:00