Commit graph

3318 commits

Author SHA1 Message Date
Jon Siwek
8c85f2135e GH-1211: Improve error message for already-defined functions 2020-10-12 16:19:19 -07:00
Johanna Amann
f4d9cf5f89 Introduce generate_all_events bif and add option to misc/dump-events
generate_all_events causes all events to be raised internally; this
makes it possible for dump_events to really capture all events (and not
just those that were handled).

Addresses GH-169
2020-10-12 16:21:18 +02:00
Jon Siwek
5904d0708f GH-779: Add "udp-state" signature condition
It accepts "originator" or "responder" states as a way to enforce that
the signature only matches packets in the associated direction.
The "established" state is rejected as an error since it doesn't
have a useful meaning like it does for the "tcp-state" condition.
2020-10-09 13:43:17 -07:00
Seth Hall
ce590ae05f Add a test for script args. 2020-10-08 16:32:54 -04:00
Vlad Grigorescu
c58cf69890 Add test for CaptureLoss::Too_Little_Traffic 2020-10-08 11:34:04 -05:00
Robin Sommer
b0bf9f02c8 Merge remote-tracking branch 'origin/topic/christian/364-logfilter-hooks' into master
(Adding a NEWS entry.)

* origin/topic/christian/364-logfilter-hooks:
  Update testing/btest/scripts/base/frameworks/logging/hooks.zeek
  Btests for log filter policy hooks
  Btest baseline updates to reflect new logging policy hooks
  Migrate existing use of filter predicates to policy hooks
  Support for log filter policy hooks
2020-10-07 08:44:50 +00:00
Robin Sommer
d782c60f19 Unify type comparisions for records.
For records, same_type(r1, r2) would not check if the fields'
attributes match as well. That seems like an oversight, and some
callers of same_type() did indeed add that check on their end. This
commit moves the check into same_type() itself. That generally doesn't
seem make any differences except for a couple of places validating
code, which we update a bit. That in turn leans to slightly different
(better?) error messages for a couple of test cases.
2020-10-05 10:47:46 +00:00
Christian Kreibich
c662237499 Update testing/btest/scripts/base/frameworks/logging/hooks.zeek
Co-authored-by: Robin Sommer <robin@corelight.com>
2020-09-30 13:10:35 -07:00
Christian Kreibich
3d2c3284c7 Btests for log filter policy hooks 2020-09-30 13:10:35 -07:00
Christian Kreibich
2a49715fe6 Btest baseline updates to reflect new logging policy hooks 2020-09-30 13:10:07 -07:00
Jon Siwek
c8545c85d8 Improve broker/ssl_auth_failure.zeek test
There was a race that previously may cause it to not truly test
whether the connecting-process exited due to SSL authentication
failure or just because the listening-process wasn't available yet
(e.g. due to process scheduling variability).
2020-09-29 17:12:03 -07:00
Jon Siwek
b065b6ed61 Merge remote-tracking branch 'origin/topic/robin/filtered-trace'
* origin/topic/robin/filtered-trace:
  Tweak find-filtered-trace to not flag traces if they have non-TCP traffic.
2020-09-25 15:12:04 -07:00
Jon Siwek
c1492942bb Merge branch 'add-dce-rpc-payloads' of https://github.com/ynadji/zeek
- Changed the new stub events to correctly check for existence of
  their associated handler before generating an event

- Added a test case for the new stub event

* 'add-dce-rpc-payloads' of https://github.com/ynadji/zeek:
  Add stub payload to dce_rpc_request and dce_rpc_response
2020-09-25 14:39:30 -07:00
Robin Sommer
1f6fc4415f Tweak find-filtered-trace to not flag traces if they have non-TCP
traffic.

Closes #160.
2020-09-25 11:29:44 +00:00
Jon Siwek
8feca7291b Merge remote-tracking branch 'origin/topic/jsiwek/gh-822-ubsan-ci'
* origin/topic/jsiwek/gh-822-ubsan-ci:
  Fix negative-value-left-shift undefined behavior in patricia trie
  Improve negation of ConstExpr
  Avoid signed integer overflow when combining SMB header PID bits
  Avoid unary negation of INT64_MIN in modp_litoa10
  Avoid double-to-int conversion overflows in modp_dtoa functions
  Fix divide-by-zero in Entropy analyzer
  Fix divide-by-zero in stats/profiling memory usage calculation
  Fix uninitialized field in POP3 fuzzer
  Add framework for running UndefinedBehaviorSanitizer in CI
2020-09-24 08:17:58 -07:00
Jan Grashoefer
7ede4f48bd Simplify packet analyzer config. 2020-09-23 11:13:29 -07:00
Jan Grashoefer
efa262a229 Make default packet analyzer definition explicit. 2020-09-23 11:13:29 -07:00
Jan Grashoefer
8f951574d7 Add explicit root analyzer for packet analysis. 2020-09-23 11:13:29 -07:00
Jan Grashoefer
d51252bb3f Extend packet analysis test. 2020-09-23 11:13:29 -07:00
Jan Grashoefer
3f3f00030d Simplify MPLS analysis. 2020-09-23 11:13:29 -07:00
Jan Grashoefer
38337d799b Improve packet analysis data flow. 2020-09-23 11:13:29 -07:00
Jan Grashoefer
90eb97876f Improve packet analyzer API. 2020-09-23 11:13:28 -07:00
Jan Grashoefer
d5ca0f9da5 Rename DefaultAnalyzer to IP. 2020-09-23 11:13:28 -07:00
Jan Grashoefer
24babf096e Move ARP analysis into packet analyzer. 2020-09-23 11:13:28 -07:00
Tim Wojtulewicz
2d7280fabd Fix some build issues from the rebase 2020-09-23 11:13:28 -07:00
Jan Grashoefer
aa4506ad06 Add skip analyzer test and update baselines. 2020-09-23 11:13:28 -07:00
Jan Grashoefer
fc814bd7e2 Add SkipAnalyzer.
This is WIP: The test case would require a new pcap or the possibility
to overwrite analyzer mappings. The CustomEncapsulationSkip method and
the corresponding options need to be removed.
2020-09-23 11:13:28 -07:00
Jan Grashoefer
6365fa6d80 Migrate all packet analyzers to new API. 2020-09-23 11:13:28 -07:00
Jan Grashoefer
96d0e11bb8 Move cur_pos from packet into packet manager loop. 2020-09-23 11:13:28 -07:00
Tim Wojtulewicz
c2500d03d6 Remove packet_analysis/Defines.h
- Replace uses of identifier_t with uint32_t
- Replace repeated usage of tuple type for Analysis results with type alias
2020-09-23 11:13:28 -07:00
Tim Wojtulewicz
bd6d3e0112 Remove enabled state from Components, ability to enable/disable from Manager 2020-09-23 11:13:28 -07:00
Tim Wojtulewicz
f39d6bb4c4 Use shared_ptr instead of raw pointers in packet_analysis for analyzers and dispatchers 2020-09-23 11:13:28 -07:00
Tim Wojtulewicz
08ceea8de1 Fixes for various btest issues
- Fix handling of truncated ethernet headers, fix core.truncation test output
- Update commit hashes for external private test repo
2020-09-23 11:13:28 -07:00
Jan Grashoefer
1e0e8e35af Minor fixes for packet analyzer renaming. 2020-09-23 11:13:28 -07:00
Jan Grashoefer
e53ec46c23 Renamed LL-Analyzers to Packet Analyzers. 2020-09-23 11:13:28 -07:00
Peter Oettig
b2e6c9ac9a Initial implementation of Lower-Level analyzers 2020-09-23 11:13:25 -07:00
Jon Siwek
01d83c3204 Avoid race condition in control/id_value btest 2020-09-22 14:11:17 -07:00
Jon Siwek
75b3ff7da4 Merge remote-tracking branch 'origin/topic/jsiwek/gh-927-when-condition-eval-order' into master
* origin/topic/jsiwek/gh-927-when-condition-eval-order:
  GH-927: Fix circumvention of evaluation order in 'when' conditions
2020-09-22 11:09:51 -07:00
Jon Siwek
a44b056e81 Improve negation of ConstExpr
* Instead of creating a NegExpr for negation of a literal/constant,
  a ConstExpr is now created directly.

* For negation of integer literals, there's now an additional check
  for whether the integer would be outside the range of possible 'int'
  values.  This can also help prevent the undefined behavior due to
  overflow as a result of trying to represent the minimum 'int' value of
  -9223372036854775808 as a literal in a script -- the unsigned value is
  cast to signed yielding INT64_MIN, then INT64_MIN is negated.
2020-09-18 11:32:05 -07:00
Jon Siwek
efcbf979f5 Add framework for running UndefinedBehaviorSanitizer in CI
Many checks are initially disabled because they do cause failures
when running the test suites.
2020-09-18 11:32:05 -07:00
Tim Wojtulewicz
6b93020c6f Merge remote-tracking branch 'origin/topic/jsiwek/gh-1164-fix-rst-conn-state'
* origin/topic/jsiwek/gh-1164-fix-rst-conn-state:
  GH-1164: Fix incorrect RSTOS0 conn_state determinations
2020-09-17 13:40:06 -07:00
Tim Wojtulewicz
3b641870dc Merge remote-tracking branch 'origin/topic/jsiwek/gh-1119-conn-removal-callbacks'
* origin/topic/jsiwek/gh-1119-conn-removal-callbacks:
  GH-1119: add base/protcols/conn/removal-hooks.zeek
  Avoid run-time cost of find-filtered-trace.zeek for live traffic
  Remove connection_successful and successful_connection_remove events
2020-09-17 13:16:19 -07:00
Tim Wojtulewicz
26808ea7d4 Merge remote-tracking branch 'origin/topic/jsiwek/gh-1155-recursive-table-index-type-check'
* origin/topic/jsiwek/gh-1155-recursive-table-index-type-check:
  GH-1155: Recursively check table index for unsupported types
2020-09-17 11:42:57 -07:00
Jon Siwek
33ca675515 GH-927: Fix circumvention of evaluation order in 'when' conditions
Historically, a 'when' condition performed an AST-traversal to locate
any index-expressions like `x[9]` and evaluated them so that it could
register the associated value as something for which it needs to receive
"modification" notifications.

Evaluating arbitrary expressions during an AST-traversal like that ignores
the typical order-of-evaluation/short-circuiting you'd expect if the
condition was evaluated normally, from its root expression.

Now, a new subclass of IndexExpr is used to keep track of all IndexExpr
results in the context of evaluating a 'when' condition without having
to do a secondary AST-traversal-and-eval.  i.e. the first evaluation of
the full 'when' condition follows the typical expression-evaluation
semantics (as always), but additionally now captures all the values
a Trigger needs to monitor for modifications.
2020-09-15 17:03:44 -07:00
Jon Siwek
a96440ea79 GH-1155: Recursively check table index for unsupported types
Previously, container types used within a table/set index were not
deeply checked to ensure all constituents could be part of an index.
2020-09-14 19:06:11 -07:00
Jon Siwek
3b334bad56 GH-1159: Fix vector-of-interval multiplication/division arithmetic
Those operations done between a vector-of-interval and a
vector-of-arithmetic-type previously threw a runtime expression error
due to an incorrect coercion being used internally.
2020-09-14 16:12:20 -07:00
Jon Siwek
1046e45dba Improve notice/suppression-cluster btest to avoid race potential 2020-09-14 12:28:25 -07:00
Jon Siwek
a16bd47bf7 GH-1164: Fix incorrect RSTOS0 conn_state determinations
The RSTOS0 `conn_state` label is documented as "Originator sent a SYN
followed by a RST, never saw SYN-ACK from responder", but was previously
applied to cases where no originator SYN exists, like a single RST-only
packet.
2020-09-11 16:14:41 -07:00
Jon Siwek
05cf511f18 GH-1119: add base/protcols/conn/removal-hooks.zeek
This adds two new functions: `Conn::register_removal_hook()` and
`Conn::unregister_removal_hook()` for registering a hook function to be
called back during `connection_state_remove`.  The benefit of using hook
callback approach is better scalability: the overhead of unrelated
protocols having to dispatch no-op `connection_state_remove` handlers is
avoided.
2020-09-11 12:12:10 -07:00
Jon Siwek
49e2047da0 Avoid run-time cost of find-filtered-trace.zeek for live traffic 2020-09-10 12:53:47 -07:00