Commit graph

3297 commits

Author SHA1 Message Date
Jan Grashoefer
aa4506ad06 Add skip analyzer test and update baselines. 2020-09-23 11:13:28 -07:00
Jan Grashoefer
fc814bd7e2 Add SkipAnalyzer.
This is WIP: The test case would require a new pcap or the possibility
to overwrite analyzer mappings. The CustomEncapsulationSkip method and
the corresponding options need to be removed.
2020-09-23 11:13:28 -07:00
Jan Grashoefer
6365fa6d80 Migrate all packet analyzers to new API. 2020-09-23 11:13:28 -07:00
Jan Grashoefer
96d0e11bb8 Move cur_pos from packet into packet manager loop. 2020-09-23 11:13:28 -07:00
Tim Wojtulewicz
c2500d03d6 Remove packet_analysis/Defines.h
- Replace uses of identifier_t with uint32_t
- Replace repeated usage of tuple type for Analysis results with type alias
2020-09-23 11:13:28 -07:00
Tim Wojtulewicz
bd6d3e0112 Remove enabled state from Components, ability to enable/disable from Manager 2020-09-23 11:13:28 -07:00
Tim Wojtulewicz
f39d6bb4c4 Use shared_ptr instead of raw pointers in packet_analysis for analyzers and dispatchers 2020-09-23 11:13:28 -07:00
Tim Wojtulewicz
08ceea8de1 Fixes for various btest issues
- Fix handling of truncated ethernet headers, fix core.truncation test output
- Update commit hashes for external private test repo
2020-09-23 11:13:28 -07:00
Jan Grashoefer
1e0e8e35af Minor fixes for packet analyzer renaming. 2020-09-23 11:13:28 -07:00
Jan Grashoefer
e53ec46c23 Renamed LL-Analyzers to Packet Analyzers. 2020-09-23 11:13:28 -07:00
Peter Oettig
b2e6c9ac9a Initial implementation of Lower-Level analyzers 2020-09-23 11:13:25 -07:00
Jon Siwek
01d83c3204 Avoid race condition in control/id_value btest 2020-09-22 14:11:17 -07:00
Jon Siwek
e1bfb55736 Update external test suite commit pointers 2020-09-22 12:56:05 -07:00
Jon Siwek
75b3ff7da4 Merge remote-tracking branch 'origin/topic/jsiwek/gh-927-when-condition-eval-order' into master
* origin/topic/jsiwek/gh-927-when-condition-eval-order:
  GH-927: Fix circumvention of evaluation order in 'when' conditions
2020-09-22 11:09:51 -07:00
Tim Wojtulewicz
6b93020c6f Merge remote-tracking branch 'origin/topic/jsiwek/gh-1164-fix-rst-conn-state'
* origin/topic/jsiwek/gh-1164-fix-rst-conn-state:
  GH-1164: Fix incorrect RSTOS0 conn_state determinations
2020-09-17 13:40:06 -07:00
Tim Wojtulewicz
3b641870dc Merge remote-tracking branch 'origin/topic/jsiwek/gh-1119-conn-removal-callbacks'
* origin/topic/jsiwek/gh-1119-conn-removal-callbacks:
  GH-1119: add base/protcols/conn/removal-hooks.zeek
  Avoid run-time cost of find-filtered-trace.zeek for live traffic
  Remove connection_successful and successful_connection_remove events
2020-09-17 13:16:19 -07:00
Tim Wojtulewicz
26808ea7d4 Merge remote-tracking branch 'origin/topic/jsiwek/gh-1155-recursive-table-index-type-check'
* origin/topic/jsiwek/gh-1155-recursive-table-index-type-check:
  GH-1155: Recursively check table index for unsupported types
2020-09-17 11:42:57 -07:00
Jon Siwek
33ca675515 GH-927: Fix circumvention of evaluation order in 'when' conditions
Historically, a 'when' condition performed an AST-traversal to locate
any index-expressions like `x[9]` and evaluated them so that it could
register the associated value as something for which it needs to receive
"modification" notifications.

Evaluating arbitrary expressions during an AST-traversal like that ignores
the typical order-of-evaluation/short-circuiting you'd expect if the
condition was evaluated normally, from its root expression.

Now, a new subclass of IndexExpr is used to keep track of all IndexExpr
results in the context of evaluating a 'when' condition without having
to do a secondary AST-traversal-and-eval.  i.e. the first evaluation of
the full 'when' condition follows the typical expression-evaluation
semantics (as always), but additionally now captures all the values
a Trigger needs to monitor for modifications.
2020-09-15 17:03:44 -07:00
Jon Siwek
a96440ea79 GH-1155: Recursively check table index for unsupported types
Previously, container types used within a table/set index were not
deeply checked to ensure all constituents could be part of an index.
2020-09-14 19:06:11 -07:00
Jon Siwek
3b334bad56 GH-1159: Fix vector-of-interval multiplication/division arithmetic
Those operations done between a vector-of-interval and a
vector-of-arithmetic-type previously threw a runtime expression error
due to an incorrect coercion being used internally.
2020-09-14 16:12:20 -07:00
Jon Siwek
1046e45dba Improve notice/suppression-cluster btest to avoid race potential 2020-09-14 12:28:25 -07:00
Jon Siwek
a16bd47bf7 GH-1164: Fix incorrect RSTOS0 conn_state determinations
The RSTOS0 `conn_state` label is documented as "Originator sent a SYN
followed by a RST, never saw SYN-ACK from responder", but was previously
applied to cases where no originator SYN exists, like a single RST-only
packet.
2020-09-11 16:14:41 -07:00
Jon Siwek
05cf511f18 GH-1119: add base/protcols/conn/removal-hooks.zeek
This adds two new functions: `Conn::register_removal_hook()` and
`Conn::unregister_removal_hook()` for registering a hook function to be
called back during `connection_state_remove`.  The benefit of using hook
callback approach is better scalability: the overhead of unrelated
protocols having to dispatch no-op `connection_state_remove` handlers is
avoided.
2020-09-11 12:12:10 -07:00
Jon Siwek
49e2047da0 Avoid run-time cost of find-filtered-trace.zeek for live traffic 2020-09-10 12:53:47 -07:00
Jon Siwek
5f435c2644 Remove connection_successful and successful_connection_remove events
Related to https://github.com/zeek/zeek/issues/1119
2020-09-10 12:06:50 -07:00
Jon Siwek
260895fcb1 Merge remote-tracking branch 'origin/topic/robin/gh-343-http-sigs'
* origin/topic/robin/gh-343-http-sigs:
  Change HTTP's DPD signatures so that each side can trigger the analyzer on its own.
2020-09-08 17:24:29 -07:00
Jon Siwek
991bbc961d Merge remote-tracking branch 'origin/topic/robin/gh-623-sampling'
- Merge adjustments:
  - Minor whitespace/style tweaks
  - Fixed portability of the btest due to differences in `uniq -c`
    output format

* origin/topic/robin/gh-623-sampling:
  Extend weird sampling with option to track selected weirds globally.
2020-09-08 17:06:40 -07:00
Robin Sommer
0af57d12b2 Change HTTP's DPD signatures so that each side can trigger the analyzer on its own.
This is to avoid missing large sessions where a single side exceeds
the DPD buffer size. It comes with the trade-off that now the analyzer
can be triggered by anybody controlling one of the endpoints (instead
of both).

Test suite changes are minor, and nothing in "external".

Closes #343.
2020-09-08 07:33:36 +00:00
Robin Sommer
5fa94973da Extend weird sampling with option to track selected weirds globally.
The new set "sampling_global_list" lists weirds to rate-limite
globally instead of per connection/flow.

Closes #623.
2020-09-07 07:31:22 +00:00
Jon Siwek
2a8de33c63 Merge remote-tracking branch 'origin/topic/jsiwek/gh-1122'
* origin/topic/jsiwek/gh-1122:
  GH-165: Fix global initializations that indirectly use builtin types
  Improve how primary/top-level BIFs get initialized
  GH-1122: Allow initializing globals with calls to subdir BIFs
  GH-1122: Improve error for global record initialization exceptions
2020-09-04 17:24:50 -07:00
Johanna Amann
425b6104c5 Merge remote-tracking branch 'origin/topic/jsiwek/gh-955-prohibit-typecasting-switch-fallthrough'
* origin/topic/jsiwek/gh-955-prohibit-typecasting-switch-fallthrough:
  GH-955: Prohibit `fallthrough` in typecasting `switch` cases

Closes GH-955
2020-09-02 11:49:28 -07:00
Jon Siwek
a7b905d389 GH-1149: Add GitHub Action to automate generation of zeek-docs 2020-09-01 17:52:00 -07:00
Jon Siwek
5e6bb843ab Merge remote-tracking branch 'origin/topic/timw/174-duplicate-attributes'
- Replaced "tag" with "attribute" in the error message since the former
  is not exactly the same concept/meaning and also not user-facing
  terminology

* origin/topic/timw/174-duplicate-attributes:
  Allow duplicate attributes in full redefs
  Short-circuit checking of whether attr exists
  Expanded check for other tag types, fixed btest to cover more tags
  GH-174: Add warning for duplicate attributes
2020-08-31 17:39:47 -07:00
Jon Siwek
5ab20dfcb9 Simplify a broker btest
The explicit sorting function definition was a temporary stability
workaround with a better fix now being available via
https://github.com/zeek/btest/pull/25
2020-08-31 15:01:23 -07:00
Johanna Amann
b8a47de26c Merge remote-tracking branch 'origin/topic/jsiwek/gh-926-ternary-type-checking'
* origin/topic/jsiwek/gh-926-ternary-type-checking:
  GH-926: Improve type-checking for ternary conditional operator

Fixes GH-926
2020-08-28 16:52:29 -07:00
Jon Siwek
dc9e175a61 Potentially improve intel/remove-item-cluster btest 2020-08-27 18:01:33 -07:00
Tim Wojtulewicz
36e3ab7177 Expanded check for other tag types, fixed btest to cover more tags 2020-08-27 15:17:38 -07:00
Jon Siwek
5ed13284c3 GH-165: Fix global initializations that indirectly use builtin types 2020-08-27 14:01:08 -07:00
Jon Siwek
4a8640d702 Improve how primary/top-level BIFs get initialized 2020-08-27 13:54:43 -07:00
Jon Siwek
1bbae2368d GH-1122: Allow initializing globals with calls to subdir BIFs 2020-08-27 12:20:37 -07:00
Jon Siwek
cf06ade325 GH-1122: Improve error for global record initialization exceptions 2020-08-26 14:24:36 -07:00
Jon Siwek
3368958ac1 GH-955: Prohibit fallthrough in typecasting switch cases 2020-08-26 12:55:10 -07:00
Jon Siwek
427a7de411 Merge remote-tracking branch 'origin/topic/timw/266-namespaces-part5'
- Did a few whitespace re-adjustments during merge

* origin/topic/timw/266-namespaces-part5:
  Update plugin btests for namespace changes
  Plugins: Clean up explicit uses of namespaces in places where they're not necessary.
  Base: Clean up explicit uses of namespaces in places where they're not necessary.
2020-08-25 19:51:42 -07:00
Jon Siwek
99d9a3a48c Fix closing timestamp of rotated log files in supervised-cluster mode 2020-08-25 17:06:10 -07:00
Jon Siwek
8d3c07b62a Improve an Intel framework btest
Relying on non-atomic `cp` operation may not cooperate well with the
Input framework file re-reading logic.
2020-08-25 14:36:52 -07:00
Tim Wojtulewicz
8012914c16 Merge remote-tracking branch 'origin/topic/robin/gh-574-processing-status'
* origin/topic/robin/gh-574-processing-status:
  Make set_processing_status() signal-safe.
2020-08-24 14:44:06 -07:00
Tim Wojtulewicz
b687730a01 Merge remote-tracking branch 'origin/topic/jsiwek/gh-710-expire-func-type-check'
* origin/topic/jsiwek/gh-710-expire-func-type-check:
  GH-710: Check that &expire_func is a function and not hook/event
2020-08-24 14:30:58 -07:00
Tim Wojtulewicz
60443e3178 Merge remote-tracking branch 'origin/topic/jsiwek/gh-594-improve-table-init-type-checking'
* origin/topic/jsiwek/gh-594-improve-table-init-type-checking:
  GH-594: Improve table initialization type-check error messages
2020-08-24 14:27:52 -07:00
Tim Wojtulewicz
989531826f GH-174: Add warning for duplicate attributes 2020-08-24 19:44:51 +00:00
Tim Wojtulewicz
874e170341 Update plugin btests for namespace changes 2020-08-24 12:07:03 -07:00