Commit graph

6667 commits

Author SHA1 Message Date
Peter Oettig
b2e6c9ac9a Initial implementation of Lower-Level analyzers 2020-09-23 11:13:25 -07:00
Jon Siwek
75b3ff7da4 Merge remote-tracking branch 'origin/topic/jsiwek/gh-927-when-condition-eval-order' into master
* origin/topic/jsiwek/gh-927-when-condition-eval-order:
  GH-927: Fix circumvention of evaluation order in 'when' conditions
2020-09-22 11:09:51 -07:00
Jon Siwek
428870be15 Fix negative-value-left-shift undefined behavior in patricia trie 2020-09-18 11:32:05 -07:00
Jon Siwek
a44b056e81 Improve negation of ConstExpr
* Instead of creating a NegExpr for negation of a literal/constant,
  a ConstExpr is now created directly.

* For negation of integer literals, there's now an additional check
  for whether the integer would be outside the range of possible 'int'
  values.  This can also help prevent the undefined behavior due to
  overflow as a result of trying to represent the minimum 'int' value of
  -9223372036854775808 as a literal in a script -- the unsigned value is
  cast to signed yielding INT64_MIN, then INT64_MIN is negated.
2020-09-18 11:32:05 -07:00
Jon Siwek
1b88e63e78 Avoid signed integer overflow when combining SMB header PID bits
Such an overflow invokes undefined behavior.
2020-09-18 11:32:05 -07:00
Jon Siwek
17e3392052 Avoid unary negation of INT64_MIN in modp_litoa10
Overlow can occur in that case, which is undefined behavior.
2020-09-18 11:32:05 -07:00
Jon Siwek
d25ead8f8e Avoid double-to-int conversion overflows in modp_dtoa functions
Those methods already had a fallback to use sprintf() for large values
except:

* The check-for-large-value was unnecessarily done after many
  operations that aren't relevant to the check and those operations can
  result in a conversion overflow (undefined behavior).

* The check-for-large-value was using the literal value for a
  32-bit INT_MAX instead of just using INT_MAX.  For a platform where
  `int` is less than 32-bits, the same conversion overflow from the
  previous point could still occur (undefined behavior).

* The check-for-large-value was not inclusive of INT_MAX.
  In a case where the conversion of INT_MAX itself to a double
  can't be represented exactly, it's implementation-defined whether
  the closest higher or closest lower representable-value is selected.
  If the higher value is selected, then a `double` value comparing equal
  to INT_MAX-as-converted-to-double would cause an overflow of an `int`
  upon conversion (undefined behavior).
2020-09-18 11:32:05 -07:00
Jon Siwek
264e6858f2 Fix divide-by-zero in Entropy analyzer 2020-09-18 11:32:05 -07:00
Jon Siwek
091424b2ad Fix divide-by-zero in stats/profiling memory usage calculation 2020-09-18 11:32:05 -07:00
Jon Siwek
5d33775430 Fix uninitialized field in POP3 fuzzer 2020-09-18 11:32:05 -07:00
Tim Wojtulewicz
3b641870dc Merge remote-tracking branch 'origin/topic/jsiwek/gh-1119-conn-removal-callbacks'
* origin/topic/jsiwek/gh-1119-conn-removal-callbacks:
  GH-1119: add base/protcols/conn/removal-hooks.zeek
  Avoid run-time cost of find-filtered-trace.zeek for live traffic
  Remove connection_successful and successful_connection_remove events
2020-09-17 13:16:19 -07:00
Tim Wojtulewicz
26808ea7d4 Merge remote-tracking branch 'origin/topic/jsiwek/gh-1155-recursive-table-index-type-check'
* origin/topic/jsiwek/gh-1155-recursive-table-index-type-check:
  GH-1155: Recursively check table index for unsupported types
2020-09-17 11:42:57 -07:00
Tim Wojtulewicz
c7445a7d43 Merge remote-tracking branch 'origin/topic/jsiwek/gh-1159-interval-vector-arithmetic'
* origin/topic/jsiwek/gh-1159-interval-vector-arithmetic:
  GH-1159: Fix vector-of-interval multiplication/division arithmetic
2020-09-17 11:35:02 -07:00
Tim Wojtulewicz
0cb39a5c31 Merge remote-tracking branch 'origin/topic/christian/1074-avoid-packed-member-warning'
* origin/topic/christian/1074-avoid-packed-member-warning:
  Avoid passing address of member in packed struct #1074
2020-09-17 11:25:15 -07:00
Jon Siwek
33ca675515 GH-927: Fix circumvention of evaluation order in 'when' conditions
Historically, a 'when' condition performed an AST-traversal to locate
any index-expressions like `x[9]` and evaluated them so that it could
register the associated value as something for which it needs to receive
"modification" notifications.

Evaluating arbitrary expressions during an AST-traversal like that ignores
the typical order-of-evaluation/short-circuiting you'd expect if the
condition was evaluated normally, from its root expression.

Now, a new subclass of IndexExpr is used to keep track of all IndexExpr
results in the context of evaluating a 'when' condition without having
to do a secondary AST-traversal-and-eval.  i.e. the first evaluation of
the full 'when' condition follows the typical expression-evaluation
semantics (as always), but additionally now captures all the values
a Trigger needs to monitor for modifications.
2020-09-15 17:03:44 -07:00
Jon Siwek
a96440ea79 GH-1155: Recursively check table index for unsupported types
Previously, container types used within a table/set index were not
deeply checked to ensure all constituents could be part of an index.
2020-09-14 19:06:11 -07:00
Jon Siwek
3b334bad56 GH-1159: Fix vector-of-interval multiplication/division arithmetic
Those operations done between a vector-of-interval and a
vector-of-arithmetic-type previously threw a runtime expression error
due to an incorrect coercion being used internally.
2020-09-14 16:12:20 -07:00
Jon Siwek
5f435c2644 Remove connection_successful and successful_connection_remove events
Related to https://github.com/zeek/zeek/issues/1119
2020-09-10 12:06:50 -07:00
Jon Siwek
b620ace06b Merge remote-tracking branch 'origin/topic/jsiwek/reporter-weird-sampling-api-tweaks'
* origin/topic/jsiwek/reporter-weird-sampling-api-tweaks:
  Improve Reporter weird-sampling-whitelist getters/setters
2020-09-09 10:47:16 -07:00
Jon Siwek
6102e09ada Fix memory leak caused by re-entering GTPv1 parsing
If the inner packet of a decapsulated GTPv1 frame maps to the same
Connection as the outer packet and also contains another GTPv1
encapsulation, the same GTPv1 analyzer attempts to parse the inner
packet.  In the process, the inner parsing method overwrites pointers to
memory resources the outer parsing method has not yet released and
results in a memory leak.
2020-09-09 10:44:33 -07:00
Jon Siwek
145b4f9dd0 Fix memory leak caused by re-entering AYIYA parsing
If the inner packet of a decapsulated AYIYA frame maps to the same
Connection as the outer packet and also contains another AYIYA
encapsulation, the same AYIYA analyzer attempts to parse the inner
packet.  In the process, the inner parsing method overwrites pointers to
memory resources the outer parsing method has not yet released and
results in a memory leak.

Credit to OSS-Fuzz for discovery
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=25256
2020-09-09 10:43:47 -07:00
Jon Siwek
e37baf09c0 Improve Reporter weird-sampling-whitelist getters/setters
- getter methods return const-ref
- setter methods pass by value and std::move()
- ranged-for loops over the whitelists access by const-ref
2020-09-08 18:04:54 -07:00
Jon Siwek
991bbc961d Merge remote-tracking branch 'origin/topic/robin/gh-623-sampling'
- Merge adjustments:
  - Minor whitespace/style tweaks
  - Fixed portability of the btest due to differences in `uniq -c`
    output format

* origin/topic/robin/gh-623-sampling:
  Extend weird sampling with option to track selected weirds globally.
2020-09-08 17:06:40 -07:00
Robin Sommer
5fa94973da Extend weird sampling with option to track selected weirds globally.
The new set "sampling_global_list" lists weirds to rate-limite
globally instead of per connection/flow.

Closes #623.
2020-09-07 07:31:22 +00:00
Jon Siwek
90f04c3d0a Merge remote-tracking branch 'origin/topic/jsiwek/exclude-installing-zeek-symlink'
* origin/topic/jsiwek/exclude-installing-zeek-symlink:
  Exclude installing "zeek -> ." include dir symlink
2020-09-04 18:30:38 -07:00
Jon Siwek
2a8de33c63 Merge remote-tracking branch 'origin/topic/jsiwek/gh-1122'
* origin/topic/jsiwek/gh-1122:
  GH-165: Fix global initializations that indirectly use builtin types
  Improve how primary/top-level BIFs get initialized
  GH-1122: Allow initializing globals with calls to subdir BIFs
  GH-1122: Improve error for global record initialization exceptions
2020-09-04 17:24:50 -07:00
Jon Siwek
bc3df06737 Exclude installing "zeek -> ." include dir symlink
The symlink only exists for use within the source-tree and isn't needed
for the install-tree.
2020-09-04 16:41:46 -07:00
Christian Kreibich
bdd624d8b8 Avoid passing address of member in packed struct #1074
This appeases -Waddress-of-packed-member warnings in some
compiler/platform combinations via use of local variables.
2020-09-02 16:04:26 -07:00
Johanna Amann
425b6104c5 Merge remote-tracking branch 'origin/topic/jsiwek/gh-955-prohibit-typecasting-switch-fallthrough'
* origin/topic/jsiwek/gh-955-prohibit-typecasting-switch-fallthrough:
  GH-955: Prohibit `fallthrough` in typecasting `switch` cases

Closes GH-955
2020-09-02 11:49:28 -07:00
Jon Siwek
d99c5aaf7d Improve documentation for rfind_str() 2020-09-01 17:54:44 -07:00
Jon Siwek
f6e48c3a18 Reformat strings.bif Zeekygen comments to fix Sphinx warnings 2020-09-01 11:58:52 -07:00
Jon Siwek
5e6bb843ab Merge remote-tracking branch 'origin/topic/timw/174-duplicate-attributes'
- Replaced "tag" with "attribute" in the error message since the former
  is not exactly the same concept/meaning and also not user-facing
  terminology

* origin/topic/timw/174-duplicate-attributes:
  Allow duplicate attributes in full redefs
  Short-circuit checking of whether attr exists
  Expanded check for other tag types, fixed btest to cover more tags
  GH-174: Add warning for duplicate attributes
2020-08-31 17:39:47 -07:00
Johanna Amann
b8a47de26c Merge remote-tracking branch 'origin/topic/jsiwek/gh-926-ternary-type-checking'
* origin/topic/jsiwek/gh-926-ternary-type-checking:
  GH-926: Improve type-checking for ternary conditional operator

Fixes GH-926
2020-08-28 16:52:29 -07:00
Johanna Amann
05f7e3fa43 Unbreak power architecture part 2
The compiler flag also refered the wrong file.
2020-08-28 12:04:52 -07:00
Johanna Amann
e8efab541b Unbreak build on power architectures.
Turns out I mistyped a filename when switching over to highwayhash.
Which went unnoticed since we do not really test against any power
architecure machines.
2020-08-28 11:40:05 -07:00
Tim Wojtulewicz
9106f3f722 Allow duplicate attributes in full redefs 2020-08-28 09:55:40 -07:00
Tim Wojtulewicz
9b2f26c0aa Short-circuit checking of whether attr exists 2020-08-28 09:37:36 -07:00
Jon Siwek
5b25870c83 Merge remote-tracking branch 'origin/topic/timw/158-packet-filter-option'
* origin/topic/timw/158-packet-filter-option:
  GH-158: Ensure packet-filter framework is loaded if -f option is passed
2020-08-27 15:34:31 -07:00
Tim Wojtulewicz
36e3ab7177 Expanded check for other tag types, fixed btest to cover more tags 2020-08-27 15:17:38 -07:00
Tim Wojtulewicz
a3d7284d5e GH-158: Ensure packet-filter framework is loaded if -f option is passed 2020-08-27 14:06:21 -07:00
Jon Siwek
5ed13284c3 GH-165: Fix global initializations that indirectly use builtin types 2020-08-27 14:01:08 -07:00
Jon Siwek
4a8640d702 Improve how primary/top-level BIFs get initialized 2020-08-27 13:54:43 -07:00
Jon Siwek
1bbae2368d GH-1122: Allow initializing globals with calls to subdir BIFs 2020-08-27 12:20:37 -07:00
Jon Siwek
cf63da7975 Merge remote-tracking branch 'origin/topic/vlad/gh-1145'
* origin/topic/vlad/gh-1145:
  Fix another umask issue. #1145

Fixes GH-1145
2020-08-27 11:15:00 -07:00
Vlad Grigorescu
c3a395a6f0 Fix another umask issue. #1145 2020-08-26 18:07:21 -05:00
Jon Siwek
cf06ade325 GH-1122: Improve error for global record initialization exceptions 2020-08-26 14:24:36 -07:00
Jon Siwek
3368958ac1 GH-955: Prohibit fallthrough in typecasting switch cases 2020-08-26 12:55:10 -07:00
Jon Siwek
ff0aa6b050 Merge remote-tracking branch 'origin/topic/timw/rename-binpac_bro'
* origin/topic/timw/rename-binpac_bro:
  Rename bro-bif.h to zeek-bif.h
  Fix uses of bro.pac to use zeek.pac
  Rename the bro-related binpac files to zeek
2020-08-26 11:21:17 -07:00
Vlad Grigorescu
e12db6bac0 Have mkdir in ensure_dir respect umask.
This also aligns with the mkdir bif. Fixes #1145
2020-08-26 10:01:20 -05:00
Jon Siwek
427a7de411 Merge remote-tracking branch 'origin/topic/timw/266-namespaces-part5'
- Did a few whitespace re-adjustments during merge

* origin/topic/timw/266-namespaces-part5:
  Update plugin btests for namespace changes
  Plugins: Clean up explicit uses of namespaces in places where they're not necessary.
  Base: Clean up explicit uses of namespaces in places where they're not necessary.
2020-08-25 19:51:42 -07:00