Commit graph

3472 commits

Author SHA1 Message Date
Jon Siwek
6ac499882c Teach Zeekygen to produce source-code-range information
Related to https://github.com/zeek/zeek-docs/issues/56
2021-02-26 14:55:59 -08:00
Jon Siwek
737d2c390b Support explicit disabling of file analyzers 2021-02-23 15:50:18 -08:00
Jon Siwek
80221cb958 Merge remote-tracking branch 'origin/topic/vern/use-defs'
- Added explicit test case of "unused assignement" warning and
  &is_used suppression during merge.

* origin/topic/vern/use-defs:
  activate &is_used
  removed unnecessary statement flagged by Coverity (thanks, Jon!)
  Adjust reference/move nitpicks in use-def/reduce code
  Adjust some whitespace in UseDefs.cc
  updates to "usage" test suite alternative now that more warnings are generated
  suppress usage warning in baseline script
  "xform" alternative baseline update, needed for recent change to master
  baseline update due to shift in number of lines in base intel framework script
  environment variable (which has precedence) not flags for baseline usage test
  splitting out "usage" test suite alternative into -u/-uu versions
  adding &is_used attribute for base scripts - not actually needed yet, but will be once optimization is added
  removing unused assignments from base scripts
  driver glue for invoking use-def construction
  classes for managing and propagating use-defs
  enhancements/changes to the Reduce class in preparation for use-defs
  some tidying with smart pointers
  flag/environment variable for dumping use-defs
  &is_used attribute to suppress set-but-not-used usage warnings
  whitespace micro-preening
2021-02-23 12:11:24 -08:00
Vern Paxson
86709c5992 activate &is_used 2021-02-23 08:32:14 -08:00
Johanna Amann
6c554ddde3 Ascii reader test with 0-bytes
Add a second test, which contains actual 0-bytes.
2021-02-12 14:49:33 +00:00
Johanna Amann
61290fc19c Fix buffer overread in ascii formatter
When a text with an (escaped) zero byte was passed to ParseValue, only
the part of the string up to the zero byte was copied, but the length of
the full string was passed to the input framework.

This leads to the input manager reading over the end of the buffer.

Fixes zeek/zeek#1398
2021-02-12 14:24:36 +00:00
Tim Wojtulewicz
a05356eb79 Add a script to cleanup the file listing before sending to Coveralls
GCC has a "feature" where it doesn't realize that files in the
src/zeek/... tree are the same as files in the normal 'src/' tree. This
leads the coverage script to send duplicates to Coveralls and pollute
the display with them. The new script scrapes the intermediate output
from lcov and de-duplicates anything in src/zeek/ to be just from src/.
2021-02-09 23:07:47 +00:00
Vern Paxson
b714fde456 updates to "usage" test suite alternative now that more warnings are generated 2021-02-06 11:07:26 -08:00
Vern Paxson
1605b57dec suppress usage warning in baseline script 2021-02-06 11:05:53 -08:00
Vern Paxson
991bd51430 "xform" alternative baseline update, needed for recent change to master 2021-02-06 11:04:53 -08:00
Vern Paxson
6838ef66b4 baseline update due to shift in number of lines in base intel framework script 2021-02-06 11:04:17 -08:00
Vern Paxson
da40c580b2 environment variable (which has precedence) not flags for baseline usage test 2021-02-06 11:01:30 -08:00
Vern Paxson
0a6270365f splitting out "usage" test suite alternative into -u/-uu versions 2021-02-06 11:01:04 -08:00
Jon Siwek
ac0b09d1a4 Allow --parse-only to work with --usage-issues flag
Use of --parse-only would previously exit before --usage-issues had a
chance to analyze scripts and report any discovered issues and it can be
useful to combine both flags for checking scripts for mistakes without
actually executing any code.

This also improves the behavior of --parse-only in combination with
reporting problems in signature files (previously, it exited before
even reading them).
2021-02-05 14:46:47 -08:00
Jon Siwek
03f74958f3 Merge remote-tracking branch 'origin/topic/vern/reaching-defs'
* origin/topic/vern/reaching-defs: (36 commits)
  added &is_assigned test case for variable rather than record field
  Speedup ReachingDefs logic by ~15%
  Simplify ReachingDefs::RDMap() accessor
  test for -uu correctly tracking $?, and not misled by conditional assignments
  &is_set => &is_assigned
  remove pending maybe-reconsider-this comment
  fixes for ?$ operator - always track it, and assume subrecords are initialized
  speedup (and more coherent memory management) for tracking RDs
  fixes for generating and evaluating RDs associate with ?$ expressions
  fix for failure to reduce InlineExpr's to CatchReturnStmt's
  inlining fix: propagate identifier attributes (such as &is_set)
  tidier memory management
  fix for an ancient bug - surprising that this hasn't caused problems previously
  Fix IntrusivePtr release leaks in reaching-def logic
  Change dynamic_cast in reaching-def logic to static_cast
  Adjust some reaching-def memory management
  Update a couple baselines for "xform" alternative
  Adjust various reaching-def move/reference semantics
  Change LambdaExpr::OuterIDs() accessor to return const-reference
  Simplify declaration of DefPointType enum
  ...
2021-02-05 11:02:01 -08:00
Vern Paxson
7f9ee3d028 added &is_assigned test case for variable rather than record field 2021-02-05 08:22:19 -08:00
Vern Paxson
5f79cb7668 test for -uu correctly tracking $?, and not misled by conditional assignments 2021-02-04 12:58:23 -08:00
Vern Paxson
c991c54690 &is_set => &is_assigned 2021-02-04 12:18:46 -08:00
Jon Siwek
c27bf62217 Merge remote-tracking branch 'origin/topic/timw/1389-vntag'
Merge adjustment: changed test case to use `zeek -b`

* origin/topic/timw/1389-vntag:
  GH-1389: Skip VN-Tag headers
2021-02-03 11:22:13 -08:00
Jon Siwek
c44cbe1feb Prefix #includes of .bif.h files with zeek/
This enables locating the headers within the install-tree using the
dirs provided by `zeek-config --include_dir`.

To enable locating these headers within the build-tree, this change also
creates a 'build/src/include/zeek -> ..' symlink.
2021-02-02 19:15:05 -08:00
Jon Siwek
01f194edbe Update a couple baselines for "xform" alternative 2021-02-01 22:07:32 -08:00
Jon Siwek
cfe29c2488 Merge branch 'master' into topic/vern/reaching-defs 2021-02-01 17:27:37 -08:00
Tim Wojtulewicz
f53448ccc9 GH-1389: Skip VN-Tag headers 2021-02-01 14:34:56 -07:00
Jon Siwek
1ca85f0221 Merge remote-tracking branch 'origin/topic/timw/deprecation-cleanup'
Merge adjustments:
- Removed some stale str_split() references from docs
- Renumbered TypeTag enum comments
- Simplified test-case for @unload (don't need .bro files anymore)

* origin/topic/timw/deprecation-cleanup:
  Doc updates
  Fix language.init-in-anon-function btest due to changes to log filter predicates
  Remove deprecated log filter predicates for 4.1
  Remove Plugin::HookCallFunction and fix tests related to it
  Remove support for .bro script extension and BRO_ environment variables
  Remove deprecated ICMP events
  Remove some deprected methods/events from bif files
  Remove TYPE_COUNTER
  Remove all of the random single-file deprecations
  Remove all fully-deprecated files
  Update bifcl submodule to remove deprecations from generated code
2021-01-29 16:40:54 -08:00
Christian Kreibich
38d6b6a98b Skip negative integers in input framework where not type-permissible
Script-layer counts, when provided as negative integers in an input
file, got cast to unsigned values because strtoull() does not complain
about negative values. For example, input string "-1" would lead to
value 18446744073709551615 (an all-ones 64-bit int) on x86_64. This is
more likely to be an error than an intent to get very large,
platform-dependent values, so these input lines are now skipped with
according messaging in the reporter.log/stderr.

This also affected ports: -1/tcp got cast to unsigned and only thrown
out because PortVal rejects values > 65535, mapping them to 0. We now
skip such inputs as well.

Updates existing input framework tests to capture the new behavior.
2021-01-28 23:46:43 -08:00
Tim Wojtulewicz
00868e00d3 Fix language.init-in-anon-function btest due to changes to log filter predicates 2021-01-27 10:52:41 -07:00
Christian Kreibich
4ce3bf3cd2 Remove deprecated log filter predicates for 4.1
Update the logging framework tests: since hooks operate
by name, they cannot be anonymous. I'm also dropping the &optional
attribute from the status field, since here know that the values are
actually defined, and access to an optional status field should
normally be guarded by the existence test operator.

Also includes baseline update for plugins.hooks, which picks up the
fact that the pred record field is now gone.
2021-01-27 10:52:40 -07:00
Tim Wojtulewicz
5f09793ce1 Remove Plugin::HookCallFunction and fix tests related to it 2021-01-27 10:52:40 -07:00
Tim Wojtulewicz
725e759560 Remove support for .bro script extension and BRO_ environment variables 2021-01-27 10:52:40 -07:00
Tim Wojtulewicz
44ad614094 Remove deprecated ICMP events 2021-01-27 10:52:40 -07:00
Tim Wojtulewicz
7292b52f66 Remove some deprected methods/events from bif files 2021-01-27 10:52:40 -07:00
Tim Wojtulewicz
b819c9d18c Remove TYPE_COUNTER
This causes some test changes because of the use of Type::Describe in the fnv1a32
BIF method called by hash_hrw::get_site. Since the enum values change with removal
of TYPE_COUNTER, the output from Type::Describe changes slightly.
2021-01-27 10:52:40 -07:00
Tim Wojtulewicz
0618be792f Remove all of the random single-file deprecations
These are the changes that don't require a ton of changes to other files outside
of the original removal.
2021-01-27 10:52:40 -07:00
Tim Wojtulewicz
7e9d89db0a Remove all fully-deprecated files 2021-01-27 10:52:40 -07:00
Vern Paxson
55c5ce06d1 updates to "xform" alternative reflecting recent tweak in main Baseline 2021-01-23 11:28:24 -08:00
Vern Paxson
c43925b7ee updates to test suite, including new baseline for "-a usage" environment 2021-01-23 10:57:08 -08:00
Jon Siwek
efba91cd26 Merge branch 'topic/oakljon/gh-1352-smtp-header-parsing' of https://github.com/theavgjojo/zeek
* 'topic/oakljon/gh-1352-smtp-header-parsing' of https://github.com/theavgjojo/zeek:
  GH-1352: Added flag to stop processing SMTP headers in attached messages
2021-01-21 17:01:13 -08:00
Jon Siwek
7f3f5a868f Merge remote-tracking branch 'origin/topic/jsiwek/gh-1296-fix-list-type-checks'
* origin/topic/jsiwek/gh-1296-fix-list-type-checks:
  GH-1296: fix type-checks related to list-type equality
2021-01-21 14:00:07 -08:00
Jon Oakley
25de6f28e5 GH-1352: Added flag to stop processing SMTP headers in attached
messages
2021-01-21 14:55:10 -05:00
Johanna Amann
079d4164c0 Merge remote-tracking branch 'origin/topic/christian/fix-config-segfault'
* origin/topic/christian/fix-config-segfault:
  Btest tweak for improved type rendering in config framework errors and set types
  More precise type information in a config framework error message
  Explicitly don't support sets with multiple index types in input/config frameworks
2021-01-21 11:30:36 +00:00
Jon Siwek
ee4c259cd4 Merge branch 'master' into topic/vern/script-xform 2021-01-13 12:18:44 -08:00
Christian Kreibich
fcab8df8fb Btest tweak for improved type rendering in config framework errors and set types 2021-01-11 13:54:13 -08:00
Jon Siwek
614fade0a4 Merge branch 'master' into topic/vern/lambda-copy-semantics 2021-01-11 11:48:45 -08:00
Jon Siwek
8c64ba6907 GH-1296: fix type-checks related to list-type equality
List-types as used in composite table/set indices, for example,
previously had incorrect same_type() comparisons due to flattening
of the list-type into a single type without checking whether the
number and kind of types all match.

This patch simply removes the flatten_type() call from same_type() since
it was already contradicting/preventing a subsequent full-comparison
between elements of two TYPE_LISTs.

There was also a superfluous special-case of the `in` operator's
type-checking for testing whether a record is in a table/set.  It's
superfluous because the general case will already do the type-checking
from MatchesIndex() after first wrapping the record operand in a
ListExpr.  The previous logic was incorrectly relying on the
flatten_type() for testing equality of a record-type against a
list-type, whereas the general case correctly normalizes to testing
equality of two list-types.

The special-cased type-checking logic for assigning a record value to a
table index during its initialization similarly needed minor
re-organization in order to maintain the same error messages as before.
2021-01-10 17:32:50 -08:00
Vern Paxson
dc38880cee canonicalization for an error message in one of the alternative test baselines 2021-01-10 14:44:46 -08:00
Vern Paxson
06f0a1b3df baseline differences for "xform" alternative: changes generally reflect exposure of transformed code, or error propagation stopping earlier due to error now occurring in an assignment (to a temporary) 2021-01-10 14:25:46 -08:00
Vern Paxson
0e4c996718 new testing alternative for script transformation (= xform) 2021-01-10 14:24:18 -08:00
Vern Paxson
117ff4eb08 bug in correctly inspecting test output file 2021-01-10 14:23:42 -08:00
Vern Paxson
bd2c94521a split bifs.string_utils into a non-error test and an only-errors test, to help control for differing error propagation 2021-01-10 14:21:51 -08:00
Vern Paxson
eb1848c547 cleaner approach for localizing errors associated with duplicated ASTs: virtualize GetLocationInfo 2021-01-07 15:14:22 -08:00