Commit graph

367 commits

Author SHA1 Message Date
Tim Wojtulewicz
67b45bc502 Merge remote-tracking branch 'origin/topic/vern/ZAM-prep'
* origin/topic/vern/ZAM-prep: (45 commits)
  whoops overlooked the need to canonicalize filenames
  another set of tweaks per review comments
  addressed a number of code review comments
  baseline updates for merge
  support "any" coercions for "-O gen-C++"
  better descriptions for named record constructors
  test suite baseline updates for "-a opt" optimize-AST alternative
  test suite baseline updates for "-a xform" alternative / AST transformation
  error propagation fix for AST reduction
  updates to "-a inline" test suite alternative baseline
  updates for the main test suite baseline
  updates to test suite tests for compatibility with upcoming ZAM functionality
  "-O compile-all" option to specify compilation of inlined functions
  compile inlined functions if they're also used indirectly
  provide ZAM-generated code with low-level access to record fields
  fix for cloning records with fields of type "any"
  direct access for ZAM to VectorVal internal vector
  ZVal constructors, accessors & methods in support of ZAM
  switch ZVal representation of types from Type objects to TypeVal's
  revised error-reporting interface for ZVal's, to accommodate ZAM inner loop
  ...
2021-06-28 11:03:13 -07:00
Christian Kreibich
0b55c55140 Remove unnecessary -B arguments from Zeek invocations in testsuite
Now that Zeek no longer silently accepts -B when not compiled in debug
mode, these tests were failing.
2021-06-24 17:05:32 -07:00
Vern Paxson
2acaacee2d whoops overlooked the need to canonicalize filenames 2021-06-09 15:29:39 -07:00
Vern Paxson
7904701ffb another set of tweaks per review comments 2021-06-09 09:41:41 -07:00
Vern Paxson
4172b9e147 Merge remote-tracking branch 'origin/master' into topic/vern/ZAM-prep 2021-06-03 08:13:48 -07:00
Vern Paxson
b6e9776a11 updates to test suite tests for compatibility with upcoming ZAM functionality 2021-06-01 09:25:30 -07:00
Jon Siwek
77f1ede661 GH-839: Fix use of &optional sub-records within table/set indices 2021-05-26 13:28:13 -07:00
Jon Siwek
48ee0f31a1 GH-1244: Change modp_dtoa2() to use scientific notation for small values
This fixes problems where printing floating point numbers less than
10^-6 output as "0.0".  Such numbers now use using scientific notation
and preserve the value's actual floating point representation.
2021-05-17 17:44:23 -07:00
Vern Paxson
f9c1f57f9c notes for tests that are problematic when using compile-to-C++ 2021-05-05 16:55:04 -07:00
Jon Siwek
13f3cb377f GH-1534: Fix excessive coredump for duplicate enum definitions
An adequate error message was previously reported for duplicate enum
definitions, this just now prevents trying to access it as a constant in
subsequent parsing and further generating a coredump.
2021-04-29 15:20:09 -07:00
Jon Siwek
486f187b15 GH-1483: Improve error for mismatched container initialization types
For example, trying to assign a vector, table, set, or record
constructor expression to a global variable of a different type now
provides a more explanatory error message than the previous
"Val::CONVERTER" fatal-error and coredump.
2021-04-20 19:14:13 -07:00
Jon Siwek
e8247c2472 Merge remote-tracking branch 'origin/topic/vern/vector-holes'
* origin/topic/vern/vector-holes:
  Remove NEWS entry regarding changed vector-holes functionality
  Fix potential segfaults in VectorVal Insert/Remove methods
  Fix copy() to work with a vector that has trailing holes
  update test suite for vector holes now being supported for numeric types
  add vector tests for creating holes, "in" operator, "?" operator, copying vectors with holes
  restore support for vectors with holes remove vestigial comment
  fix using ++/-- to vectors that contain holes
2021-04-20 14:43:01 -07:00
Jon Siwek
fd5cdbbe50 Fix potential segfaults in VectorVal Insert/Remove methods
The existence/hole check for managed types was needed to prevent
accessing a nil-optional value.
2021-04-19 19:29:37 -07:00
Jon Siwek
819fc1aac0 Fix copy() to work with a vector that has trailing holes
Previously, the trailing holes would not be preserved in the copy.
2021-04-19 19:23:04 -07:00
Jon Siwek
846cc53780 Merge remote-tracking branch 'origin/topic/jsiwek/fix-table-vector-index'
* origin/topic/jsiwek/fix-table-vector-index:
  Fix indexing of set/table types with a vector
2021-04-15 17:59:31 -07:00
Jon Siwek
df9b57118d Merge remote-tracking branch 'origin/topic/jsiwek/gh-1503-improve-expire-func-type-check'
* origin/topic/jsiwek/gh-1503-improve-expire-func-type-check:
  Extend language.expire-func-type-check btest
  GH-1503: Improve &expire_func compatibility type-checking
2021-04-15 17:42:33 -07:00
Jon Siwek
05b992b865 Extend language.expire-func-type-check btest
Now covers more forms of both valid and invalid &expire_funcs
2021-04-15 16:13:35 -07:00
Jon Siwek
735ac0b4fa Fix indexing of set/table types with a vector
Previous behavor:

  internal error: bad index type in CompositeHash::ComputeSingletonHash
  Aborted (core dumped)
2021-04-14 21:58:14 -07:00
Vern Paxson
e96b29d4d8 add vector tests for creating holes, "in" operator, "?" operator, copying vectors with holes 2021-04-14 18:19:13 -07:00
Jon Siwek
01f6264c87 GH-1503: Improve &expire_func compatibility type-checking
Previously, incompatible &expire_funcs could mistakenly be used, such as
when using that attribute on the unspecified table()/set()
initializations/assignments, resulting in invalid function calls that
eventually crash Zeek.
2021-04-12 16:27:50 -07:00
Jon Siwek
952d8de8fd Add btest for unterminated pattern parsing error behavior
And clarified the error message that it's more about finding an
unterminated pattern than knowing for sure there's remaining pattern
text spanning multiple lines.
2021-04-09 11:15:06 -07:00
Jon Siwek
312547ab0a GH-1497: Support CRLF line-endings in Zeek scripts and signature files 2021-04-08 20:32:30 -07:00
Jon Siwek
3e9ac00f21 Separate stdout from stderr in btest baselines
Redirecting both to the same file can show platform-specific differences
(e.g. Alpine), likely due to different buffering defaults.
2021-03-30 16:23:23 -07:00
Tim Wojtulewicz
f45df63cd0 Merge remote-tracking branch 'origin/topic/vern/zval'
* origin/topic/vern/zval: (42 commits)
  whitespace tweaks
  resolved some TODO comments
  remove unnecessary casts, and change necessary ones to use static_cast<>
  explain cmp_func default
  change functions for ZVal type management to static members
  fix some unsigned/signed integer warnings
  address lint concern about uninitialized variable
  Remove use of obsolete forward-declaration macros
  fix #include's that lack zeek/ prefixes
  explicitly populate holes created in vectors
  fixes for now-incorrect assumption that GetField always returns an existing ValPtr
  memory management for assignment to vector elements
  memory management for assignment to record fields
  destructor cleanup from ZAM_vector/ZAM_record
  fix #include's that lack zeek/ prefixes
  overlooked another way in which vector holes can be created
  initialize vector holes to the correct corresponding type
  explicitly populate holes created in vectors
  fix other instances of GetField().get() assuming long-lived ValPtr's
  fix for now-incorrect assumption that GetField always returns an existing ValPtr
  ...
2021-03-23 20:44:19 -07:00
Jon Siwek
f46d3dec8f Merge remote-tracking branch 'origin/topic/vern/cpp-new-func'
- Removed a couple of dead statements during merge

* origin/topic/vern/cpp-new-func:
  option for deterministic descriptions of sets & tables
  determinism for concurrent Zeek test suite invocations; split out deprecations
  disambiguate descriptions of enums; include attributes when describing records
  more liberal view of attribute equality; allow suppressing attr type-checking
  support for operations on sets that return new values
  low-level addition of enum values
  sundry accessors/cast-ers; RE_Matcher's track their construction values
  convenience functions for comparing IP addresses
2021-03-23 19:27:11 -07:00
Vern Paxson
a808e166a4 determinism for concurrent Zeek test suite invocations; split out deprecations 2021-03-18 16:17:25 -07:00
Vern Paxson
e21c0f1115 correcting some simple typos 2021-03-18 08:19:46 -07:00
Vern Paxson
5084584020 test suite update for Smith-Waterman now using 0-based vectors 2021-02-25 17:13:56 -08:00
Vern Paxson
e64805430b fix off-by-one assumption regarding vector indexing dating to 2011 2021-02-24 16:31:07 -08:00
Jon Siwek
80221cb958 Merge remote-tracking branch 'origin/topic/vern/use-defs'
- Added explicit test case of "unused assignement" warning and
  &is_used suppression during merge.

* origin/topic/vern/use-defs:
  activate &is_used
  removed unnecessary statement flagged by Coverity (thanks, Jon!)
  Adjust reference/move nitpicks in use-def/reduce code
  Adjust some whitespace in UseDefs.cc
  updates to "usage" test suite alternative now that more warnings are generated
  suppress usage warning in baseline script
  "xform" alternative baseline update, needed for recent change to master
  baseline update due to shift in number of lines in base intel framework script
  environment variable (which has precedence) not flags for baseline usage test
  splitting out "usage" test suite alternative into -u/-uu versions
  adding &is_used attribute for base scripts - not actually needed yet, but will be once optimization is added
  removing unused assignments from base scripts
  driver glue for invoking use-def construction
  classes for managing and propagating use-defs
  enhancements/changes to the Reduce class in preparation for use-defs
  some tidying with smart pointers
  flag/environment variable for dumping use-defs
  &is_used attribute to suppress set-but-not-used usage warnings
  whitespace micro-preening
2021-02-23 12:11:24 -08:00
Vern Paxson
1605b57dec suppress usage warning in baseline script 2021-02-06 11:05:53 -08:00
Vern Paxson
da40c580b2 environment variable (which has precedence) not flags for baseline usage test 2021-02-06 11:01:30 -08:00
Vern Paxson
7f9ee3d028 added &is_assigned test case for variable rather than record field 2021-02-05 08:22:19 -08:00
Vern Paxson
5f79cb7668 test for -uu correctly tracking $?, and not misled by conditional assignments 2021-02-04 12:58:23 -08:00
Vern Paxson
c991c54690 &is_set => &is_assigned 2021-02-04 12:18:46 -08:00
Jon Siwek
cfe29c2488 Merge branch 'master' into topic/vern/reaching-defs 2021-02-01 17:27:37 -08:00
Tim Wojtulewicz
00868e00d3 Fix language.init-in-anon-function btest due to changes to log filter predicates 2021-01-27 10:52:41 -07:00
Tim Wojtulewicz
7292b52f66 Remove some deprected methods/events from bif files 2021-01-27 10:52:40 -07:00
Vern Paxson
c43925b7ee updates to test suite, including new baseline for "-a usage" environment 2021-01-23 10:57:08 -08:00
Jon Siwek
7f3f5a868f Merge remote-tracking branch 'origin/topic/jsiwek/gh-1296-fix-list-type-checks'
* origin/topic/jsiwek/gh-1296-fix-list-type-checks:
  GH-1296: fix type-checks related to list-type equality
2021-01-21 14:00:07 -08:00
Jon Siwek
ee4c259cd4 Merge branch 'master' into topic/vern/script-xform 2021-01-13 12:18:44 -08:00
Jon Siwek
8c64ba6907 GH-1296: fix type-checks related to list-type equality
List-types as used in composite table/set indices, for example,
previously had incorrect same_type() comparisons due to flattening
of the list-type into a single type without checking whether the
number and kind of types all match.

This patch simply removes the flatten_type() call from same_type() since
it was already contradicting/preventing a subsequent full-comparison
between elements of two TYPE_LISTs.

There was also a superfluous special-case of the `in` operator's
type-checking for testing whether a record is in a table/set.  It's
superfluous because the general case will already do the type-checking
from MatchesIndex() after first wrapping the record operand in a
ListExpr.  The previous logic was incorrectly relying on the
flatten_type() for testing equality of a record-type against a
list-type, whereas the general case correctly normalizes to testing
equality of two list-types.

The special-cased type-checking logic for assigning a record value to a
table index during its initialization similarly needed minor
re-organization in order to maintain the same error messages as before.
2021-01-10 17:32:50 -08:00
Vern Paxson
117ff4eb08 bug in correctly inspecting test output file 2021-01-10 14:23:42 -08:00
Vern Paxson
35421b07f1 tests added for new capture-by-reference closure semantics & errors 2021-01-04 14:38:12 -08:00
Vern Paxson
fd309676bd test suite updates and additions for new capture semantics & suppression of error cascades 2021-01-04 14:34:18 -08:00
Jon Siwek
b0505277ae
Fix typo in table iterator invalidation test comment
Co-authored-by: Tim Wojtulewicz <tim@corelight.com>
2020-12-14 10:42:37 -08:00
Jon Siwek
8f98b068c8 GH-1328: Improve behavior of Dictionary iterator invalidation
Previously, an assertion was triggered in debug builds upon any attempt
to insert or remove a Dictionary entry while any iteration of that
Dictionary is underway and also even in cases where Dictionary membership
was not actually modified (and thus invalidates a loop).

Now, it emits run-time warnings regardless of build-type and only when
insert/remove operations truly change the Dictionary membership.  In the
context of a Zeek script causing an invalidation, the warning message
also now helps pinpoint the exact expression that causes it.
2020-12-11 18:52:31 -08:00
Christian Kreibich
b04082c140 Additional use of btest-diff --binary 2020-12-06 20:19:52 -08:00
Jon Siwek
d4528162d1 GH-1273: Change SizeExpr to yield "any" type when operating on "any" 2020-11-09 19:03:42 -08:00
Jon Siwek
fb85a19fa8 GH-1245: require TLD of hostname literals to start with a letter 2020-10-23 14:17:10 -07:00