Commit graph

2647 commits

Author SHA1 Message Date
Vern Paxson
a288800b41 test suite update due to factoring out coerce_to_record() 2021-03-24 15:16:03 -07:00
Vern Paxson
91c2e05099 factor out record coercion; modernize management of coercion "map" 2021-03-24 15:03:00 -07:00
Tim Wojtulewicz
f45df63cd0 Merge remote-tracking branch 'origin/topic/vern/zval'
* origin/topic/vern/zval: (42 commits)
  whitespace tweaks
  resolved some TODO comments
  remove unnecessary casts, and change necessary ones to use static_cast<>
  explain cmp_func default
  change functions for ZVal type management to static members
  fix some unsigned/signed integer warnings
  address lint concern about uninitialized variable
  Remove use of obsolete forward-declaration macros
  fix #include's that lack zeek/ prefixes
  explicitly populate holes created in vectors
  fixes for now-incorrect assumption that GetField always returns an existing ValPtr
  memory management for assignment to vector elements
  memory management for assignment to record fields
  destructor cleanup from ZAM_vector/ZAM_record
  fix #include's that lack zeek/ prefixes
  overlooked another way in which vector holes can be created
  initialize vector holes to the correct corresponding type
  explicitly populate holes created in vectors
  fix other instances of GetField().get() assuming long-lived ValPtr's
  fix for now-incorrect assumption that GetField always returns an existing ValPtr
  ...
2021-03-23 20:44:19 -07:00
Vern Paxson
a808e166a4 determinism for concurrent Zeek test suite invocations; split out deprecations 2021-03-18 16:17:25 -07:00
Vern Paxson
ef3eaed3fb disambiguate descriptions of enums; include attributes when describing records 2021-03-18 10:53:36 -07:00
Gorka Olalde Mendia
ec8c5f6c07 Add tests for ERSPAN Type I patch
Co-authored-by: Markel Elorza Alvarez <melorzaalvarez@gmail.com>
Co-authored-by: Ivan Arrizabalaga Cupido <ivanarrcup@gmail.com>
2021-03-17 14:41:29 +01:00
Jon Siwek
75d3cbda4b Merge branch 'master' into topic/vern/ast-opt 2021-03-08 12:35:04 -08:00
Jon Siwek
6af436aad3 GH-1426: Improve handling of Broker data store creation failures
Broker::create_master() and Broker::create_clone() now return
a valid value even when there's a failure to open the backend database
(e.g. SQLite filesystem error).  In that case, the returned value can
still be passed into other data store operations, but they'll fail
immediately with an error.  Broker::is_closed() can now also be used to
determine whether the data store creation calls failed.
2021-03-06 02:32:29 -08:00
Vern Paxson
9496379b95 don't know how I overlooked these minor test suite updates 2021-02-27 13:47:49 -08:00
Jon Siwek
6ac499882c Teach Zeekygen to produce source-code-range information
Related to https://github.com/zeek/zeek-docs/issues/56
2021-02-26 14:55:59 -08:00
Vern Paxson
98ada4e419 test suite updates for expanded sort() BiF semantics (bools, doubles) 2021-02-25 17:13:50 -08:00
Vern Paxson
ab0b773522 test suite updates reflecting that vectors no longer support arbitrary "holes" 2021-02-25 17:00:36 -08:00
Vern Paxson
e64805430b fix off-by-one assumption regarding vector indexing dating to 2011 2021-02-24 16:31:07 -08:00
Jon Siwek
737d2c390b Support explicit disabling of file analyzers 2021-02-23 15:50:18 -08:00
Jon Siwek
80221cb958 Merge remote-tracking branch 'origin/topic/vern/use-defs'
- Added explicit test case of "unused assignement" warning and
  &is_used suppression during merge.

* origin/topic/vern/use-defs:
  activate &is_used
  removed unnecessary statement flagged by Coverity (thanks, Jon!)
  Adjust reference/move nitpicks in use-def/reduce code
  Adjust some whitespace in UseDefs.cc
  updates to "usage" test suite alternative now that more warnings are generated
  suppress usage warning in baseline script
  "xform" alternative baseline update, needed for recent change to master
  baseline update due to shift in number of lines in base intel framework script
  environment variable (which has precedence) not flags for baseline usage test
  splitting out "usage" test suite alternative into -u/-uu versions
  adding &is_used attribute for base scripts - not actually needed yet, but will be once optimization is added
  removing unused assignments from base scripts
  driver glue for invoking use-def construction
  classes for managing and propagating use-defs
  enhancements/changes to the Reduce class in preparation for use-defs
  some tidying with smart pointers
  flag/environment variable for dumping use-defs
  &is_used attribute to suppress set-but-not-used usage warnings
  whitespace micro-preening
2021-02-23 12:11:24 -08:00
Johanna Amann
6c554ddde3 Ascii reader test with 0-bytes
Add a second test, which contains actual 0-bytes.
2021-02-12 14:49:33 +00:00
Johanna Amann
61290fc19c Fix buffer overread in ascii formatter
When a text with an (escaped) zero byte was passed to ParseValue, only
the part of the string up to the zero byte was copied, but the length of
the full string was passed to the input framework.

This leads to the input manager reading over the end of the buffer.

Fixes zeek/zeek#1398
2021-02-12 14:24:36 +00:00
Vern Paxson
6838ef66b4 baseline update due to shift in number of lines in base intel framework script 2021-02-06 11:04:17 -08:00
Jon Siwek
ac0b09d1a4 Allow --parse-only to work with --usage-issues flag
Use of --parse-only would previously exit before --usage-issues had a
chance to analyze scripts and report any discovered issues and it can be
useful to combine both flags for checking scripts for mistakes without
actually executing any code.

This also improves the behavior of --parse-only in combination with
reporting problems in signature files (previously, it exited before
even reading them).
2021-02-05 14:46:47 -08:00
Jon Siwek
03f74958f3 Merge remote-tracking branch 'origin/topic/vern/reaching-defs'
* origin/topic/vern/reaching-defs: (36 commits)
  added &is_assigned test case for variable rather than record field
  Speedup ReachingDefs logic by ~15%
  Simplify ReachingDefs::RDMap() accessor
  test for -uu correctly tracking $?, and not misled by conditional assignments
  &is_set => &is_assigned
  remove pending maybe-reconsider-this comment
  fixes for ?$ operator - always track it, and assume subrecords are initialized
  speedup (and more coherent memory management) for tracking RDs
  fixes for generating and evaluating RDs associate with ?$ expressions
  fix for failure to reduce InlineExpr's to CatchReturnStmt's
  inlining fix: propagate identifier attributes (such as &is_set)
  tidier memory management
  fix for an ancient bug - surprising that this hasn't caused problems previously
  Fix IntrusivePtr release leaks in reaching-def logic
  Change dynamic_cast in reaching-def logic to static_cast
  Adjust some reaching-def memory management
  Update a couple baselines for "xform" alternative
  Adjust various reaching-def move/reference semantics
  Change LambdaExpr::OuterIDs() accessor to return const-reference
  Simplify declaration of DefPointType enum
  ...
2021-02-05 11:02:01 -08:00
Vern Paxson
7f9ee3d028 added &is_assigned test case for variable rather than record field 2021-02-05 08:22:19 -08:00
Vern Paxson
5f79cb7668 test for -uu correctly tracking $?, and not misled by conditional assignments 2021-02-04 12:58:23 -08:00
Jon Siwek
cfe29c2488 Merge branch 'master' into topic/vern/reaching-defs 2021-02-01 17:27:37 -08:00
Tim Wojtulewicz
f53448ccc9 GH-1389: Skip VN-Tag headers 2021-02-01 14:34:56 -07:00
Jon Siwek
1ca85f0221 Merge remote-tracking branch 'origin/topic/timw/deprecation-cleanup'
Merge adjustments:
- Removed some stale str_split() references from docs
- Renumbered TypeTag enum comments
- Simplified test-case for @unload (don't need .bro files anymore)

* origin/topic/timw/deprecation-cleanup:
  Doc updates
  Fix language.init-in-anon-function btest due to changes to log filter predicates
  Remove deprecated log filter predicates for 4.1
  Remove Plugin::HookCallFunction and fix tests related to it
  Remove support for .bro script extension and BRO_ environment variables
  Remove deprecated ICMP events
  Remove some deprected methods/events from bif files
  Remove TYPE_COUNTER
  Remove all of the random single-file deprecations
  Remove all fully-deprecated files
  Update bifcl submodule to remove deprecations from generated code
2021-01-29 16:40:54 -08:00
Christian Kreibich
38d6b6a98b Skip negative integers in input framework where not type-permissible
Script-layer counts, when provided as negative integers in an input
file, got cast to unsigned values because strtoull() does not complain
about negative values. For example, input string "-1" would lead to
value 18446744073709551615 (an all-ones 64-bit int) on x86_64. This is
more likely to be an error than an intent to get very large,
platform-dependent values, so these input lines are now skipped with
according messaging in the reporter.log/stderr.

This also affected ports: -1/tcp got cast to unsigned and only thrown
out because PortVal rejects values > 65535, mapping them to 0. We now
skip such inputs as well.

Updates existing input framework tests to capture the new behavior.
2021-01-28 23:46:43 -08:00
Christian Kreibich
4ce3bf3cd2 Remove deprecated log filter predicates for 4.1
Update the logging framework tests: since hooks operate
by name, they cannot be anonymous. I'm also dropping the &optional
attribute from the status field, since here know that the values are
actually defined, and access to an optional status field should
normally be guarded by the existence test operator.

Also includes baseline update for plugins.hooks, which picks up the
fact that the pred record field is now gone.
2021-01-27 10:52:40 -07:00
Tim Wojtulewicz
725e759560 Remove support for .bro script extension and BRO_ environment variables 2021-01-27 10:52:40 -07:00
Tim Wojtulewicz
44ad614094 Remove deprecated ICMP events 2021-01-27 10:52:40 -07:00
Tim Wojtulewicz
7292b52f66 Remove some deprected methods/events from bif files 2021-01-27 10:52:40 -07:00
Tim Wojtulewicz
b819c9d18c Remove TYPE_COUNTER
This causes some test changes because of the use of Type::Describe in the fnv1a32
BIF method called by hash_hrw::get_site. Since the enum values change with removal
of TYPE_COUNTER, the output from Type::Describe changes slightly.
2021-01-27 10:52:40 -07:00
Vern Paxson
c43925b7ee updates to test suite, including new baseline for "-a usage" environment 2021-01-23 10:57:08 -08:00
Jon Siwek
efba91cd26 Merge branch 'topic/oakljon/gh-1352-smtp-header-parsing' of https://github.com/theavgjojo/zeek
* 'topic/oakljon/gh-1352-smtp-header-parsing' of https://github.com/theavgjojo/zeek:
  GH-1352: Added flag to stop processing SMTP headers in attached messages
2021-01-21 17:01:13 -08:00
Jon Siwek
7f3f5a868f Merge remote-tracking branch 'origin/topic/jsiwek/gh-1296-fix-list-type-checks'
* origin/topic/jsiwek/gh-1296-fix-list-type-checks:
  GH-1296: fix type-checks related to list-type equality
2021-01-21 14:00:07 -08:00
Jon Oakley
25de6f28e5 GH-1352: Added flag to stop processing SMTP headers in attached
messages
2021-01-21 14:55:10 -05:00
Johanna Amann
079d4164c0 Merge remote-tracking branch 'origin/topic/christian/fix-config-segfault'
* origin/topic/christian/fix-config-segfault:
  Btest tweak for improved type rendering in config framework errors and set types
  More precise type information in a config framework error message
  Explicitly don't support sets with multiple index types in input/config frameworks
2021-01-21 11:30:36 +00:00
Jon Siwek
ee4c259cd4 Merge branch 'master' into topic/vern/script-xform 2021-01-13 12:18:44 -08:00
Christian Kreibich
fcab8df8fb Btest tweak for improved type rendering in config framework errors and set types 2021-01-11 13:54:13 -08:00
Jon Siwek
614fade0a4 Merge branch 'master' into topic/vern/lambda-copy-semantics 2021-01-11 11:48:45 -08:00
Jon Siwek
8c64ba6907 GH-1296: fix type-checks related to list-type equality
List-types as used in composite table/set indices, for example,
previously had incorrect same_type() comparisons due to flattening
of the list-type into a single type without checking whether the
number and kind of types all match.

This patch simply removes the flatten_type() call from same_type() since
it was already contradicting/preventing a subsequent full-comparison
between elements of two TYPE_LISTs.

There was also a superfluous special-case of the `in` operator's
type-checking for testing whether a record is in a table/set.  It's
superfluous because the general case will already do the type-checking
from MatchesIndex() after first wrapping the record operand in a
ListExpr.  The previous logic was incorrectly relying on the
flatten_type() for testing equality of a record-type against a
list-type, whereas the general case correctly normalizes to testing
equality of two list-types.

The special-cased type-checking logic for assigning a record value to a
table index during its initialization similarly needed minor
re-organization in order to maintain the same error messages as before.
2021-01-10 17:32:50 -08:00
Vern Paxson
bd2c94521a split bifs.string_utils into a non-error test and an only-errors test, to help control for differing error propagation 2021-01-10 14:21:51 -08:00
Jon Siwek
64631a2d9f Merge branch 'master' into topic/vern/script-inlining 2021-01-06 22:08:29 -08:00
Vern Paxson
35421b07f1 tests added for new capture-by-reference closure semantics & errors 2021-01-04 14:38:12 -08:00
Vern Paxson
fd309676bd test suite updates and additions for new capture semantics & suppression of error cascades 2021-01-04 14:34:18 -08:00
Jon Siwek
604fbea00d Merge remote-tracking branch 'origin/topic/johanna/1323'
* origin/topic/johanna/1323:
  TLS 1.3 changes: Address review feedback
  Add one more TLS 1.3 testcase and update NEWS
  Fix TLS 1.3 session resumption detection.
  Introduce ssl_probable_encrypted_handshake_message event
  SSL Analyzer: ignore CCS for TLS 1.3
  TLS analyzer: change logic to track TLS 1.3 connection establishment
2020-12-22 11:00:59 -08:00
Johanna Amann
22ed75c3ce Add one more TLS 1.3 testcase and update NEWS 2020-12-15 16:57:26 +00:00
Johanna Amann
3c95c9a956 Fix TLS 1.3 session resumption detection.
Now we detect TLS 1.3 session resumption by looking if both sides have
the PSK extension set, which is much more exact than the previous
approach.
2020-12-15 16:34:47 +00:00
Jon Siwek
206c674cc9 Merge EDNS ECS option parsing security/bug fixes 2020-12-15 08:03:40 -08:00
Jon Siwek
30df2c58c0 Merge branch 'master' into topic/vern/script-inlining 2020-12-13 10:42:26 -08:00
Jon Siwek
8f98b068c8 GH-1328: Improve behavior of Dictionary iterator invalidation
Previously, an assertion was triggered in debug builds upon any attempt
to insert or remove a Dictionary entry while any iteration of that
Dictionary is underway and also even in cases where Dictionary membership
was not actually modified (and thus invalidates a loop).

Now, it emits run-time warnings regardless of build-type and only when
insert/remove operations truly change the Dictionary membership.  In the
context of a Zeek script causing an invalidation, the warning message
also now helps pinpoint the exact expression that causes it.
2020-12-11 18:52:31 -08:00