Commit graph

3525 commits

Author SHA1 Message Date
Florian Wilkens
a7fe2bd9c5 testing: feature gate ssl/decryption test 2021-06-25 11:05:29 +02:00
Florian Wilkens
8c67b9c8fc testing: add ssl/decryption test 2021-06-25 11:05:29 +02:00
Florian Wilkens
2d950ffde9 ssl: rudimentary decryption for TLS 1.2
Several limitations still apply:
- TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 is the only supported cipher suite
- Some tests are broken due to a failing assertion regarding bytestring
- No newly written tests for decryption (the patch was tested extensively for our paper)
- Several small open technical questions marked with FIXME
- Architecture in the ssl module might not be optimal
2021-04-19 17:38:02 +02:00
Jon Siwek
f176da200e Merge remote-tracking branch 'origin/topic/jsiwek/gh-1506-fix-broker-func-indices'
- Extended a btest to cover tables/sets with index types
  (this was originally broken until fixed by GH-1514)

* origin/topic/jsiwek/gh-1506-fix-broker-func-indices:
  GH-1506: Fix Broker unserialization of set/table function indices
2021-04-16 16:03:29 -07:00
Tim Wojtulewicz
df22bdd52e Merge remote-tracking branch 'origin/topic/timw/1487-not-valid-enum'
* origin/topic/timw/1487-not-valid-enum:
  Move an assert() in input/Manager.cc to account for ValueToVal errors
  Add test for config framework
  Fix similar issues with ValueTo* methods in the input framework
  GH-1487: Handle error from ValueToVal instead of ignoring it
2021-04-16 08:08:59 -07:00
Jon Siwek
846cc53780 Merge remote-tracking branch 'origin/topic/jsiwek/fix-table-vector-index'
* origin/topic/jsiwek/fix-table-vector-index:
  Fix indexing of set/table types with a vector
2021-04-15 17:59:31 -07:00
Jon Siwek
7d0d57d33b Merge remote-tracking branch 'origin/topic/jsiwek/gh-1507-sip-junk'
* origin/topic/jsiwek/gh-1507-sip-junk:
  GH-1507: Tolerate junk data before SIP requests
2021-04-15 17:43:51 -07:00
Jon Siwek
df9b57118d Merge remote-tracking branch 'origin/topic/jsiwek/gh-1503-improve-expire-func-type-check'
* origin/topic/jsiwek/gh-1503-improve-expire-func-type-check:
  Extend language.expire-func-type-check btest
  GH-1503: Improve &expire_func compatibility type-checking
2021-04-15 17:42:33 -07:00
Jon Siwek
05b992b865 Extend language.expire-func-type-check btest
Now covers more forms of both valid and invalid &expire_funcs
2021-04-15 16:13:35 -07:00
Jon Siwek
d3ace9b7ac GH-1506: Fix Broker unserialization of set/table function indices
Zeek function types are serialized as a broker::vector, but the
unserialization logic for Zeek set/table types that use a function for
as an index incorrectly identified it as a composite-index, which also
use broker::vector, rather than a singleton-index, and makes such
unserialization fail.

A general example where this failure can happen in practice is when
trying to unserialize a connection record for which there's a
Conn::RemovalHook, since that's a set[function], and a specific case of
that is use of the Intel Framework in a Zeek cluster.
2021-04-15 01:13:51 -07:00
Jon Siwek
735ac0b4fa Fix indexing of set/table types with a vector
Previous behavor:

  internal error: bad index type in CompositeHash::ComputeSingletonHash
  Aborted (core dumped)
2021-04-14 21:58:14 -07:00
Tim Wojtulewicz
fc3438bda5 Merge remote-tracking branch 'origin/topic/jsiwek/gh-1450-large-double-format'
* origin/topic/jsiwek/gh-1450-large-double-format:
  GH-1450: Improve printing/logging of large double/interval/time values
2021-04-14 16:06:11 -07:00
Jon Siwek
183789294f GH-1507: Tolerate junk data before SIP requests
This allows for data that won't match a SIP request method to precede an
actual request and generates a new 'sip_junk_before_request' weird when
encountering such a situation.
2021-04-14 15:34:07 -07:00
Jon Siwek
01f6264c87 GH-1503: Improve &expire_func compatibility type-checking
Previously, incompatible &expire_funcs could mistakenly be used, such as
when using that attribute on the unspecified table()/set()
initializations/assignments, resulting in invalid function calls that
eventually crash Zeek.
2021-04-12 16:27:50 -07:00
Jon Siwek
952d8de8fd Add btest for unterminated pattern parsing error behavior
And clarified the error message that it's more about finding an
unterminated pattern than knowing for sure there's remaining pattern
text spanning multiple lines.
2021-04-09 11:15:06 -07:00
Jon Siwek
312547ab0a GH-1497: Support CRLF line-endings in Zeek scripts and signature files 2021-04-08 20:32:30 -07:00
Tim Wojtulewicz
e5d3d009fc Add test for config framework 2021-04-08 14:07:40 -07:00
Tim Wojtulewicz
04c201393f Fix similar issues with ValueTo* methods in the input framework 2021-04-08 14:07:16 -07:00
Jon Siwek
cc15c985ca GH-1450: Improve printing/logging of large double/interval/time values
The modp_dtoa/modp_dtoa2 functions aren't capable of handling double
values larger than INT_MAX and fallback on using sprintf() in that
situation.  Previously, the format string to that sprintf() was "%e",
defaulting to a precision of 6, which is already too few digits to
represent a number known to be larger than INT_MAX.  Now, an sprintf()
is still performed for values larger than INT_MAX and still uses a
scientific notation format, but in a way that uses as many decimal
digits as needed to preserve information.
2021-04-06 19:54:02 -07:00
Jon Siwek
cd8f19328a Merge remote-tracking branch 'origin/topic/neverlord/gh-1408'
- Minor adjustments to header includes, whitespace, and a
  compiler warning fix during merge

* origin/topic/neverlord/gh-1408:
  Add new Timer class to the telemetry API
  Fix build with latest VectorVal API
  Update baselines
  Prefix telemetry BIFs with __
  Expose telemetry API to scripting land
  Add handle types for histogram metrics
  Move duplicated code to detail header
  Adhere to Zeek coding style
  Apply suggestions from code review
  Add telemetry Manager to Zeek setup / teardown
  Add missing whitespaces for Zeek coding style
  Add gauge metric types
  Add scaffold for new metrics API with counters
2021-04-05 14:21:44 -07:00
Jon Siwek
e6955d55dd Increase timeout of plugins.reader btest 2021-04-03 09:40:45 -07:00
Jon Siwek
3e9ac00f21 Separate stdout from stderr in btest baselines
Redirecting both to the same file can show platform-specific differences
(e.g. Alpine), likely due to different buffering defaults.
2021-03-30 16:23:23 -07:00
Jon Siwek
8e47494dff Remove newline-eof canonification attempt in diff-remove-timestamps
On Alpine (BusyBox `sed`), the previous `sed -e '$a\'` invocation always
added a newline, breaking most every diff.  There doesn't seem to be a
need to attempt normalizing EOF newlines at the moment and doing it that
way doesn't seem to be portable anyway.  If canonifiers need to be
portable, `sed` should be treated as a text-processing tool and POSIX
definition of text-file is zero or more newline-terminated
character-sequences, so if canonification of Baselines via `sed` is
required, those Baselines should always end with a newline to be
considered text files.  I.e. that's not the job of this canonifier, and
changing it also doesn't necessarily generalize since it could be
considered coincidental that diff-remove-timestamps in particular is the
default canonifier that's commonly used while there's still others that
also make use of `sed`.
2021-03-30 16:04:34 -07:00
Jon Siwek
c033bd20aa Merge remote-tracking branch 'origin/topic/vern/global-stmts'
* origin/topic/vern/global-stmts:
  whitespace adjustment
  "balance" tests with multiple Zeek scripts to load the same elements
  put global statements into a quasi-function to support script optimization
2021-03-29 15:42:43 -07:00
Dominik Charousset
600d1118cd Update baselines 2021-03-29 10:47:32 +02:00
Dominik Charousset
e476911075 Prefix telemetry BIFs with __ 2021-03-29 10:47:32 +02:00
Dominik Charousset
2bac354e43 Expose telemetry API to scripting land 2021-03-29 10:47:32 +02:00
Jon Siwek
a0859276bf Merge remote-tracking branch 'origin/topic/timw/analyzer-crash'
* origin/topic/timw/analyzer-crash:
  Fix crash in Analyzer::ForwardPacket due to recursive analyzer calls.
2021-03-26 16:40:12 -07:00
Tim Wojtulewicz
4015beb732 Fix crash in Analyzer::ForwardPacket due to recursive analyzer calls.
The change in 44f558df7b that made analyzer_list
a std::vector instead of a std::list doesn't take into account that in some
cases an analyzer may chain back into itself, such as with UDP-in-UDP tunnels.
In these cases, the second call to ForwardPacket may cause iterator
invalidation, leading to a crash.
2021-03-25 08:45:38 -07:00
Vern Paxson
3139cf2594 "balance" tests with multiple Zeek scripts to load the same elements 2021-03-24 21:37:26 -07:00
Vern Paxson
a288800b41 test suite update due to factoring out coerce_to_record() 2021-03-24 15:16:03 -07:00
Vern Paxson
91c2e05099 factor out record coercion; modernize management of coercion "map" 2021-03-24 15:03:00 -07:00
Tim Wojtulewicz
f45df63cd0 Merge remote-tracking branch 'origin/topic/vern/zval'
* origin/topic/vern/zval: (42 commits)
  whitespace tweaks
  resolved some TODO comments
  remove unnecessary casts, and change necessary ones to use static_cast<>
  explain cmp_func default
  change functions for ZVal type management to static members
  fix some unsigned/signed integer warnings
  address lint concern about uninitialized variable
  Remove use of obsolete forward-declaration macros
  fix #include's that lack zeek/ prefixes
  explicitly populate holes created in vectors
  fixes for now-incorrect assumption that GetField always returns an existing ValPtr
  memory management for assignment to vector elements
  memory management for assignment to record fields
  destructor cleanup from ZAM_vector/ZAM_record
  fix #include's that lack zeek/ prefixes
  overlooked another way in which vector holes can be created
  initialize vector holes to the correct corresponding type
  explicitly populate holes created in vectors
  fix other instances of GetField().get() assuming long-lived ValPtr's
  fix for now-incorrect assumption that GetField always returns an existing ValPtr
  ...
2021-03-23 20:44:19 -07:00
Jon Siwek
569830526f Increase timeout of openflow.log-cluster btest 2021-03-23 19:34:34 -07:00
Jon Siwek
f46d3dec8f Merge remote-tracking branch 'origin/topic/vern/cpp-new-func'
- Removed a couple of dead statements during merge

* origin/topic/vern/cpp-new-func:
  option for deterministic descriptions of sets & tables
  determinism for concurrent Zeek test suite invocations; split out deprecations
  disambiguate descriptions of enums; include attributes when describing records
  more liberal view of attribute equality; allow suppressing attr type-checking
  support for operations on sets that return new values
  low-level addition of enum values
  sundry accessors/cast-ers; RE_Matcher's track their construction values
  convenience functions for comparing IP addresses
2021-03-23 19:27:11 -07:00
Vern Paxson
a808e166a4 determinism for concurrent Zeek test suite invocations; split out deprecations 2021-03-18 16:17:25 -07:00
Vern Paxson
ef3eaed3fb disambiguate descriptions of enums; include attributes when describing records 2021-03-18 10:53:36 -07:00
Vern Paxson
e21c0f1115 correcting some simple typos 2021-03-18 08:19:46 -07:00
Gorka Olalde Mendia
ec8c5f6c07 Add tests for ERSPAN Type I patch
Co-authored-by: Markel Elorza Alvarez <melorzaalvarez@gmail.com>
Co-authored-by: Ivan Arrizabalaga Cupido <ivanarrcup@gmail.com>
2021-03-17 14:41:29 +01:00
Jon Siwek
07bad2d40e Merge remote-tracking branch 'origin/topic/seth/fix-smb-ts-fields'
* origin/topic/seth/fix-smb-ts-fields:
  Updating external test commit pointers
  ts fields in SMB logs now default to network_time()
2021-03-11 12:44:04 -08:00
Seth Hall
87950cb833 Updating external test commit pointers 2021-03-10 18:23:15 +00:00
Jon Siwek
4b6369a333 Add alternative 'xform' baseline for test: broker.store.create-failure 2021-03-08 12:59:30 -08:00
Jon Siwek
75d3cbda4b Merge branch 'master' into topic/vern/ast-opt 2021-03-08 12:35:04 -08:00
Jon Siwek
6af436aad3 GH-1426: Improve handling of Broker data store creation failures
Broker::create_master() and Broker::create_clone() now return
a valid value even when there's a failure to open the backend database
(e.g. SQLite filesystem error).  In that case, the returned value can
still be passed into other data store operations, but they'll fail
immediately with an error.  Broker::is_closed() can now also be used to
determine whether the data store creation calls failed.
2021-03-06 02:32:29 -08:00
Vern Paxson
9496379b95 don't know how I overlooked these minor test suite updates 2021-02-27 13:47:49 -08:00
Vern Paxson
9953961569 "opt" btest baseline updates due to AST optimizations changing printed code 2021-02-27 11:43:17 -08:00
Vern Paxson
4847c357ca "opt" baseline exceptions due to incompatibility with optimize-AST and -u 2021-02-27 11:42:24 -08:00
Vern Paxson
8f21432f76 new "opt" btest alternative 2021-02-27 11:41:23 -08:00
Vern Paxson
42d59203a1 update to "xform" test suite baseline due to recent line number changes 2021-02-27 11:40:33 -08:00
Jon Siwek
6ac499882c Teach Zeekygen to produce source-code-range information
Related to https://github.com/zeek/zeek-docs/issues/56
2021-02-26 14:55:59 -08:00