Commit graph

16273 commits

Author SHA1 Message Date
Arne Welzel
d51613a7b2 Merge remote-tracking branch 'origin/topic/awelzel/fix-cluster-testing-docker-compose-v2'
* origin/topic/awelzel/fix-cluster-testing-docker-compose-v2:
  testing/external/cluster: Switch to docker-compose v2
2024-03-15 13:37:20 +01:00
Arne Welzel
de7292e5f0 testing/external/cluster: Switch to docker-compose v2
The Python docker-compose executable in the docker_builder image is not
functional anymore (TypeError: kwargs_from_env() got an unexpected
keyword argument 'ssl_version').

docker-compose v1 has been sun-setted for quite some time. Do the
minimal to switch to docker compose v2.
2024-03-15 13:19:03 +01:00
Tim Wojtulewicz
6fa34c202f Merge remote-tracking branch 'origin/topic/christian/news-contributors'
* origin/topic/christian/news-contributors:
  Add community contributions for the 6.2 timeframe to NEWS. [skip ci]
2024-03-13 10:17:59 -07:00
Christian Kreibich
af83f1f1a3 Add community contributions for the 6.2 timeframe to NEWS. [skip ci] 2024-03-12 20:19:52 -07:00
zeek-bot
672eb0dc2c Update doc submodule [nomail] [skip ci] 2024-03-13 00:10:12 +00:00
Arne Welzel
2d1588277e Merge remote-tracking branch 'origin/topic/awelzel/3640-void-size-expr-assign-stmt'
* origin/topic/awelzel/3640-void-size-expr-assign-stmt:
  Expr: Handle TYPE_VOID in SizeExpr and AssignExpr::Typecheck()
2024-03-12 10:31:46 +01:00
Arne Welzel
bbc7272a42 Merge remote-tracking branch 'origin/topic/vern/script-opt-maint.Mar24'
* origin/topic/vern/script-opt-maint.Mar24:
  AST location fixes for -O gen-C++
  script optimization support for clearing tables/vectors using "delete"
  fix for ZAM printing of instructions that modify slots other than slot 1
  fixed bug where appending an empty vector to a vector-of-vector's had no effect
2024-03-08 19:10:13 +01:00
Vern Paxson
99f43d3c8b AST location fixes for -O gen-C++ 2024-03-08 19:09:46 +01:00
Vern Paxson
74d36eb759 script optimization support for clearing tables/vectors using "delete" 2024-03-08 19:09:46 +01:00
Vern Paxson
ce6d77e2ce fix for ZAM printing of instructions that modify slots other than slot 1 2024-03-08 19:09:46 +01:00
Vern Paxson
54877e9dec fixed bug where appending an empty vector to a vector-of-vector's had no effect 2024-03-08 19:09:43 +01:00
Arne Welzel
8cb1a1518f Expr: Handle TYPE_VOID in SizeExpr and AssignExpr::Typecheck()
@vpax reported surprising behavior when working with "void values".
While these are not exposed to script land, plumb the places he
pointed out are causing confusing behavior.

Closes #3640.
2024-03-07 11:24:28 +01:00
Robin Sommer
8b214862c5
Merge remote-tracking branch 'origin/topic/robin/bump-spicy'
* origin/topic/robin/bump-spicy:
  Bump Spicy.
2024-03-07 09:56:03 +01:00
Robin Sommer
732a3e1d9a
Bump Spicy. 2024-03-07 09:08:44 +01:00
Arne Welzel
eeeaffb42c Merge remote-tracking branch 'origin/topic/jgras/fix-cluster-started-btesting'
* origin/topic/jgras/fix-cluster-started-btesting:
  Make worker restart in cluster btest race-proof
2024-03-06 10:39:05 +01:00
Jan Grashoefer
08daef8d42 Make worker restart in cluster btest race-proof 2024-03-05 23:10:18 +01:00
Arne Welzel
3734adef2d Merge remote-tracking branch 'origin/topic/awelzel/bump-highwayhash'
* origin/topic/awelzel/bump-highwayhash:
  highwayhash: Point to github.com/google/highwayhash, bump to master
2024-03-05 19:23:38 +01:00
Arne Welzel
09ad9b193a Merge remote-tracking branch 'origin/topic/awelzel/re-compile-syntax-error-propagation'
* origin/topic/awelzel/re-compile-syntax-error-propagation:
  RE/Specific_RE_Matcher: Make Compile() return false for syntax errors
2024-03-05 19:22:46 +01:00
Arne Welzel
c1f344e7ca highwayhash: Point to github.com/google/highwayhash, bump to master 2024-03-05 18:01:39 +01:00
zeek-bot
8a9148483f Update doc submodule [nomail] [skip ci] 2024-03-05 00:22:37 +00:00
Tim Wojtulewicz
ce8310ec89 Merge remote-tracking branch 'origin/topic/timw/ci-freebsd-13'
* origin/topic/timw/ci-freebsd-13:
  CI: Bump FreeBSD 13 build to 13.3
2024-03-04 12:33:11 -07:00
Tim Wojtulewicz
ab9a1e88c7 CI: Bump FreeBSD 13 build to 13.3 2024-03-04 12:02:16 -07:00
Arne Welzel
6867eda621 Merge remote-tracking branch 'origin/topic/awelzel/fix-assert-cond-twice'
* origin/topic/awelzel/fix-assert-cond-twice:
  Stmt: Fix assert evaluating cond twice
2024-03-04 18:22:43 +01:00
Robin Sommer
ce344d3f84
Merge remote-tracking branch 'origin/topic/robin/node-rewrite'
* origin/topic/robin/node-rewrite:
  Port Spicy integration to new AST API.
2024-03-04 15:49:29 +01:00
Robin Sommer
db98dc4193
Port Spicy integration to new AST API. 2024-03-04 15:48:39 +01:00
Arne Welzel
6572324b8c Stmt: Fix assert evaluating cond twice
Since 81a9745fb3, the assert condition is
evaluated twice. This leads to unexpected behavior when cond has a side
effect like publishing a message or creating a log stream or filter.

Found while using the following in ad-hoc testing code and wondering
why two messages were published.

    assert publish(Cluster::worker_topic, hello, "abc")
2024-03-04 15:30:09 +01:00
Arne Welzel
724daa2792 Merge remote-tracking branch 'origin/topic/awelzel/zeekjs-0.12.0'
* origin/topic/awelzel/zeekjs-0.12.0:
  Bump zeekjs to 0.12.0
2024-02-28 17:10:44 +01:00
Arne Welzel
15ad59bac5 RE/Specific_RE_Matcher: Make Compile() return false for syntax errors
When creating RE_Matcher instances at runtime and verifying the pattern
compiles via Compile(), the syntax_error flag wasn't respected and
Compile() would return true even for some invalid regular expressions.

For example, compiling /a{1,b}/, Compile() would return true even though
it produced a reporter error while parsing complaining about b not
being valid.

This patch improves the error handling, so that calling Compile() returns
false whenever zeek::detail::synerr() was called while a pattern was
parsed. The use-case is creation of patterns at runtime based on
JavaScript strings. These might be entered or received at runtime via
an API. This change allows to be a bit more robust to detect invalid
input and raising exceptions to notify the user.

This also move syntax_error and csize out of global scope.

If RE_Matcher was to be used as an actual API, we likely should squelch
the reporter errors and mark it as not thread safe, but this is a small
step forward.
2024-02-28 17:07:51 +01:00
Arne Welzel
e3fab940b3 Bump zeekjs to 0.12.0
0a89ca6 doc: Expand zeek.as() description and add an example
  7e814d7 Types: Implement basic pattern support
  43df9d2 Update docs to provide example of shared Node.js openSSL configuration
  3bf2ea5 lsan suppressions: Add some for 21.11
  640affa zeek.global_vars: Remove leftover internal field usage
  3ee53c7 zeek.global_vars: Implement setter
  8144061 zeek.as: Support more Zeek types
  b453483 zeek.as: Fix crash for non-atomic types
2024-02-28 11:10:56 +01:00
zeek-bot
45152e4879 Update doc submodule [nomail] [skip ci] 2024-02-28 00:17:13 +00:00
Arne Welzel
e33bf64dc9 Merge remote-tracking branch 'origin/topic/awelzel/3619-zeekygen-spicyc-mtime'
* origin/topic/awelzel/3619-zeekygen-spicyc-mtime:
  spicy/zeekygen: Remove mtime from generated code
2024-02-27 15:18:58 +01:00
Arne Welzel
ecdd2b0b29 spicy/zeekygen: Remove mtime from generated code
Zeekygen implements its own make-style update logic to prevent
re-creation of files that have not changed. To fulfill this, we
currently encode the current time into spicyz generated .cc files.

This degrades ccache efficiency for built-in analyzers and also
for all .evt files compiled during testing. Switch SpicyModuleInfo
to return current time instead. This results in the re-generation
of documentation files unconditionally when running Zeekygen, but
that seems more acceptable IMO.

Generally wonder if Zeekygen should produce output unconditionally
and if we need to clobber prevention, compare with the content of
the existing file.

Closes #3619
2024-02-27 15:06:02 +01:00
zeek-bot
961bac0220 Update doc submodule [nomail] [skip ci] 2024-02-27 00:12:07 +00:00
Arne Welzel
d728981b77 Merge remote-tracking branch 'origin/topic/awelzel/move-iso-9660-sig-to-policy'
* origin/topic/awelzel/move-iso-9660-sig-to-policy:
  signatures/iso-9660: Add \x01 suffix to CD001
  test-all-policy: Do not load iso-9660.zeek
  signatures: Move ISO 9660 signature to policy
2024-02-26 21:16:48 +01:00
Arne Welzel
e87272f5a7 Merge remote-tracking branch 'origin/topic/awelzel/3490-global-ids-type-name-change'
* origin/topic/awelzel/3490-global-ids-type-name-change:
  global_ids: Align script_id$type_name field with type_name()
2024-02-26 21:15:25 +01:00
Arne Welzel
77d57fccfd Merge remote-tracking branch 'origin/topic/awelzel/find-in-zeekpath'
* origin/topic/awelzel/find-in-zeekpath:
  NEWS: Mention can_load() and find_in_zeekpath()
  utils: Introduce packages.zeek with can_load() helper
  zeek.bif: Add find_in_zeekpath() helper
2024-02-26 21:14:59 +01:00
Arne Welzel
f4ed1e05fc signatures/iso-9660: Add \x01 suffix to CD001
As discussed with Tim, that should make it a bit more robust against
false positives.
2024-02-26 21:00:01 +01:00
Tim Wojtulewicz
c315202ece Merge remote-tracking branch 'origin/topic/johanna/update-tls-ca-stuff'
* origin/topic/johanna/update-tls-ca-stuff:
  Update Mozilla CA and Google CT list and related tests.
2024-02-26 11:10:32 -07:00
Arne Welzel
e11c20e1eb test-all-policy: Do not load iso-9660.zeek
Changing the default_file_bof_buffer_size has subtle impact on
MIME type detection and changed the zeek-testing baseline. Do
not load this new script via test-all-policy to avoid this.

The new test was mainly an aid to understand what is actually going on.
In short, if default_file_bof_buffer_size is larger than the file MIME
detection only runs when the buffer is full, or when the file is removed.
When a file transfer happens over multiple HTTP connections, only
some or one of the http.log entries will have a proper response MIME type.

PCAP extracted from 2009-M57-day11-18.trace.gz.
2024-02-26 17:58:26 +01:00
Arne Welzel
1a5ce65e3d signatures: Move ISO 9660 signature to policy
The previous "fix" caused significant performance degradation without
the signature ever having a chance to trigger. Moving it to policy
seems the best compromise, the alternative being outright removing it.
2024-02-26 13:35:23 +01:00
Arne Welzel
1c2d1b3717 global_ids: Align script_id$type_name field with type_name()
Populate script_id$type_name with what the type_name() bif
would produce for the same identifier.

Closes #3490
2024-02-23 17:26:56 +01:00
Johanna Amann
c81274b549 Update Mozilla CA and Google CT list and related tests.
Test updates are necessary due to removed CT logs.

Fixes GH-3620
2024-02-23 11:50:31 +00:00
zeek-bot
f96600391a Update doc submodule [nomail] [skip ci] 2024-02-23 00:12:41 +00:00
Arne Welzel
0dfee33c83 Merge remote-tracking branch 'origin/topic/awelzel/3139-zeek-version-spicy-ccache-busting'
* origin/topic/awelzel/3139-zeek-version-spicy-ccache-busting:
  spicy: Remove Zeek version from spicyz/config.h.in
2024-02-22 17:00:46 +01:00
Arne Welzel
02703eeb9d spicy: Remove Zeek version from spicyz/config.h.in
It seems that Zeek's version number and string only need to be
available at runtime, so this change removes it from spicyz/configh.in
to avoid needlessly busting ccache for the src/spicyz tree for on a
Zeek version bump.

Closes #3139.
2024-02-22 16:59:46 +01:00
Arne Welzel
5742f7cc21 spicy: Drop ZEEK_VERSION_NUMBER ifdef'ery
Given src/spicy is part of the Zeek tree, these are not necessary and
are probably just remains from the spicy-plugin era.
2024-02-22 13:00:15 +01:00
Arne Welzel
08bf8d47f4 Merge remote-tracking branch 'origin/topic/awelzel/iso-9660-image-signature'
* origin/topic/awelzel/iso-9660-image-signature:
  signatures: Fix ISO 9960 signature
2024-02-22 12:38:55 +01:00
Arne Welzel
d2409dd432 signatures: Fix ISO 9960 signature
This signature only really works when default_file_bof_buffer_size is bumped
to a sufficient value (40k).
2024-02-22 12:37:40 +01:00
Arne Welzel
036bcfe919 Merge remote-tracking branch 'origin/topic/awelzel/ftp-clear-fuid-after-logging'
* origin/topic/awelzel/ftp-clear-fuid-after-logging:
  testing: Update zeek-testing-private commit hash
  ftp: Reset fuid after logging
2024-02-22 12:23:21 +01:00
Arne Welzel
e9e930bdfa Merge remote-tracking branch 'origin/topic/awelzel/delete-stmt-table-set'
* origin/topic/awelzel/delete-stmt-table-set:
  Allow delete statement for tables, sets and vectors
2024-02-22 12:02:37 +01:00