Commit graph

15389 commits

Author SHA1 Message Date
Tim Wojtulewicz
34f7c95a51 Add missing assignment operators to RobustDictIterator 2023-01-27 13:03:52 -07:00
Tim Wojtulewicz
4cdd68f019 Fix some member-variable shadowing issues 2023-01-27 13:03:19 -07:00
Tim Wojtulewicz
5bfd84a903 Fix/simplify some if statement comparisons 2023-01-27 13:03:19 -07:00
Tim Wojtulewicz
103b381874 Fix a few minor int pointer value initializations 2023-01-27 13:03:18 -07:00
Tim Wojtulewicz
3b0e8ee6f1 Fix a bunch of missing class member initializations 2023-01-27 13:03:18 -07:00
Tim Wojtulewicz
7374688d0d Remove unnecessary virtual specifiers from final classes 2023-01-27 13:03:18 -07:00
Arne Welzel
adf56ef4d8 Skip somer error reporting when the record type has errors
The added test cases around function/event invocations report the
following flurry of errors when only the first one is relevant and
actionable. There's little use in reporting a mismatch with "error".
Squelch them.

    error in <...>/function-invoke-mismatch-error.zeek, line 8: identifier not defined: MyEnumTypo
    error in <...>/function-invoke-mismatch-error.zeek, line 12 and error: type mismatch (M::MY_ENUM_A and error)
    error in <...>/function-invoke-mismatch-error.zeek, line 12: argument type mismatch in function call (M::to_string(M::MY_ENUM_A))
    error in <...>/function-invoke-mismatch-error.zeek, line 16 and error: type mismatch (M::MY_ENUM_B and error)
    error in <...>/function-invoke-mismatch-error.zeek, line 16: argument type mismatch in function call (M::to_string(M::MY_ENUM_B))
    error in <...>/function-invoke-mismatch-error.zeek, line 20 and error: type mismatch (M::e and error)
    error in <...>/function-invoke-mismatch-error.zeek, line 20: argument type mismatch in function call (M::to_string(M::e))

Record coercion also reports noisy errors when coercing to a type that
has errors for individual fields, type clashing with "error":

    $ zeek language/record-field-error.zeek
    error in ./language/record-coerce-error.zeek, line 8: identifier not defined: MyEnumTypo
    error in ./language/record-coerce-error.zeek, line 19 and ./language/record-coerce-error.zeek, line 5: type clash for field "e" ((coerce [$e=MY_ENUM_B, $s=test] to MyRecord) and MyEnum)
2023-01-27 20:49:22 +01:00
Tim Wojtulewicz
6876462f48 Merge remote-tracking branch 'origin/topic/timw/teredo-detect-length-check'
* origin/topic/timw/teredo-detect-length-check:
  Add a few extra length checks to Teredo::DetectProtocol
2023-01-27 12:08:34 -07:00
Arne Welzel
8be8c22b3e smb1: Prevent accessing uninitialized referenced_tree
The added pcap was created from an OSS Fuzz test case and is borderline
valid SMB traffic, but it triggered a scripting error.

Closes #2726
2023-01-27 19:22:13 +01:00
Arne Welzel
589e042e26 Merge branch 'topic/awelzel/ci-notification-adaption'
* topic/awelzel/ci-notification-adaption:
  github: cat the check_suite payload
  github: Send emails for success conclusion, too
2023-01-27 18:45:32 +01:00
Tim Wojtulewicz
2866bbea4d Add a few extra length checks to Teredo::DetectProtocol 2023-01-27 10:37:24 -07:00
Arne Welzel
24652d07b6 github: cat the check_suite payload
Seeing spurious emails being sent when rerunning tasks. Maybe
the event contains some details.
2023-01-27 18:25:21 +01:00
Arne Welzel
61fb0be021 github: Send emails for success conclusion, too
Default behavior of the ci-email-action is to filter
cancelled,neutral,success. We're interested in
success for the time being.
2023-01-27 17:35:27 +01:00
Arne Welzel
fa48c88533 mysql: Recognize when client/server negotiate SSL
This instantiates the SSL analyzer when the client requests SSL
so that Zeek now has a bit more visibility into encrypted MySQL
connections.

The pattern used is the same as in the IMAP, POP or XMPP analyzer.
2023-01-27 12:42:19 +01:00
Arne Welzel
e9caea9694 Merge branch 'topic/awelzel/2696-mysql-analyzer-issues'
* topic/awelzel/2696-mysql-analyzer-issues:
  testing/mysql: Add traces recorded with a free-tier MySQL instance
  MySQL: Fix endianness, introduce mysql_eof() event
2023-01-27 11:25:23 +01:00
Arne Welzel
03dc21a861 testing/mysql: Add traces recorded with a free-tier MySQL instance
Not sure this adds much more coverage then there was, but minimally
more recent software versions.

The instances/passwords were ephemeral, so hostname and password hashes
etc aren't useful to anyone.
2023-01-27 10:59:23 +01:00
Arne Welzel
672602dae7 MySQL: Fix endianness, introduce mysql_eof() event
We were parsing MySQL using bigendian even though the protocol is
specified as with "least significant byte first" [1]. This is most
problematic when parsing length encoded strings with 2 byte length
fields...

Further, I think, the EOF_Packet parsing was borked, either due to
testing the CLIENT_DEPRECATE_EOF with the wrong endianness, or due to
the workaround in Resultset processing raising mysql_ok(). Introduce a
new mysql_eof() that triggers for EOF_Packet's and remove the fake
mysql_ok() Resultset invocation to fix. Adapt the mysql script and tests
to account for the new event.

This is a quite backwards incompatible change on the event level, but
due to being quite buggy in general, doubt this matters to many.

I think there is more buried, but this fixes the violation of the simple
"SHOW ENGINE INNODB STATUS" and the existing tests continue to
succeed...

[1] https://dev.mysql.com/doc/dev/mysql-server/latest/page_protocol_basic_dt_integers.html
2023-01-27 10:59:23 +01:00
zeek-bot
b1b25e4d34 Update doc submodule [nomail] [skip ci] 2023-01-27 00:43:20 +00:00
Arne Welzel
1772a1bd6d Merge branch 'topic/awelzel/no-abort-identified-not-defined'
* topic/awelzel/no-abort-identified-not-defined:
  Tame error reporting and abort() for undefined types
2023-01-26 20:07:14 +01:00
Arne Welzel
25ee288a76 Tame error reporting and abort() for undefined types
The script added as a test case reports the following messages *and*
dumps a core file. Printing the first error and a normal failure exit
seems sufficient. IMO triggering an abort() due to user scripting issues
is not something that Zeek should do

    $ zeek ./identifier-not-defined-error.zeek
    error in ./identifier-not-defined-error.zeek, line 10: identifier not defined: MyEnu
    error in ./identifier-not-defined-error.zeek, line 10 and error: &default value has inconsistent type (M::MY_ENUM_A and error)
    internal error in ./identifier-not-defined-error.zeek, line 11: type inconsistency in ZVal constructor
    Aborted (core dumped)

Change is to skip certain checks when an error type is propagated.
2023-01-26 20:05:02 +01:00
Arne Welzel
c4302ec280 testing/http: http-11-request-then-cruft
A client sends a "proper" HTTP/1.1 request and afterwards a few T /\n\n sequences.
The latter ones aren't logged.
2023-01-26 19:59:39 +01:00
Arne Welzel
0b26866ecf testing/http: Add pcap extracted from m5-long external test-suite
This tests that the HTTP version is now updated if it changes in the
course of a connection.
2023-01-26 19:59:39 +01:00
Arne Welzel
dab551aaa3 testing/external: m57-long baseline update
There's a HTTP server that first replies with HTTP/1.1, then HTTP/1.0.
Seems actually nicer to have the real value within the log/event.
2023-01-26 19:59:28 +01:00
Arne Welzel
3af6b97c63 analyzers/http: Update request_version on subsequent SetVersion() calls
The #124 PR introduced special treatment when HTTP version 0.9
was set. With #127, a reproducer that set HTTP/1.0 in the first
request was created and subsequent requests wouldn't reset to
HTTP version 0.9.

This is subtle, but doesn't seem like things fall apart.

Improves runtime from 20 seconds to 2 seconds for the given
reproducer.

Fixes #127.
2023-01-26 19:59:02 +01:00
Arne Welzel
942f829825 testing/ftp: Add tests and pcaps with invalid reply lines
These have been created artificially. The tests show that for an
invalid reply line without a numeric code, with a numeric code < 100
or a numeric code not followed by a space we now raise an analyzer
violation and disable the analyzer.
2023-01-26 19:56:20 +01:00
Arne Welzel
cf375cf362 ftp: Harden reply handing a bit and don't raise bad replies to script-land
This improves runtime of the oss-fuzz generated traffic in #125.
Specifically, that reproducers included a 064- reply code that was
interpreted as needing to be continued.

Also, return after AnalyzerViolations() for server replies rather than
propagating bad replies them to script-land. This trusts server's to
generally behave according to specification.
2023-01-26 19:56:20 +01:00
Arne Welzel
8f96ac3b77 ftp: ignore invalid commands
Do not propagate wrong FTP commands to script land.
2023-01-26 19:56:20 +01:00
Tim Wojtulewicz
9a96e3b52a Merge remote-tracking branch 'origin/topic/bbannier/bump-spicy'
* origin/topic/bbannier/bump-spicy:
  Bump spicy to v1.6.1.
2023-01-26 10:55:08 -07:00
Tim Wojtulewicz
a27a3532d8 Merge remote-tracking branch 'origin/topic/timw/2658-avoid-teredo-violations'
* origin/topic/timw/2658-avoid-teredo-violations:
  Update external testing repo commit hashes for Teredo changes
  Update scripts.base.frameworks.analyzer.logging btest to use a different trace file
  Update btest baselines due to reduced calls to UID::Set
  Rework TeredoAnalzyer::DetectProtocol to not catch as many packets
2023-01-26 09:44:48 -07:00
Tim Wojtulewicz
f842a86e80 Update external testing repo commit hashes for Teredo changes 2023-01-26 09:23:22 -07:00
Tim Wojtulewicz
6ebfa02199 Update scripts.base.frameworks.analyzer.logging btest to use a different trace file 2023-01-26 09:14:00 -07:00
Tim Wojtulewicz
3d5918e6b3 Update btest baselines due to reduced calls to UID::Set
UID::Set is called by AnalyzerViolation, and since Teredo isn't calling nearly
as many of those anymore the UIDs used by other calls changed as well.
2023-01-26 09:14:00 -07:00
Tim Wojtulewicz
b0ac20b2f2 Rework TeredoAnalzyer::DetectProtocol to not catch as many packets 2023-01-26 09:14:00 -07:00
Tim Wojtulewicz
8a9dd7b0ae Merge remote-tracking branch 'origin/topic/timw/2669-for-loop-error'
* origin/topic/timw/2669-for-loop-error:
  GH-2669: Give script location for type clash in for-loop construction
2023-01-26 09:11:51 -07:00
Tim Wojtulewicz
5e68880470 GH-2669: Give script location for type clash in for-loop construction 2023-01-26 09:11:10 -07:00
Arne Welzel
c414136288 Merge branch 'topic/awelzel/ci-skip-some-tasks'
* topic/awelzel/ci-skip-some-tasks:
  cirrus: Introduce SKIP_TASK_ON_PR
2023-01-26 14:38:08 +01:00
Benjamin Bannier
885a2fb695 Bump spicy to v1.6.1. 2023-01-26 12:43:39 +01:00
Tim Wojtulewicz
252fb581e7 Merge remote-tracking branch 'origin/topic/vern/add-C++-removal'
* origin/topic/vern/add-C++-removal:
  removed -O add-C++ option and updated documentation
2023-01-25 14:11:10 -07:00
Tim Wojtulewicz
8d815feb34 Merge remote-tracking branch 'origin/topic/timw/2684-remove-vxlan-violations'
* origin/topic/timw/2684-remove-vxlan-violations:
  GH-2684: Stop violating VXLAN for forwarding failures
2023-01-25 11:12:25 -07:00
Tim Wojtulewicz
e61c3a95ad GH-2684: Stop violating VXLAN for forwarding failures 2023-01-25 10:50:07 -07:00
Tim Wojtulewicz
e0553d451f Revert "Update external test repo commits"
This reverts commit 8e405ef2f0.
2023-01-25 08:36:41 -07:00
Tim Wojtulewicz
8e405ef2f0 Update external test repo commits 2023-01-25 08:36:16 -07:00
Tim Wojtulewicz
31ff9a45de Merge remote-tracking branch 'origin/topic/timw/testing-c++-branch-benchmarks'
* origin/topic/timw/testing-c++-branch-benchmarks:
  Fix CI benchmark script to properly urlencode arguments
2023-01-25 08:19:08 -07:00
Arne Welzel
5d708b257f cirrus: Introduce SKIP_TASK_ON_PR
Limit the number of tasks we run on PRs to only a subset instead of
the full set to reduce the amount of resources we use for PRs. Labeling
a PR with "fullci" can be used enable running all tasks.

This removes a total of 14 task execution from PRs by default at the cost of
needing to monitor master builds more closely for distro specific breakage.

This could probably be even be more advanced:

Leveraging changesIncludeOnly('scripts/*', 'testing/*'), we might be able to
limit to running a single Linux distro rather than a whole set of different
for simple script changes. For now I've left that out, however.
2023-01-25 12:54:10 +01:00
Tim Wojtulewicz
b522f6ea08 Fix CI benchmark script to properly urlencode arguments 2023-01-24 14:53:46 -07:00
Vern Paxson
65a2900bb7 removed -O add-C++ option and updated documentation 2023-01-24 10:52:58 -08:00
Tim Wojtulewicz
a82f743bcc Merge remote-tracking branch 'origin/topic/timw/test-bad-rdp-keyboard'
* origin/topic/timw/test-bad-rdp-keyboard:
  Add test for new handling of unknown RDP keyboards
2023-01-24 09:01:51 -07:00
zeek-bot
e3494b9483 Update doc submodule [nomail] [skip ci] 2023-01-24 00:31:03 +00:00
Tim Wojtulewicz
ac7e7f9ed9 Merge remote-tracking branch 'origin/topic/vern/Jan23-C++-maint'
* origin/topic/vern/Jan23-C++-maint:
  Maintenance updates for -O gen-C++ / -O gen-standalone-C++   fixes for using BiFs in standalone global initializations   avoiding redundant global initializations   updates to maintenance scripts and notes   removal of an unused member variable
2023-01-23 14:35:24 -07:00
Tim Wojtulewicz
89675fc7de Add test for new handling of unknown RDP keyboards 2023-01-23 13:01:48 -07:00