Commit graph

18601 commits

Author SHA1 Message Date
Tim Wojtulewicz
b783441428 Merge branch 'topic/timw/file-analyzer-dict-hash'
* topic/timw/file-analyzer-dict-hash:
  Move allocation of analyzer_hash to file_mgr
2023-09-11 10:16:05 -07:00
Tim Wojtulewicz
743d7e96f5 Move allocation of analyzer_hash to file_mgr 2023-09-11 10:15:38 -07:00
zeek-bot
607d72f7b9 Update doc submodule [nomail] [skip ci] 2023-09-09 00:38:25 +00:00
Arne Welzel
f4322f9584 Merge remote-tracking branch 'origin/topic/vern/script-opt-maint.Sep23'
* origin/topic/vern/script-opt-maint.Sep23:
  fix for ZAM statement-level profiling (broken by GH-3199)
  ZAM fixes for compatibility with GH-3249 changes
  -O gen-C++ fixes for compatibility with GH-3249 changes minor -O gen-C++ BTest updates
  minor BTest reordering to diminish differences with script optimization
2023-09-08 16:47:29 +02:00
Arne Welzel
02184625b0 Merge remote-tracking branch 'origin/topic/awelzel/3267-avoid-unnecessary-inits'
* origin/topic/awelzel/3267-avoid-unnecessary-inits:
  ID/Stmt: Introduce INIT_SKIP and use in ForStmt
  CompositeHash: Skip record initialization when recovering vals
2023-09-08 13:48:47 +02:00
Arne Welzel
cea7c0ab46 ID/Stmt: Introduce INIT_SKIP and use in ForStmt
Currently, loop vars are added to a function scope's inits and
initialized upon entering a function with default values. This
applies to vector, record and table types.

This is unnecessary for variables used in for loops as they are
guaranteed to be initialized while iterating.
2023-09-08 13:05:44 +02:00
Arne Welzel
aaa81cae5d CompositeHash: Skip record initialization when recovering vals
Initializing fields of recovered records caused running &default expression
of fields just so that they are re-assigned in the next step with the
recovered fields. The second test case still shows that the loop var
is initialized as well even though that's not needed.

Add tests for iterating over records with &default attributes for both,
tables and vectors.

Fixes #3267
2023-09-08 13:02:34 +02:00
Arne Welzel
057bc673a8 Merge remote-tracking branch 'origin/topic/bbannier/issue-3276'
* origin/topic/bbannier/issue-3276:
  Propagate failure reason in `spicy::decline_input`/`zeek::reject_protocol`.
2023-09-07 16:18:20 +02:00
Arne Welzel
2fd83acc49 Merge remote-tracking branch 'origin/topic/awelzel/3275-fix-with-python'
* origin/topic/awelzel/3275-fix-with-python:
  ci: update-alternative to have python3 be python3.9
  Add OpenSUSE Leap 15.5 to ci
  configure/CMakelists: Use Python_EXECUTABLE as hint for interpreter
2023-09-07 14:28:16 +02:00
Arne Welzel
6e6a2bee8a Merge remote-tracking branch 'origin/topic/awelzel/3278-spicy-fix-port-range-off-by-one'
* origin/topic/awelzel/3278-spicy-fix-port-range-off-by-one:
  spicy: Do not register port N+1 for port N in .evt file
2023-09-07 13:29:03 +02:00
Arne Welzel
d7b9d70fa8 ci: update-alternative to have python3 be python3.9
Tweak so that python3 is correct for the Python_FIND_UNVERSIONED_NAMES
change. python3 on 15.4 was pointing at Python3.6 still and for that
there are no dev headers installed.

A normal user could use --with-python instead, but in CI this is easier.
2023-09-07 13:03:03 +02:00
Arne Welzel
8cd2eceed1 spicy: Do not register port N+1 for port N in .evt file
Closes #3278
2023-09-07 12:58:34 +02:00
Johanna Amann
4ad2a6ab9a Add OpenSUSE Leap 15.5 to ci
This also adds 15.4 to the list of distributions that are skipped by
default - let me know if this is not a good idea. I assume that by now
most people will be running 15.5 (15.4 will be EOL in ~3 months).
2023-09-07 12:48:46 +02:00
Arne Welzel
09b7caf986 configure/CMakelists: Use Python_EXECUTABLE as hint for interpreter
The PYTHON_EXECUTABLE variable was only interpreted by our own FindPythonDev.
cmake's FindPython uses Python_EXECUTABLE instead.

Also, if --with-python is not used, prefer unversioned executables that
match the version requirement. Further, remove Python_ADDITIONAL_VERSIONS
that is interpreted by FindPythonInterp which we're not using anymore.

https://cmake.org/cmake/help/latest/module/FindPython.html#artifacts-specification
https://cmake.org/cmake/help/latest/module/FindPythonInterp.html

Closes #3275
2023-09-07 12:48:37 +02:00
Christian Kreibich
21562cfbb2 Updating auxil/zeek-client submodule [nomail] 2023-09-06 23:26:25 -07:00
zeek-bot
151a4999a3 Update doc submodule [nomail] [skip ci] 2023-09-07 00:22:26 +00:00
Benjamin Bannier
d7c3e0ac69 Propagate failure reason in spicy::decline_input/zeek::reject_protocol.
Closes #3276.
2023-09-06 10:32:11 +02:00
Arne Welzel
1441b83411 Merge remote-tracking branch 'origin/topic/awelzel/dns-mgr-fixes'
* origin/topic/awelzel/dns-mgr-fixes:
  DNS_Mgr: Use Process() for timeout expiration
  DNS_Mgr: Fix GetNextTimeout() returning absolute values
2023-09-05 19:57:10 +02:00
Arne Welzel
a57c45428f DNS_Mgr: Use Process() for timeout expiration
DNS_Mgr has a GetNextTimeout() implementation that may return 0.0. When
that is the case, its IO source is enqueued as ready with an fd of -1.
This in turn results in Process() being called instead of ProcessFd()
in RunState.cc.

Ensure timeouts behavior is properly handled by actually forwarding
timeout indications to c-ares via DNS_Mgr::Process(). This results
in pending DNS queries for which a timeout happened to actually
timeout (when there's no other connectivity).
2023-09-04 18:09:40 +02:00
Arne Welzel
62c06f7e3a DNS_Mgr: Fix GetNextTimeout() returning absolute values
Not sure, must have been some sort of left-over, but wasn't really
effective due to Process() not being implemented.
2023-09-04 17:50:15 +02:00
Arne Welzel
7e11501d3c Merge remote-tracking branch 'origin/topic/jazoff/gh-3268t '
* origin/topic/jazoff/gh-3268:
  Fix check for emailed notices

Changes: Added a test-case printing email_delay_tokens to compare email vs
non-email notice types. Previously, both notice types would have email
delay tokens at that point in the flow.
2023-09-04 14:01:23 +02:00
zeek-bot
7a04dad95c Update doc submodule [nomail] [skip ci] 2023-09-02 00:20:12 +00:00
Vern Paxson
a36c06e5cd fix for ZAM statement-level profiling (broken by GH-3199) 2023-09-01 12:20:46 -07:00
Vern Paxson
e78570ad89 ZAM fixes for compatibility with GH-3249 changes 2023-09-01 12:20:35 -07:00
Vern Paxson
80eed34985 -O gen-C++ fixes for compatibility with GH-3249 changes
minor -O gen-C++ BTest updates
2023-09-01 12:17:38 -07:00
Vern Paxson
85cd1d69a4 minor BTest reordering to diminish differences with script optimization 2023-09-01 12:17:06 -07:00
Justin Azoff
047232df37 Fix check for emailed notices
This bug was introduced in 9d0bd96
2023-09-01 13:23:45 -04:00
Arne Welzel
14a2c02f9d Merge remote-tracking branch 'origin/topic/awelzel/1705-http-pending-requests'
* origin/topic/awelzel/1705-http-pending-requests:
  http: Prevent request/response de-synchronization and unbounded state growth
2023-09-01 11:54:10 +02:00
Arne Welzel
4b71b7d05c Merge branch 'topic/awelzel/no-telemetry-log-in-external-baselines'
* topic/awelzel/no-telemetry-log-in-external-baselines:
  testing/external: Exclude telemetry.log from baselines
2023-08-30 11:25:39 +02:00
Arne Welzel
e12e873461 testing/external: Exclude telemetry.log from baselines
The diffs produced by telemetry.log when introducing a weird or
removing/adding protocol specific logs is overwhelming and distracting
without providing value. Exclude telemetry.log similar to how we already
exclude stats.log.

Some more targeted telemetry.log tests exists in the normal testing/btest
suite and that appears more sensible.
2023-08-30 10:49:46 +02:00
Tim Wojtulewicz
19719b1862 Merge remote-tracking branch 'origin/topic/neverlord/gh-3247'
* origin/topic/neverlord/gh-3247:
  Forward OpenSSL include path to plugins
2023-08-29 12:14:04 -07:00
Tim Wojtulewicz
c9f4bacd73 Revert "Merge remote-tracking branch 'origin/topic/neverlord/gh-3247'"
This reverts commit 1673f1db05, reversing
changes made to 619113f674.
2023-08-29 12:13:40 -07:00
Tim Wojtulewicz
1673f1db05 Merge remote-tracking branch 'origin/topic/neverlord/gh-3247'
* origin/topic/neverlord/gh-3247:
  Forward OpenSSL include path to plugins
2023-08-29 12:07:56 -07:00
Dominik Charousset
f60a208b8d Forward OpenSSL include path to plugins 2023-08-29 20:05:41 +02:00
Tim Wojtulewicz
619113f674 Merge remote-tracking branch 'origin/topic/timw/remove-ubuntu-22-10'
* origin/topic/timw/remove-ubuntu-22-10:
  CI: Add ubuntu 23.04 build
  CI: Remove EOL ubuntu 22.10 build
2023-08-29 09:11:42 -07:00
Tim Wojtulewicz
ac82d4bcbd CI: Add ubuntu 23.04 build 2023-08-29 09:11:07 -07:00
Tim Wojtulewicz
b551bc9fc3 CI: Remove EOL ubuntu 22.10 build 2023-08-29 09:11:07 -07:00
Arne Welzel
c1edf2e276 Merge remote-tracking branch 'origin/topic/awelzel/pre-commit-autoupdate'
* origin/topic/awelzel/pre-commit-autoupdate:
  pre-commit: autoupdate
2023-08-29 09:39:20 +02:00
Arne Welzel
f766669b73 pre-commit: autoupdate
...except for clang-format, because versions after v13.0.0 have
borked the Whitesmith formatting. Also moves yapf from
pre-commit/mirrors-yapf to google/yapf.
2023-08-29 09:38:06 +02:00
zeek-bot
f4de133c12 Update doc submodule [nomail] [skip ci] 2023-08-29 00:28:06 +00:00
Dominik Charousset
80846a7971 Forward OpenSSL include path to plugins 2023-08-28 20:25:08 +02:00
Arne Welzel
d81cb9d10e Merge remote-tracking branch 'origin/topic/awelzel/ditch-file-krb-include'
* origin/topic/awelzel/ditch-file-krb-include:
  cmake: Bump submodule for removal of NEED_KRB5_H
  zeek-config.h: Drop NEED_KRB5_H
  File: Drop krb5.h include
2023-08-28 20:08:01 +02:00
Tim Wojtulewicz
9c15a2f6a6 Merge remote-tracking branch 'origin/topic/vern/trigger-constructor'
* origin/topic/vern/trigger-constructor:
  change Trigger constructor to not potentially auto-delete itself
2023-08-28 10:54:06 -07:00
Vern Paxson
e8f4e54475 change Trigger constructor to not potentially auto-delete itself 2023-08-28 10:17:26 -07:00
Arne Welzel
af1714853f http: Prevent request/response de-synchronization and unbounded state growth
When http_reply events are received before http_request events, either
through faking traffic or possible re-ordering, it is possible to trigger
unbounded state growth due to later http_requests never being matched
again with responses.

Prevent this by synchronizing request/response counters when late
requests come in.

Also forcefully flush pending requests when http_replies are never
observed either due to the analyzer having been disabled or because
half-duplex traffic.

Fixes #1705
2023-08-28 15:02:58 +02:00
Arne Welzel
8c2a9ec5f5 Update doc, cmake submodules for installing .spicy files 2023-08-28 12:57:21 +02:00
Arne Welzel
a84d6cd66c Merge branch 'topic/awelzel/3235-dont-flip-broadcasts'
* topic/awelzel/3235-dont-flip-broadcasts:
  testing: Bump external test suite
  dhcp: Handle is_orig=T for connections from server to 255.255.255.255
  IPBasedAnalyzer: Don't flip connections when destination is broadcast
2023-08-28 12:20:18 +02:00
Arne Welzel
4cabcd43c6 testing: Bump external test suite 2023-08-28 12:15:58 +02:00
Arne Welzel
fc768a9e01 dhcp: Handle is_orig=T for connections from server to 255.255.255.255
This works around the new semantics of is_orig=T for "connections"
from DHCP servers to broadcast addresses. IMO, having the server address
as originator in the conn.log is still more intuitive.
2023-08-28 12:15:55 +02:00
Arne Welzel
ba04f4c31d IPBasedAnalyzer: Don't flip connections when destination is broadcast
Closes #3235
2023-08-28 12:15:55 +02:00