Commit graph

16999 commits

Author SHA1 Message Date
Arne Welzel
6a930c1cf8 teredo: Move conn member from analyzer to encapsulation
There's only a single instance of the Teredo analyzer. Mutating the conn
member for every new packet and leaving it set after processing the
packet is confusing. Move conn into TeredoEncapsulation instead, or pass
it explicitly.
2024-09-17 18:13:33 +02:00
Tim Wojtulewicz
ba91de59b0 Merge remote-tracking branch 'mp/topic/mp-corelight/update-docker-env-syntax'
* mp/topic/mp-corelight/update-docker-env-syntax:
  Update Dockerfile to avoid LegacyKeyValueFormat warning
2024-09-12 09:18:56 +02:00
Tim Wojtulewicz
923ca7e817 Merge remote-tracking branch 'origin/topic/timw/strreplace-endless-loop'
* origin/topic/timw/strreplace-endless-loop:
  Fix potential endless loop in util::strreplace
2024-09-12 08:59:20 +02:00
Johanna Amann
899f7297d7 Merge remote-tracking branch 'origin/topic/timw/remove-negative-timestamp-test'
* origin/topic/timw/remove-negative-timestamp-test:
  Remove core.negative-time btest
2024-09-12 08:53:00 +02:00
Johanna Amann
a0ff0fb18b Merge remote-tracking branch 'origin/topic/johanna/reformat-spicy-ssl'
* origin/topic/johanna/reformat-spicy-ssl:
  Bump spicy-format to 0.16.2
  Bump spicy-format to 0.16.1
  Spicy SSL: reformat with new version of spicy format
2024-09-12 08:51:22 +02:00
MP-Corelight
eed7982154
Update Dockerfile to avoid LegacyKeyValueFormat warning 2024-09-11 22:17:46 +02:00
Johanna Amann
0ef7a4e5dc Bump spicy-format to 0.16.2 2024-09-11 17:58:13 +02:00
Johanna Amann
c9e84883d6 Bump spicy-format to 0.16.1 2024-09-11 17:08:16 +02:00
Johanna Amann
86c542fd47 Spicy SSL: reformat with new version of spicy format 2024-09-11 17:06:01 +02:00
Johanna Amann
743e2a3663 Merge remote-tracking branch 'origin/topic/johanna/spicy-tls'
* origin/topic/johanna/spicy-tls: (31 commits)
  Spicy SSL: don't turn on compilation of Spicy SSL analyzer in all cases
  Spicy SSL analyzer - address feedback
  Spicy TLS: re-enable test and update test comments
  Spicy SSL: Skip private test suite for now.
  Spicy SSL - finish SSLv2 implementation
  Spicy TLS: SSLv2 client hello support.
  Spicy TLS - full test suite pass
  Spicy TLS - refactoring and partial connection fix
  Spicy SSL - run spicy-format
  Spicy-TLS: address review feedback re convert-functions
  Spicy TLS: address review feedback & run spicy-lint
  Spicy TLS: disable debug prints
  Add configure option to enable/disable spicy SSL
  Spicy TLS: inout/reference updates for recent spicy changes
  Spicy TLS: parse OCSP replies in TLS extension
  Spicy TLS: generate same file IDs as binpac analyzer
  Spicy TLS: skip CI; run performance tests
  Spicy TLS: fix parsing of no-extension hellos, port registration
  Spicy TLS: Re-enable starttls for RDP
  Spicy TLS: rename the analyzer to SSL
  ...
2024-09-11 16:59:17 +02:00
Johanna Amann
94f93d0704 Merge remote-tracking branch 'origin/master' into topic/johanna/spicy-tls
* origin/master: (27 commits)
  Update doc submodule [nomail] [skip ci]
  btest/ldap: Add regression test for #3919
  postgresql: Simplify SSL buffering and forwarding
  postgresql: Initial parser implementation
  testing/external: Update private baselines
  analyzer/syslog: Reformat with spicy-format
  analyzer/finger: Reformat with spicy-format
  scripts/spicy: Reformat with spicy-format
  pre-commit: Add spicy-format
  Check for netbios to avoid reporting extra bad DNS opcodes
  Add weird for unhandled opcodes in DNS analyzer
  Bump zeek-aux for zeek/zeek-aux#57
  Remove pre-commit exclusions for clang-format
  Bump clang-format
  Bump auxil/spicy to latest development snapshot
  RunState: Drop broker_mgr->Active() usage
  script_opt/ZAM/IterInfo.h: Add missing Dict.h dependency
  script_opt/ZAM: ZBody.h / Support.h: Cleanup includes, use forward declarations
  script_opt/ZAM/Profile: Remove Zeek header includes
  script_opt: Extend Support.h to break include dependencies
  ...
2024-09-11 14:13:19 +02:00
Johanna Amann
fc5047be94 Spicy SSL: don't turn on compilation of Spicy SSL analyzer in all cases
Turned out to be more complex than hoped.
2024-09-11 12:10:59 +02:00
Johanna Amann
cf1074518e Spicy SSL analyzer - address feedback
Minor proposed changes from https://github.com/zeek/zeek/pull/3765,
mostly cosmetic.

Changes CI to be part of an already existing spicy CI job.
2024-09-11 11:30:16 +02:00
Tim Wojtulewicz
6b11252b8f Fix potential endless loop in util::strreplace 2024-09-11 11:21:25 +02:00
Tim Wojtulewicz
50ce13b201 Remove core.negative-time btest 2024-09-11 10:14:34 +02:00
Arne Welzel
ede4e661cd Merge remote-tracking branch 'origin/topic/awelzel/script-opt-less-includes'
* origin/topic/awelzel/script-opt-less-includes:
  script_opt/ZAM/IterInfo.h: Add missing Dict.h dependency
  script_opt/ZAM: ZBody.h / Support.h: Cleanup includes, use forward declarations
  script_opt/ZAM/Profile: Remove Zeek header includes
  script_opt: Extend Support.h to break include dependencies
  script_opt: Do not include ZBody.h in ZAM/Validate.cc
2024-09-09 13:25:33 +02:00
zeek-bot
3dfd20ef50 Update doc submodule [nomail] [skip ci] 2024-09-07 00:11:33 +00:00
Arne Welzel
a339cfa4c0 Merge remote-tracking branch 'origin/topic/awelzel/3919-ldap-logs-missing'
* origin/topic/awelzel/3919-ldap-logs-missing:
  btest/ldap: Add regression test for #3919
2024-09-06 19:17:49 +02:00
Arne Welzel
26c4d0df8b btest/ldap: Add regression test for #3919
This works as expected in master, it's just that we forgot to backport
PR #3845 to 7.0.1. Add the PCAP from Martin anyhow.

Closes #3919.
2024-09-06 19:16:49 +02:00
Arne Welzel
b9ca50c6bb Merge remote-tracking branch 'origin/topic/awelzel/add-postgresql-parser'
* origin/topic/awelzel/add-postgresql-parser:
  postgresql: Simplify SSL buffering and forwarding
  postgresql: Initial parser implementation
2024-09-06 17:47:52 +02:00
Arne Welzel
d0da13413b postgresql: Simplify SSL buffering and forwarding
The ssl_sink can only be connected form the backend side, so don't
overcomplicate it.
2024-09-06 16:11:19 +02:00
Arne Welzel
85ca59484b postgresql: Initial parser implementation
This adds a protocol parser for the PostgreSQL protocol and a new
postgresql.log similar to the existing mysql.log.

This should be considered preliminary and hopefully during 7.1 and 7.2
with feedback from the community, we can improve on the events and logs.
Even if most PostgreSQL communication is encrypted in the real-world, this
will minimally allow monitoring of the SSLRequest and hand off further
analysis to the SSL analyzer.

This originates from github.com/awelzel/spicy-postgresql, with lots of
polishing happening in the past two days.
2024-09-06 16:10:48 +02:00
Arne Welzel
2907d9feee Merge remote-tracking branch 'origin/topic/awelzel/add-spicy-format'
* origin/topic/awelzel/add-spicy-format:
  testing/external: Update private baselines
  analyzer/syslog: Reformat with spicy-format
  analyzer/finger: Reformat with spicy-format
  scripts/spicy: Reformat with spicy-format
  pre-commit: Add spicy-format
2024-09-05 19:19:49 +02:00
Arne Welzel
f35835c8f9 testing/external: Update private baselines 2024-09-05 19:11:05 +02:00
Arne Welzel
526c903c01 analyzer/syslog: Reformat with spicy-format 2024-09-05 19:11:05 +02:00
Arne Welzel
dae4b1fcd9 analyzer/finger: Reformat with spicy-format 2024-09-05 19:11:05 +02:00
Arne Welzel
cde5662779 scripts/spicy: Reformat with spicy-format 2024-09-05 19:11:05 +02:00
Arne Welzel
01ea74f8a8 pre-commit: Add spicy-format
This excludes the larger protocol analyzers. These should be formatted
during the 7.2 cycle, shortly before 8.0.
2024-09-05 19:11:05 +02:00
Tim Wojtulewicz
d4a646cd21 Merge remote-tracking branch 'origin/topic/timw/dns-opcode-check'
* origin/topic/timw/dns-opcode-check:
  Check for netbios to avoid reporting extra bad DNS opcodes
  Add weird for unhandled opcodes in DNS analyzer
2024-09-04 14:53:31 -07:00
Tim Wojtulewicz
6394f9893e Check for netbios to avoid reporting extra bad DNS opcodes 2024-09-04 13:07:00 -07:00
Tim Wojtulewicz
269ca3189c Add weird for unhandled opcodes in DNS analyzer 2024-09-04 13:01:24 -07:00
Tim Wojtulewicz
cd10bd3747 Merge remote-tracking branch 'origin/topic/bbannier/bump-spicy'
* origin/topic/bbannier/bump-spicy:
  Bump auxil/spicy to latest development snapshot
2024-09-03 09:04:03 -07:00
Benjamin Bannier
dffef94d4d Merge branch 'topic/bbannier/bump-clang-format' 2024-09-03 10:18:39 +02:00
Arne Welzel
cdae755a0c Bump zeek-aux for zeek/zeek-aux#57 2024-09-03 10:03:22 +02:00
Benjamin Bannier
629d273f44 Remove pre-commit exclusions for clang-format
We previously would ignore any files under `auxil/`. Since pre-commit
only formats files tracked in this repo, and `auxil/` today contains
only submodules this exclusion did not exclude anything; if however we
ever end up adding C++ files under `auxil/` we would like them to be
formatted consistently.
2024-09-02 16:49:07 +02:00
Benjamin Bannier
db42b2cfe5 Bump clang-format
This patch contains a bump of the configured clang-format version from
17.0.3 to 18.1.8 and automatically generated C++ source updates. The
main difference we are seeing from this is fixes for previously
incomplete reformats.
2024-09-02 16:46:59 +02:00
Benjamin Bannier
06d2568c58 Bump auxil/spicy to latest development snapshot 2024-09-02 16:36:27 +02:00
Arne Welzel
c27e18631c Merge branch 'fix-http-password-capture' of https://github.com/p-l-/zeek
* 'fix-http-password-capture' of https://github.com/p-l-/zeek:
  http: fix password capture when enabled
2024-08-30 18:56:48 +02:00
Arne Welzel
435770f99a Merge remote-tracking branch 'origin/topic/awelzel/no-broker-mgr-active'
* origin/topic/awelzel/no-broker-mgr-active:
  RunState: Drop broker_mgr->Active() usage
2024-08-30 15:39:06 +02:00
Arne Welzel
f6fdd16b81 Merge remote-tracking branch 'origin/topic/awelzel/less-threading-star-star'
* origin/topic/awelzel/less-threading-star-star:
  logging/WriterFrontend: No need for explicit CleanupWriteBuffer()
  logging: Switch index-assignment of raw pointers to emplace_back()
  broker/logging: Change threading::Value** usage std::vector instead
  threading/Value: Support move and copy constructors
2024-08-30 15:38:47 +02:00
Arne Welzel
a9e4a45310 RunState: Drop broker_mgr->Active() usage
...and deprecate it. The communication_enabled flag doesn't really
make a difference in how the IO loop works.
2024-08-30 15:36:56 +02:00
Arne Welzel
c9ff8d0312 script_opt/ZAM/IterInfo.h: Add missing Dict.h dependency
Triggered when trying to remove some other includes elsewhere.
2024-08-30 14:20:17 +02:00
Arne Welzel
d843eb5c2b script_opt/ZAM: ZBody.h / Support.h: Cleanup includes, use forward declarations
Not sure the IsAny() convenience functions were really worth the
include zeek/Expr.h.
2024-08-30 14:20:17 +02:00
Arne Welzel
29536d6453 script_opt/ZAM/Profile: Remove Zeek header includes
This can stand by itself.
2024-08-30 14:19:26 +02:00
Arne Welzel
4e7bb59bb1 script_opt: Extend Support.h to break include dependencies
Rebuilding ZBody.cc is annoyingly slow with UBSAN or ASAN enabled. A rebuild
is currently triggered when touching Manager.h files like logging/Manager.h,
broker/Manager.h or file_analysis/Manager.h and various other headers that are
included from there or directly in ZBody.h.

Add tiny wrapper functions to Support.h that encapsulate the actual Zeek
functionality and move the includes to Support.cc instead. This will
cause frequent Support.cc rebuilds, but that is decently fast.

This comes at the expense of one indirect function call for some ops, but
its overhead should be negligible in comparison to a lookup for a
connection or component or a log_mgr->Write() call.
2024-08-30 14:19:26 +02:00
Arne Welzel
f0ab10a46c logging/WriterFrontend: No need for explicit CleanupWriteBuffer()
Any pending records will be cleaned in the destructor of WriterFrontend
and WriteBuffer, no need to do this explicitly.
2024-08-30 11:00:17 +02:00
Arne Welzel
a9290cc031 logging: Switch index-assignment of raw pointers to emplace_back() 2024-08-30 10:59:55 +02:00
Arne Welzel
245fd0c94f broker/logging: Change threading::Value** usage std::vector instead
This allows to leverage automatic memory management, less allocations
and using move semantics for expressing ownership.

This breaks the existing logging and broker API, but keeps the plugin
DoWrite() and HookLogWrite() methods functioning.

It further changes ValToLogVal to return a threading::Value rather than
a threading::Value*. The vector_val and set_val fields unfortunately
use the same pointer-to-array-of-pointers approach. this can'tbe changed
as it'd break backwards compatibility for plugin provided input readers
and log writers.
2024-08-30 10:58:57 +02:00
Arne Welzel
e79ce27c9f threading/Value: Support move and copy constructors 2024-08-30 10:58:23 +02:00
Arne Welzel
bbd7adc0dc script_opt: Do not include ZBody.h in ZAM/Validate.cc
Touching broker/Manager.h or logging/Manager.h or anything that's
included in ZBody.h results in a rebuild of a bunch of files in ZAM.
Particularly, ZAM/Validate.cc takes much too long to recompile for
me (45+ seconds) and apparently that's mostly by accident.

Clean up the ZAM/Validate.cc includes to prevent this.
2024-08-29 10:21:56 +02:00