Commit graph

16709 commits

Author SHA1 Message Date
Arne Welzel
3846db6ccf ldap: Reintroduce encryption after SASL heuristic
@dopheide-esnet provided sample captures where SASL SRP is used as
a SASL mechanism and the follow-up LDAP messages are encrypted. It's
not clear how to determine whether encryption will or will not happen,
so re-add a heuristic to determine this based on the first byte of
the first message *after* the successful bindResponse handshake. If
that byte is 0x30, assume cleartext.

I haven't been able to produce such pcaps, unfortunately, but the
cleartext path is tested via the existing sasl-ntlm.pcap.
2024-07-23 11:29:00 +02:00
Arne Welzel
724c08f286 ldap: Fix assuming GSS-SPNEGO for all bindResponses
In retrospect that's an obvious bug.
2024-07-23 11:29:00 +02:00
Arne Welzel
09a48c7028 ldap: Implement extended request/response and StartTLS support
PCAP was produced with a local OpenLDAP server configured to support StartTLS.

This puts the Zeek calls into a separate ldap_zeek.spicy file/module
to separate it from LDAP.
2024-07-23 11:29:00 +02:00
Arne Welzel
f4a79fa703 Merge remote-tracking branch 'origin/topic/awelzel/deprecate-prometheus-script'
* origin/topic/awelzel/deprecate-prometheus-script:
  telemetry: Deprecate prometheus.zeek policy script
2024-07-23 10:03:13 +02:00
Tim Wojtulewicz
f500c424fe Update broker submodule [nomail] 2024-07-22 14:59:17 -07:00
Arne Welzel
bf9704f339 telemetry: Deprecate prometheus.zeek policy script
With Cluster::Node$metrics_port being optional, there's not really
a need for the extra script. New rule, if a metrics_port is set, the
node will attempt to listen on it.

Users can still redef Telemetry::metrics_port *after*
base/frameworks/telemetry was loaded to change the port defined
in cluster-layout.zeek.
2024-07-21 17:49:21 +02:00
Arne Welzel
e982a18792 Merge remote-tracking branch 'origin/topic/awelzel/3836-input-create-any-abort-crash'
* origin/topic/awelzel/3836-input-create-any-abort-crash:
  input/Manager: Improve type checks of record fields with type any
2024-07-19 19:51:52 +02:00
Arne Welzel
3797622152 input/Manager: Improve type checks of record fields with type any
Calling AsRecordType() or AsFunc() on a Val of type any isn't safe.

Closes #3836
2024-07-19 11:35:54 +02:00
Christian Kreibich
0ba80d13b4 Merge branch 'topic/christian/bump-cluster-testsuite'
* topic/christian/bump-cluster-testsuite:
  Bump zeek-testing-cluster to pull in tee SIGPIPE fix
2024-07-17 15:37:29 -07:00
Christian Kreibich
b51a46f94d Bump zeek-testing-cluster to pull in tee SIGPIPE fix 2024-07-17 15:34:24 -07:00
Arne Welzel
2ea3a651bd Merge remote-tracking branch 'origin/topic/awelzel/spicy-ldap-krb-wrap-tokens'
* origin/topic/awelzel/spicy-ldap-krb-wrap-tokens:
  ldap: Remove MessageWrapper with magic 0x30 searching
  ldap: Harden parsing a bit
  ldap: Handle integrity-only KRB wrap tokens
2024-07-17 16:48:58 +02:00
Arne Welzel
e7aca5b388 ldap: Remove MessageWrapper with magic 0x30 searching
This unit implements a heuristic to search for the 0x30 sequence
byte if Message couldn't readily be parsed. Remove it with the
idea of explicit and predictable support for SASL mechanisms.
2024-07-17 15:28:27 +02:00
Arne Welzel
0cab87c185 ldap: Harden parsing a bit
ASN1Message(True) may go off parsing arbitrary input data as
"something ASN.1" This could be GBs of octet strings or just very
long sequences. Avoid this by open-coding some top-level types expected.

This also tries to avoid some of the &parse-from usages that result
in unnecessary copies of data.

Adds a locally generated PCAP with addRequest/addResponse that we
don't currently handle.
2024-07-17 15:27:14 +02:00
Arne Welzel
31122f335f ldap: Handle integrity-only KRB wrap tokens
Mostly staring at the PCAPs and opened a few RFCs. For now, only if the
MS_KRB5 OID is used and accepted in a bind response, start stripping
KRB5 wrap tokens for both, client and server traffic.

Would probably be nice to forward the GSS-API data to the analyzer...

Closes zeek/spicy-ldap#29.
2024-07-17 15:26:30 +02:00
Tim Wojtulewicz
9ba7c2ddaf Merge remote-tracking branch 'origin/topic/bbannier/bump-spicy'
* origin/topic/bbannier/bump-spicy:
  Bump auxil/spicy to latest development snapshot
2024-07-16 10:16:15 -07:00
Benjamin Bannier
41db05238b Bump auxil/spicy to latest development snapshot
This patch bump Spicy to the latest development snapshot. This
introduces a backwards-incompatible change in that it removes support
for a never officially supported syntax to specify unit fields (so I
would argue: not strictly a breaking change).
2024-07-16 08:17:31 +02:00
Tim Wojtulewicz
cb56837f07 Merge remote-tracking branch 'origin/topic/timw/fetchcontent_mkdir'
* origin/topic/timw/fetchcontent_mkdir:
  CI: Set FETCH_CONTENT_FULLY_DISCONNECTED flag for configure
  Update broker and cmake submodules [nomail]
2024-07-12 16:02:37 -07:00
Tim Wojtulewicz
4757536d7d CI: Set FETCH_CONTENT_FULLY_DISCONNECTED flag for configure 2024-07-12 15:14:31 -07:00
Tim Wojtulewicz
a46be1ea29 Update broker and cmake submodules [nomail] 2024-07-12 15:14:31 -07:00
Tim Wojtulewicz
f3bcf1a55d Fix a broken merge
I merged an old version of the branch on accident and then merged the right
one over top of it, but git ended up including both versions. This fixes
that mistake.
2024-07-12 10:00:13 -07:00
Tim Wojtulewicz
b935d2f59a Merge remote-tracking branch 'origin/topic/bbannier/lib-spicy-hooks'
* origin/topic/bbannier/lib-spicy-hooks:
  Do not emit hook files for builtin modules
2024-07-12 09:51:55 -07:00
Tim Wojtulewicz
7a38cee81f Merge remote-tracking branch 'origin/topic/bbannier/lib-spicy-hooks'
* origin/topic/bbannier/lib-spicy-hooks:
  Do not emit hook files for builtin modules
2024-07-12 09:48:12 -07:00
Tim Wojtulewicz
e4716b6c91 Merge remote-tracking branch 'origin/topic/timw/grealpath-make-dist-warning'
* origin/topic/timw/grealpath-make-dist-warning:
  Fix warning about grealpath when running 'make dist' on Linux
2024-07-12 09:46:51 -07:00
Benjamin Bannier
24d3454d61 Do not emit hook files for builtin modules
We would previously emit a C++ file with hooks for at least the builtin
`spicy` module even though that module like any other builtin module
never contains implementations of hooks for types in user code.

This patch prevents modules with skipped implementations (such as our
builtin modules) from being added to the compilation which prevents
generating their hook files.
2024-07-12 09:06:12 +02:00
Tim Wojtulewicz
5fd563da79 Fix warning about grealpath when running 'make dist' on Linux 2024-07-11 13:45:14 -07:00
Tim Wojtulewicz
822102382c Start of 7.1.0 development 2024-07-11 13:19:15 -07:00
Tim Wojtulewicz
f124b31904 Updating submodule(s) [nomail] 2024-07-11 12:21:02 -07:00
Christian Kreibich
c4bc9078ef Merge branch 'topic/christian/broker-prometheus-cpp'
* topic/christian/broker-prometheus-cpp:
  Update the scripts.base.frameworks.telemetry.internal-metrics test
  Revert "Temporarily disable the scripts/base/frameworks/telemetry/internal-metrics btest"
  Bump Broker to pull in new Prometheus support and pass in Zeek's registry
2024-07-11 12:15:16 -07:00
Christian Kreibich
77816f9a6b Update the scripts.base.frameworks.telemetry.internal-metrics test
This now uses different record fields, and for now we no longer have CAF
telemetry. We indicate we're running under test to get reliable ordering in the
baselined output.
2024-07-11 12:13:05 -07:00
Christian Kreibich
b387da3489 Revert "Temporarily disable the scripts/base/frameworks/telemetry/internal-metrics btest"
This reverts commit d6e97ab306.
Broker's telemetry is now available again.
2024-07-11 12:13:05 -07:00
Dominik Charousset
40e1e43c25 Bump Broker to pull in new Prometheus support and pass in Zeek's registry
Co-authored-by: Christian Kreibich <christian@corelight.com>
2024-07-11 12:13:05 -07:00
Benjamin Bannier
596f4114fc Do not emit hook files for builtin modules
We would previously emit a C++ file with hooks for at least the builtin
`spicy` module even though that module like any other builtin module
never contains implementations of hooks for types in user code.

This patch adds a blocklist of builtin modules which are skipped for
generating hook files.
2024-07-11 14:15:21 +02:00
zeek-bot
e99b94c18f Update doc submodule [nomail] [skip ci] 2024-07-11 00:10:52 +00:00
Tim Wojtulewicz
3f475ebf7d Bump cmake submodule [nomail] 2024-07-10 12:37:21 -07:00
Arne Welzel
a9c4daaa22 Merge remote-tracking branch 'jgras/topic/jgras/log-disable-analyzer'
* jgras/topic/jgras/log-disable-analyzer:
  Extend btest for logging of disabled analyzers
  Add logging of disabled analyzers to analyzer.log
2024-07-10 18:51:26 +02:00
Arne Welzel
1c8fedaf09 Merge remote-tracking branch 'origin/topic/awelzel/move-hilti-jit-parallelism-to-btest-cfg'
* origin/topic/awelzel/move-hilti-jit-parallelism-to-btest-cfg:
  testing/btest: Default to HILTI_JIT_PARALLELISM=1
  Revert "CI: Use ccache and a single CPU when building spicy analyzers for btests"
2024-07-10 11:06:46 +02:00
Arne Welzel
7b99fc01a9 testing/btest: Default to HILTI_JIT_PARALLELISM=1
This is a rework of b59bed9d06 moving
HILTI_JIT_PARALLELISM=1 into btest.cfg to make it default applicable to
btest -j users (and CI).

The background for this change is that spicyz may spawn up to nproc compiler
instances by default. Combined with btest -j, this may be nproc x nproc
instances worst case. Particularly with gcc, this easily overloads CI or
local systems, putting them into hard-to-recover-from thrashing/OOM states.

Exporting HILTI_JIT_PARALLELISM in the shell allows overriding.
2024-07-10 11:04:47 +02:00
Arne Welzel
cb500536ea Revert "CI: Use ccache and a single CPU when building spicy analyzers for btests"
This reverts commit b59bed9d06.
2024-07-10 10:02:53 +02:00
zeek-bot
b995924b21 Update doc submodule [nomail] [skip ci] 2024-07-10 00:21:07 +00:00
Tim Wojtulewicz
b309c423f8 Merge remote-tracking branch 'origin/topic/timw/ci-reduce-spicy-test-load'
* origin/topic/timw/ci-reduce-spicy-test-load:
  CI: Use ccache and a single CPU when building spicy analyzers for btests
2024-07-09 15:12:29 -07:00
Tim Wojtulewicz
b64f0e8fd8 Merge remote-tracking branch 'origin/topic/bbannier/bump-spicy'
* origin/topic/bbannier/bump-spicy:
  Bump auxil/spicy to latest development snapshot
2024-07-09 14:53:49 -07:00
Tim Wojtulewicz
b59bed9d06 CI: Use ccache and a single CPU when building spicy analyzers for btests 2024-07-09 14:31:09 -07:00
Jan Grashoefer
c6c8d078c0 Extend btest for logging of disabled analyzers 2024-07-09 20:15:46 +02:00
Tim Wojtulewicz
c557b2156a Merge remote-tracking branch 'origin/topic/vern/script-opt-maint.Jul24'
* origin/topic/vern/script-opt-maint.Jul24:
  minor script optimization updates to reflect recent changes, Coverity findings
2024-07-09 10:15:26 -07:00
Tim Wojtulewicz
03e17a530a Update zeek-aux submodule [nomail] 2024-07-09 09:33:29 -07:00
Jan Grashoefer
0c06c604ab Add logging of disabled analyzers to analyzer.log 2024-07-09 18:22:43 +02:00
Benjamin Bannier
fcded0fc6a Bump auxil/spicy to latest development snapshot 2024-07-09 18:22:21 +02:00
Arne Welzel
e102715bc8 Merge remote-tracking branch 'cknill/topic/cknill/display_cmake_fix'
* cknill/topic/cknill/display_cmake_fix:
  Fix for --display-cmake in configure Moved build directory creation further down in the script so that --display-cmake has a chance to happen before build tree setup.
2024-07-09 17:10:28 +02:00
Christian Kreibich
6552c4e0fe Merge branch 'topic/christian/management-telemetry-additions'
* topic/christian/management-telemetry-additions:
  Management framework: bump cluster testsuite to pull in telemetry tests
  Management framework: bump zeek-client
  Management framework: augment deployed configs with instance IP addresses
  Management framework: add auto-enumeration of metrics ports
  Management framework: propagate metrics port from agent
  Management framework: add metrics port in management & Supervisor node records
  Harden the telemetry manager against unset Telemetry::metrics_address
  Comment-only tweaks for telemetry-related settings.
2024-07-09 00:08:28 -07:00
Christian Kreibich
cdd5062f45 Management framework: bump cluster testsuite to pull in telemetry tests 2024-07-08 23:05:24 -07:00