Commit graph

17166 commits

Author SHA1 Message Date
Arne Welzel
aabc4a4114 sumstats: Remove copy() for Broker::publish() calls
Serialization happens immediately at Broker::publish() time, there
should be no caching issues.
2024-11-14 12:59:22 +01:00
Arne Welzel
831614f907 broker/Publish: Use event time instead of network time
Discussed with @J-Gras, calling Broker::publish() within a scheduled
should use the "intended timestamp" implicitly.

This is subtle, but supposedly more expected when running
a pcap replay cluster.
2024-11-14 12:59:22 +01:00
Arne Welzel
6abb9d7eda broker/Eventhandler: Deprecate Broker::auto_publish() for v8.1
Relates to #3637
2024-11-14 12:59:22 +01:00
Arne Welzel
455e05bc2e btest: Remove Broker::auto_publish() usages
The ones that seemed to test Broker::auto_publish() were annotated
for removal.
2024-11-14 12:59:22 +01:00
Arne Welzel
927e936653 frameworks/control: Remove Broker::auto_publish() 2024-11-14 12:59:22 +01:00
Arne Welzel
6aca4d1dc7 catch-and-release: Remove Broker::auto_publish() 2024-11-14 12:59:22 +01:00
Arne Welzel
44c4a91cc8 ssl/validate-certs: Remove Broker::auto_publish() 2024-11-14 12:59:22 +01:00
Arne Welzel
883ae3694c sumstats: Remove Broker::auto_publish() 2024-11-14 12:59:22 +01:00
Arne Welzel
416887157c cluster_started: No Broker::auto_publish() use 2024-11-14 12:59:22 +01:00
Arne Welzel
b32153037a openflow: Remove Broker::auto_publish() 2024-11-14 12:59:22 +01:00
Arne Welzel
cb10852f99 dhcp: Remove Broker::auto_publish()
This isn't prettier, but neither worse IMO. A test would be good.
2024-11-14 12:59:22 +01:00
Arne Welzel
08f2198d3e frameworks/notice: Remove Broker::auto_publish() 2024-11-14 12:59:22 +01:00
Arne Welzel
219d621234 netcontrol: Replace Broker::auto_publish()
I'd think we could drop the cluster.zeek and non-cluster.zeek and
just unconditionally do the publish(), but keeping it for now.
2024-11-06 15:27:48 +01:00
Arne Welzel
93478a246e intel: Switch to Cluster::publish()
This isn't quite making things a lot nicer, but more explicit.
2024-11-06 15:27:48 +01:00
Arne Welzel
91c03cd988 broker: Support publish() of unspecified set() / table()
Calling Broker::make_event(..., table()) or Broker::publish(..., table())
caused runtime type errors as same_type() would result in false.
2024-11-06 15:27:26 +01:00
Arne Welzel
189565d131 types: Fix table() resulting in table_type->IsSet() == true
typename(table()) apparently always resulted in a set[] type
being rendered. Make the yield type of an unspecified table
ANY so that type->IsSet() ends up false.

While at it, also render unspecified types as table(), set() and
vector() rather than vector of void, set[] or table[] of any which
IMO should help to figure out what's going.
2024-11-06 13:28:57 +01:00
Benjamin Bannier
e4dbba20a4 Merge remote-tracking branch 'origin/topic/bbannier/bump-spicy' 2024-11-06 11:57:01 +01:00
Benjamin Bannier
f57955321e Make Spicy hook execution order deterministic in test 2024-11-06 09:42:33 +01:00
Benjamin Bannier
f47a089576 Bump auxil/spicy to latest development snapshot 2024-11-06 09:26:00 +01:00
Benjamin Bannier
2e8d6e86e7 Merge remote-tracking branch 'origin/topic/bbannier/spicy-cookie-nullptr-deref' 2024-11-05 17:39:12 +01:00
Benjamin Bannier
93c201df57 Fix potential nullptr deref in Spicy plugin runtime
If we added a file but the other side of the connection had already run
into a protocol violation and shut down we could previously have
dereferenced a null cookie. This patch fixes the code so it now throws
in such scenarios.
2024-11-05 16:48:35 +01:00
Benjamin Bannier
0028ba320d Assume no Spicy cookie in most places
We would previously assert that it was available which could have lead
to aborts since when the analyzer for either side of a connection shuts
down the connection cookie could get cleared and become nil. This patch
reworks the code slightly so we now never assume it is available. We do
this by either throwing or by making the whole operation requesting the
cookie a noop.
2024-11-05 16:48:35 +01:00
Arne Welzel
a3a0155825 Merge remote-tracking branch 'origin/topic/awelzel/sqli-test-followup'
* origin/topic/awelzel/sqli-test-followup:
  btest: Add detect-sqli test script
2024-11-04 19:35:34 +01:00
Arne Welzel
c171d9e8c2 Merge remote-tracking branch 'origin/topic/vern/canonicalize-std-string'
* origin/topic/vern/canonicalize-std-string:
  switched CPPCompile::Canonicalize() to take std::string instead of const char*
2024-11-04 18:47:18 +01:00
Arne Welzel
b69222d5f6 btest: Add detect-sqli test script
...I added the baselines during the merge, just not the test itself.
2024-11-04 18:42:55 +01:00
Vern Paxson
519fec2592 switched CPPCompile::Canonicalize() to take std::string instead of const char* 2024-11-04 08:22:37 -08:00
Arne Welzel
8935dc9ed4 Merge remote-tracking branch 'origin/topic/vern/script-opt-fnv1a64'
* origin/topic/vern/script-opt-fnv1a64:
  minor ZAM BTest updates for recently added fnv1a64 BiF
2024-11-04 16:33:21 +01:00
Arne Welzel
db497d1da7 Merge remote-tracking branch 'origin/topic/awelzel/pattern-matches-rework'
* origin/topic/awelzel/pattern-matches-rework:
  RuleMatcher: Use a single list for tracking pattern_matches
  ZeekString: Implement move constructor
2024-11-04 16:32:48 +01:00
Arne Welzel
e443624c32 RuleMatcher: Use a single list for tracking pattern_matches 2024-11-04 16:17:49 +01:00
Arne Welzel
0ecbd4435e ZeekString: Implement move constructor 2024-11-04 16:17:47 +01:00
Benjamin Bannier
8fc5b7fcda Merge remote-tracking branch 'origin/topic/bbannier/bump-spicy-format' 2024-11-04 09:43:28 +01:00
Benjamin Bannier
b829b2165e Bump and reenable spicy-format pre-commit hook 2024-11-04 07:25:44 +01:00
Vern Paxson
d5c820d2c5 minor ZAM BTest updates for recently added fnv1a64 BiF 2024-11-03 14:05:19 -08:00
Johanna Amann
84d8e74884 Merge remote-tracking branch 'origin/topic/johanna/snap-tcp-test'
* origin/topic/johanna/snap-tcp-test:
  Add test for TCP over 802.3/SNAP
2024-10-31 16:17:31 +00:00
Johanna Amann
807bb142a7 Add test for TCP over 802.3/SNAP
This admittedly is a quite esoteric combination of protocols. But - as
we do correctly support them, it seems nice to have a slightly more
complete testcase that covers this.
2024-10-31 14:37:44 +00:00
zeek-bot
56514d98d6 Update doc submodule [nomail] [skip ci] 2024-10-31 00:14:49 +00:00
Arne Welzel
7b3bc891f4 Merge remote-tracking branch 'origin/topic/etyp/ubuntu-clang-ci'
Modified ubuntu24_clang_libcpp_task to be skipped on PRs.

* origin/topic/etyp/ubuntu-clang-ci:
  CI: Upgrade FreeBSD 14 to 14.1
  CI: Add Ubuntu Clang task
  Fix up minor warnings in touched files
  Fix Clang 19 deprecation failure
2024-10-30 14:54:08 +01:00
Arne Welzel
9fa4d6f5ed pre-commit-config: Disable spicy-format
This fails to install currently: bbannier/spicy-format#157
2024-10-30 14:43:42 +01:00
Arne Welzel
fd0181c2cb Merge branch 'modbus-fc-fixes' of https://github.com/zambo99/zeek
* 'modbus-fc-fixes' of https://github.com/zambo99/zeek:
  Fix issues with Modbus message logging
2024-10-30 14:38:53 +01:00
Arne Welzel
00a0f9c920 Merge remote-tracking branch 'origin/topic/awelzel/data-end-offset-v1'
* origin/topic/awelzel/data-end-offset-v1:
  signatures: Add data_end_offset to signature_match() and custom events
  Add pattern_end_offset to signature_state
2024-10-30 14:34:08 +01:00
Evan Typanski
8b64d12167 CI: Upgrade FreeBSD 14 to 14.1 2024-10-30 09:04:24 -04:00
Evan Typanski
04c3c2ec72 CI: Add Ubuntu Clang task
This feels more like a band-aid solution - it still requires very manual
intervention to get the next builds, but at least this way we don't rely
on whatever Clang/libc++ happens to ship with FreeBSD for the only
libc++ testing. It seems like sanitizer builds don't use libc++, but I'd
rather a dedicated Linux/Clang/libc++ task at least.
2024-10-30 09:04:24 -04:00
Evan Typanski
36af0591a6 Fix up minor warnings in touched files 2024-10-30 08:47:57 -04:00
Evan Typanski
985f4f7c72 Fix Clang 19 deprecation failure
Fixes #3994

Clang 19 with libc++ started failing to compile because the default
implementation of `std::char_traits` was removed, making uses of
`std::char_traits<unsigned char>` invalid. This was more of used for
convenience before, but it should be roughly the same behavior with
`char`.

See relevant LLVM commits:

aeecef08c3

08a0faf4cd
2024-10-30 08:47:15 -04:00
Arne Welzel
04872d4e78 signatures: Add data_end_offset to signature_match() and custom events
This change tracks the current offset (number of bytes fed into matchers)
on the top-level RuleEndpointState such that we can compute the relative ending
for matched texts individually.

Additionally, it adds the data_end_offset as a new optional parameter to
signature_match().
2024-10-30 13:29:58 +01:00
Emmanuele Zambon
792aacc8e5 Fix issues with Modbus message logging
This commit fixes three issues with Zeek's Modbus message logging:

1 - Some exception responses (e.g., READ_COILS_EXCEPTION) are logged
    twice: once without and once with the exception message.
2 - Some exception responses (e.g., PROGRAM_484_EXCEPTION) are not
    logged.
3 - Some known but reserved function codes (e.g., PROGRAM_UNITY) are
    logged as unk-xxx (e.g., unk-90), while it would be possible to
    log their known name.

To address these inconsistencies, the modbus parser has been updated
to parse all exception responses (i.e., all responses where the MSB
of the function code is set) using the already defined Exception
message.
Also, the Modbus main.zeek script has been updated to consistently
demand logging exception responses to the specialized
modbus_exception event, rather than logging some exception responses
in the modbus_message event and others in the modbus_exception event.
Finally, the main.zeek script has been updated to make sure that
for every known function code, the corresponding exception code was
also present, and the enumeration of known function codes in
consts.zeek has been expanded.

Closes #3984
2024-10-30 13:12:27 +01:00
Johanna Amann
681fd37a6d Merge remote-tracking branch 'origin/topic/awelzel/skip-modbus-when-spicy-ssl-enabled'
* origin/topic/awelzel/skip-modbus-when-spicy-ssl-enabled:
  modbus: Skip non-modbus traffic testing with Spicy SSL
  testing: Add have-spicy-ssl helper and update tests
2024-10-30 11:48:15 +00:00
Johanna Amann
eb74790d90 Merge branch 'improvement/fnv1a_hashing_algorithm_improvements' of https://github.com/martinvanhensbergen/zeek
Fixes indentation.

Fixes GH-3970
2024-10-30 11:46:26 +00:00
ronny8360988
5859e23198 Add pattern_end_offset to signature_state
Add pattern_end_offset to signature_state

Update init-bare.zeek

Update RuleMatcher.cc

Update RuleMatcher.h

Update init-bare.zeek

clang format

clang format

clang format

Using Match Offsets List

Temp commit
2024-10-30 12:32:00 +01:00
zeek-bot
ab4d8744b6 Update doc submodule [nomail] [skip ci] 2024-10-30 00:10:54 +00:00