Commit graph

3433 commits

Author SHA1 Message Date
Tim Wojtulewicz
488ac7ddf0 Merge remote-tracking branch 'origin/topic/vern/zam-pattern-comparison'
* origin/topic/vern/zam-pattern-comparison:
  update of BTest that tracks number of (and validates) ZAM operations
  ZAM support for pattern equality/inequality operations
  expanded ZAM operations for bit-shifting to allow for int/count shift values
  added type coercion for bit-shifting expressions
2024-12-05 11:09:15 -07:00
Arne Welzel
6482a9a8c2 Merge remote-tracking branch 'origin/topic/awelzel/ctu-sme-11-vm-win7ad'
* origin/topic/awelzel/ctu-sme-11-vm-win7ad:
  zeek-testing: Add Experiment-VM-Microsoft-Windows7AD-1 PCAP and baselines
2024-12-05 14:52:14 +01:00
Arne Welzel
a44cafafe7 Merge remote-tracking branch 'origin/topic/awelzel/pluggable-cluster-backends-part3'
* origin/topic/awelzel/pluggable-cluster-backends-part3:
  init-bare/zeek-setup: Groundwork for instantiating cluster backends
  cluster/serializer: Add binary-serialization-format
  logging/WriterFrontend: Add logic for non-broker cluster backends
  logging/WriterBackend: Include logging/Types.h
  logging/Manager: Implement new WriteBatchFromRemote()
  logging/WriterFrontend: Add LogWriteHeader as member
  logging: Add filter_name to WriterInfo
2024-12-04 13:47:49 +01:00
Arne Welzel
2c52e533b2 Merge remote-tracking branch 'origin/topic/awelzel/spicy-type-convert-asan-options'
* origin/topic/awelzel/spicy-type-convert-asan-options:
  btest/spicy.type-converter: Drop ASAN_OPTIONS from test
2024-12-04 11:14:10 +01:00
Arne Welzel
067c40a545 Merge remote-tracking branch 'origin/topic/neverlord/broker-cleanup'
* origin/topic/neverlord/broker-cleanup:
  Remove obsolete c_str_safe utility
  Remove obsolete Broker compatibility layer
2024-12-04 11:08:08 +01:00
Tim Wojtulewicz
deb2acaeda Merge remote-tracking branch 'origin/topic/timw/json-interval-conversion'
* origin/topic/timw/json-interval-conversion:
  Add interval_as_double argument to control how intervals are converted to JSON
  Add btest for round-trip JSON conversion
  Allow comparing two PatternVals
  Handle conversion between data from Val::ToJSON and ValFromJSON better
2024-12-03 10:10:37 -07:00
Arne Welzel
d2ecdc2a11 Merge remote-tracking branch 'origin/topic/vern/ZAM-cluster-bifs'
* origin/topic/vern/ZAM-cluster-bifs:
  added new Cluster:: BiFs to script optimization tracking
2024-12-02 13:43:36 +01:00
Arne Welzel
423c2de4c8 Merge remote-tracking branch 'origin/topic/vern/dump-no-opt-ZAM'
* origin/topic/vern/dump-no-opt-ZAM:
  fixed -O dump-final-ZAM to work in conjunction with -O no-ZAM-opt
2024-12-02 10:59:16 +01:00
Benjamin Bannier
f1bb009e44 Merge branch 'topic/bbannier/bump-spicy' 2024-12-02 10:35:34 +01:00
Tim Wojtulewicz
8d625715ca Merge remote-tracking branch 'origin/topic/johanna/jq-guard'
* origin/topic/johanna/jq-guard:
  Add TEST-REQUIRES: which jq to two new tests that are using jq
2024-11-27 12:31:59 -07:00
Johanna Amann
7b582bc345 Merge remote-tracking branch 'origin/topic/johanna/sqlite-pragmas'
* origin/topic/johanna/sqlite-pragmas:
  Options for SQLite log writer, eliminate duplicate definitions
  Test synchronous/journal mode options for SQLite log writer
  Added default options for synchronous and journal mode
  Support for synchronous and journal_mode
2024-11-27 08:32:08 +00:00
Arne Welzel
3ce41f1eff Merge remote-tracking branch 'origin/topic/awelzel/pluggable-cluster-backends-part2'
* origin/topic/awelzel/pluggable-cluster-backends-part2:
  ci/test.sh: Run doctest with TZ=UTC
  cluster/setup-connections: Switch to Cluster::subscribe(), short-circuit broker
  cluster/serializer: Add Broker based event serializers
  cluster: Add Cluster scoped bifs
  Reporter: Add ScriptLocationScope helper
  init-bare/zeek-setup: Add Cluster::backend const &redef
  broker: Implement cluster::Backend interface
  Broker: Fix some error messages
  broker: Remove MakeEvent(ArgsSpan)
2024-11-26 18:01:28 +01:00
Arne Welzel
c3b30b187e Merge remote-tracking branch 'origin/topic/vern/zam-exception-leaks'
* origin/topic/vern/zam-exception-leaks:
  More robust memory management for ZAM execution - fixes #4052
2024-11-26 17:17:10 +01:00
Arne Welzel
f65c4f0b7e Merge remote-tracking branch 'origin/topic/awelzel/deprecate-val-list-to-args'
* origin/topic/awelzel/deprecate-val-list-to-args:
  ZeekArgs: Deprecate val_list_to_args()
2024-11-26 15:25:25 +01:00
Arne Welzel
40a780bb24 Merge remote-tracking branch 'origin/topic/awelzel/test-init-hooks-plugin'
* origin/topic/awelzel/test-init-hooks-plugin:
  btest/plugins: Add a plugin testing Init and Done hooks
2024-11-22 18:38:06 +01:00
Arne Welzel
542640db00 Merge remote-tracking branch 'origin/topic/awelzel/skip-core-expr-error'
* origin/topic/awelzel/skip-core-expr-error:
  Disable core.expr-execption btest under ZAM to fix CI builds
2024-11-22 15:29:48 +01:00
Arne Welzel
97f05b2f8c Merge remote-tracking branch 'origin/topic/awelzel/pluggable-cluster-backends-part1'
* origin/topic/awelzel/pluggable-cluster-backends-part1:
  btest: Test Broker::make_event() together with Cluster::publish_hrw()
  btest: Add cluster dir, minimal test for enum value
  broker: Add shim plugin adding a backend component
  zeek-setup: Instantiate backend::manager
  cluster: Add to src/CMakeLists.txt
  cluster: Add Components and ComponentManager for new components
  cluster/Backend: Interface for cluster backends
  cluster/Serializer: Interface for event and log serializers
  logging: Introduce logging/Types.h
  SerialTypes/Field: Allow default construction and add move constructor
  DebugLogger: Add cluster debugging stream
  plugin: Add component enums for pluggable cluster backends
  broker: Pass frame to MakeEvent()
2024-11-22 12:53:23 +01:00
Arne Welzel
9883b3a917 Merge remote-tracking branch 'origin/topic/awelzel/sumstats-non-cluster-final-epoch'
* origin/topic/awelzel/sumstats-non-cluster-final-epoch:
  sumstat/non-cluster: Move last epoch processing to zeek_done()
2024-11-21 16:39:18 +01:00
Arne Welzel
94ffcd6535 Merge remote-tracking branch 'origin/topic/awelzel/plugin-init-pre-execution'
* origin/topic/awelzel/plugin-init-pre-execution:
  plugin: Add InitPreExecution()
2024-11-21 14:41:14 +01:00
Arne Welzel
aac640ebff Merge remote-tracking branch 'origin/topic/awelzel/bump-zeekjs-0-14-0'
* origin/topic/awelzel/bump-zeekjs-0-14-0:
  Bump zeekjs to v0.14.0
2024-11-21 10:22:44 +01:00
Arne Welzel
6e916efe8d Merge remote-tracking branch 'origin/topic/awelzel/bump-zeekjs-0-13-2'
* origin/topic/awelzel/bump-zeekjs-0-13-2:
  Bump zeekjs to 0.13.2
2024-11-19 19:17:48 +01:00
Tim Wojtulewicz
d9a74680e0 Merge remote-tracking branch 'origin/topic/timw/speed-up-zam-ci-testing'
* origin/topic/timw/speed-up-zam-ci-testing:
  CI: Use test.sh script for running ZAM tests, but disable parts of it
2024-11-19 10:53:21 -07:00
Christian Kreibich
96273b4550 Merge branch 'topic/christian/fix-rulematcher-iteration'
* topic/christian/fix-rulematcher-iteration:
  Order rule traversal in RuleMatcher::Match() operations by Rule index
2024-11-18 10:13:18 -08:00
Christian Kreibich
5e0e2a8bd8 Merge branch 'topic/christian/fix-rulematcher-iteration'
* topic/christian/fix-rulematcher-iteration:
  Define ordering on Rule instances and use on sets in RuleMatcher
2024-11-18 09:39:22 -08:00
Arne Welzel
7ae52ea4d2 Merge remote-tracking branch 'origin/topic/awelzel/zam-baseline-updates'
* origin/topic/awelzel/zam-baseline-updates:
  communityid: Use GetField() over GetFieldAs()
  btest/Baseline.zam: Updates for proto addition
2024-11-18 17:28:37 +01:00
Tim Wojtulewicz
62bc324fe1 Merge remote-tracking branch 'origin/topic/christian/debug-stream-tweaks'
* origin/topic/christian/debug-stream-tweaks:
  Make debug stream names use "-", and handle "_" transparently
  Make "-B all" apply to plugin streams as well.
  Sort streams in "-B help" output, and match case-insensitively throughout
2024-11-16 09:34:42 -07:00
Christian Kreibich
26cc378471 Merge branch 'topic/timw/avoid-unknown-ip-transport-weirds'
* topic/timw/avoid-unknown-ip-transport-weirds:
  Don't forward out of the unknown IP transport analyzer
2024-11-15 19:08:33 -08:00
Arne Welzel
2125a1f558 Merge remote-tracking branch 'origin/topic/awelzel/log-broker-io-telemetry-rules-includes'
* origin/topic/awelzel/log-broker-io-telemetry-rules-includes:
  zeek-setup: Remove some unused headers
  clang-format: Sort doctest header at the bottom
  RuleMatcher: Move plugin/Manager.h include from .h to .cc
  iosource/Manager: Remove superflous includes
  telemetry/Manager: Remove broker header include, add fnmatch.h
  logging/Manager: Fix using filename from input.h in debug log
2024-11-15 17:40:40 +01:00
Arne Welzel
18bfdb8a2b Merge remote-tracking branch 'origin/topic/awelzel/deprecate-broker-auto-publish'
* origin/topic/awelzel/deprecate-broker-auto-publish:
  sumstats: Remove copy() for Broker::publish() calls
  broker/Publish: Use event time instead of network time
  broker/Eventhandler: Deprecate Broker::auto_publish() for v8.1
  btest: Remove Broker::auto_publish() usages
  frameworks/control: Remove Broker::auto_publish()
  catch-and-release: Remove Broker::auto_publish()
  ssl/validate-certs: Remove Broker::auto_publish()
  sumstats: Remove Broker::auto_publish()
  cluster_started: No Broker::auto_publish() use
  openflow: Remove Broker::auto_publish()
  dhcp: Remove Broker::auto_publish()
  frameworks/notice: Remove Broker::auto_publish()
  netcontrol: Replace Broker::auto_publish()
  intel: Switch to Cluster::publish()
  broker: Support publish() of unspecified set() / table()
  types: Fix table() resulting in table_type->IsSet() == true
2024-11-14 14:17:13 +01:00
Christian Kreibich
af4c21763f Merge branch 'topic/christian/ci-updates'
* topic/christian/ci-updates:
  CI: Use FEDORA40 crypto policy in Fedora 41
  Bump zeekjs to 0.13.0
  CI: bump FreeBSD 13 to 13.4, released in September
  CI: drop Fedora 39, add 41
2024-11-13 17:29:23 -08:00
Tim Wojtulewicz
3c08c57be3 Merge remote-tracking branch 'origin/topic/timw/3915-unknown-ip-protocol'
* origin/topic/timw/3915-unknown-ip-protocol:
  Add NEWS entry for ip_proto feature
  Move IP protocol names table out of policy script to init-bare
  Minor review nits
  Fixes for community ID hashing with new proto values
  Use new_connection instead of connection_state_remove
  Add policy script to remove ip_proto field, rename protocol naming script
  Rename protocol_id field to ip_proto and similar renaming for name field
  Increase size of proto fields to uint16_t, add common default value
  Disable part of core/dict-iteration-expire5 btest to avoid iteration bug
  Add conn.log entries for connections with unhandled IP protocols
2024-11-13 14:36:22 -07:00
Tim Wojtulewicz
0217208c49 Merge remote-tracking branch 'origin/topic/timw/remove-abspath-cleanup'
* origin/topic/timw/remove-abspath-cleanup:
  diff-remove-abspath: Add separate handling of Windows paths
  diff-remove-abspath: Remove capture of windows drive letters from POSIX regex
2024-11-12 12:26:56 -07:00
Robin Sommer
0ea2a35d7a
Merge remote-tracking branch 'origin/topic/robin/spicy-bump'
* origin/topic/robin/spicy-bump:
  Bump Spicy to current `main`.
2024-11-12 16:16:23 +01:00
Arne Welzel
d0bf4e428a Merge remote-tracking branch 'origin/topic/awelzel/pseudo-realtime-again'
* origin/topic/awelzel/pseudo-realtime-again:
  PktSrc: Remove first_timestamp condition check
  PktSrc: Fix includes
  PktSrc/RunState: Scale on first_wallclock and move pseudo realtime logic to RunState
  RunState.h: Deprecate misleadingly named current_packet_timestamp()
  debug: Add processing suspended/continued to debug.log
2024-11-12 16:00:19 +01:00
Arne Welzel
9e27334596 Merge remote-tracking branch 'origin/topic/vern/zam-asan-fixes'
* origin/topic/vern/zam-asan-fixes:
  ZAM fixes for assignments involving "any" record fields
  fixes for (mostly ZAM) vector operation issues found by ASAN

Including a fix for mmdb/explicit-open.zeek to avoid using assert.
2024-11-12 10:29:56 +01:00
Benjamin Bannier
1d38c31071 Merge remote-tracking branch 'origin/topic/etyp/cookie-nullptr-spicy-dpd' 2024-11-11 22:30:50 +01:00
Arne Welzel
50c2b10cfb Merge remote-tracking branch 'origin/topic/awelzel/run-zam-ci-if-changes-include-src-script-opt'
* origin/topic/awelzel/run-zam-ci-if-changes-include-src-script-opt:
  ci: Run ZAM CI if src/script_opt is modified
2024-11-11 10:25:56 +01:00
Arne Welzel
f598c89f17 Merge remote-tracking branch 'origin/topic/timw/update-c-ares-to-latest-release'
* origin/topic/timw/update-c-ares-to-latest-release:
  DNS_Mgr: Remove processing of dns aliases in general
  ci: Add dnsmasq to a few platforms for testing
  DNS_Mgr: Fix aliases memory issues
  btest: Add integration test for DNS_Mgr
  DNS_Mgr: Remove usage of ares_getsock from Lookup
  DNS_Mgr: Remove usage of ares_getsock from GetNextTimeout
  DNS_Mgr: Switch to ares_set_servers_csv
  DNS_Mgr: Use ares_dns_record methods for queries
  Update vcpkg submodule to pick up c-ares v1.34.2
  Update c-ares submodule to v1.34.2
2024-11-11 09:53:04 +01:00
Arne Welzel
d3579c1f34 Merge remote-tracking branch 'origin/topic/awelzel/community-id-new-connection'
* origin/topic/awelzel/community-id-new-connection:
  policy/community-id: Populate conn$community_id in new_connection()
2024-11-11 09:35:49 +01:00
Arne Welzel
346a9233da Merge remote-tracking branch 'origin/topic/vern/zam-any-coerce-leak'
* origin/topic/vern/zam-any-coerce-leak:
  Fixed ZAM memory leak when coercing values to "any"
2024-11-08 18:36:34 +01:00
Arne Welzel
4f8ef3c792 Merge remote-tracking branch 'origin/topic/vern/zam-degenerate-CFT-propagation'
* origin/topic/vern/zam-degenerate-CFT-propagation:
  fixes for ZAM's propagation of control flow information for some degenerate constructs
2024-11-08 15:35:08 +01:00
Arne Welzel
bc75b1811d Merge remote-tracking branch 'origin/topic/vern/zam-vector-loop-leak'
* origin/topic/vern/zam-vector-loop-leak:
  fixed ZAM memory leak when looping over vectors of records
2024-11-08 15:32:57 +01:00
Arne Welzel
8613f821f9 Merge remote-tracking branch 'origin/topic/vern/cat-builtin-tmp'
* origin/topic/vern/cat-builtin-tmp:
  fixed access to uninitialized memory in ZAM's "cat" built-in
2024-11-08 15:23:00 +01:00
Benjamin Bannier
e4dbba20a4 Merge remote-tracking branch 'origin/topic/bbannier/bump-spicy' 2024-11-06 11:57:01 +01:00
Benjamin Bannier
2e8d6e86e7 Merge remote-tracking branch 'origin/topic/bbannier/spicy-cookie-nullptr-deref' 2024-11-05 17:39:12 +01:00
Arne Welzel
a3a0155825 Merge remote-tracking branch 'origin/topic/awelzel/sqli-test-followup'
* origin/topic/awelzel/sqli-test-followup:
  btest: Add detect-sqli test script
2024-11-04 19:35:34 +01:00
Arne Welzel
c171d9e8c2 Merge remote-tracking branch 'origin/topic/vern/canonicalize-std-string'
* origin/topic/vern/canonicalize-std-string:
  switched CPPCompile::Canonicalize() to take std::string instead of const char*
2024-11-04 18:47:18 +01:00
Arne Welzel
8935dc9ed4 Merge remote-tracking branch 'origin/topic/vern/script-opt-fnv1a64'
* origin/topic/vern/script-opt-fnv1a64:
  minor ZAM BTest updates for recently added fnv1a64 BiF
2024-11-04 16:33:21 +01:00
Arne Welzel
db497d1da7 Merge remote-tracking branch 'origin/topic/awelzel/pattern-matches-rework'
* origin/topic/awelzel/pattern-matches-rework:
  RuleMatcher: Use a single list for tracking pattern_matches
  ZeekString: Implement move constructor
2024-11-04 16:32:48 +01:00
Arne Welzel
7b3bc891f4 Merge remote-tracking branch 'origin/topic/etyp/ubuntu-clang-ci'
Modified ubuntu24_clang_libcpp_task to be skipped on PRs.

* origin/topic/etyp/ubuntu-clang-ci:
  CI: Upgrade FreeBSD 14 to 14.1
  CI: Add Ubuntu Clang task
  Fix up minor warnings in touched files
  Fix Clang 19 deprecation failure
2024-10-30 14:54:08 +01:00