Commit graph

17062 commits

Author SHA1 Message Date
Arne Welzel
1822f4f6fe telemetry/Manager: Remove variant include 2024-10-22 18:49:16 +02:00
Arne Welzel
70872673a1 telemetry: Invoke Telemetry::sync() only at scrape/collection time
This stops invoking Telemetry::sync() via a scheduled event and instead
only invokes it on-demand. This makes metric collection network time
independent and lazier, too.

With Prometheus scrape requests being processed on Zeek's main thread
now, we can safely invoke the script layer Telemetry::sync() hook.

Closes #3947
2024-10-22 18:49:11 +02:00
Arne Welzel
e118887771 Merge remote-tracking branch 'origin/topic/etyp/bg-test-nondeterminism'
* origin/topic/etyp/bg-test-nondeterminism:
  Fix input error test nondeterminism
2024-10-02 20:17:38 +02:00
Evan Typanski
e8872b4c7f Fix input error test nondeterminism
It seems like other similar tests get by because they have more "stuff"
before they call `terminate()` most likely. But, to be safe, just
removing the "received termination signal" line seems like the best
approach.
2024-10-02 13:26:41 -04:00
Arne Welzel
6aeb302bfc Merge remote-tracking branch 'origin/topic/etyp/remove-duplicate-table-warns'
* origin/topic/etyp/remove-duplicate-table-warns:
  Report suppressed warnings count
  Avoid duplicating warnings when reading table
2024-10-01 11:52:02 +02:00
Evan Typanski
ecabf882ac Report suppressed warnings count
This also triggers if there is one warning, which seems a little weird,
but it seems mostly reasonable.
2024-09-30 08:20:19 -04:00
Evan Typanski
6e8d43a552 Avoid duplicating warnings when reading table
Invalid lines in a file was the one case that would not suppress future
warnings. Just make it suppress warnings too, but clear that suppression
if there is a field in between that doesn't error.

Fixes #3692
2024-09-30 08:20:19 -04:00
zeek-bot
54391f50d9 Update doc submodule [nomail] [skip ci] 2024-09-28 00:32:20 +00:00
Christian Kreibich
b4c43e1607 Merge branch 'topic/vern/script-opt-maint.Sep24B'
* topic/vern/script-opt-maint.Sep24B:
  factoring of logic used by ZAM's low-level optimizer when adjusting control flow info
  BTest baseline update for more complete function/lambda names
  tweak to -O gen-C++ maintenance script to avoid treating plugins as BTests
  fixed lambda hash collision bug due to function descriptions lacking full parameter information
  fixes (to avoid collisions) for AST profiling's function hash computations
  removed unused ZAM cast-to-any operation
  fixes for ZAM tracking the return type associated with function calls
  ZAM control-flow tracking now explicitly includes the ends of loops
  fix for ZAM identification of common subexpressions
  "-O dump-final-ZAM" option similar to "dump-ZAM" only prints final version of functions
  fix for setting object locations to avoid use-after-free situation
  extended "-O allow-cond" to apply to both gen-C++ and gen-standalone-C++
  -O gen-C++ fix for run-time warnings for "when" lambdas
  fix to -O gen-C++ for recent AST profiling changes for identifying function parameters
  fix to -O gen-C++ for dealing with "hidden" parameters
  tweak to prevent an incorrect warning for scripts compiled to C++
  fixed overly narrow Spicy test for manipulating packet analyzers
  fixed memory leak for recursive ZAM functions that exit via an exception
  remove unnecessary header include
2024-09-27 15:20:21 -07:00
Vern Paxson
110e5d0787 factoring of logic used by ZAM's low-level optimizer when adjusting control flow info 2024-09-27 14:16:15 -07:00
Vern Paxson
61258587bf BTest baseline update for more complete function/lambda names 2024-09-27 14:16:10 -07:00
Vern Paxson
32d254bd47 tweak to -O gen-C++ maintenance script to avoid treating plugins as BTests 2024-09-27 13:25:00 -07:00
Vern Paxson
c0ffaabe2e fixed lambda hash collision bug due to function descriptions lacking full parameter information 2024-09-27 13:24:57 -07:00
Vern Paxson
adf3648554 fixes (to avoid collisions) for AST profiling's function hash computations 2024-09-27 13:24:35 -07:00
Vern Paxson
a390b3665b removed unused ZAM cast-to-any operation 2024-09-27 13:24:35 -07:00
Vern Paxson
e7817b709c fixes for ZAM tracking the return type associated with function calls 2024-09-27 13:24:34 -07:00
Vern Paxson
56d01ea83b ZAM control-flow tracking now explicitly includes the ends of loops 2024-09-27 13:24:34 -07:00
Vern Paxson
b0e21b7e64 fix for ZAM identification of common subexpressions 2024-09-27 13:24:34 -07:00
Vern Paxson
e0e6c462ce "-O dump-final-ZAM" option similar to "dump-ZAM" only prints final version of functions 2024-09-27 13:24:30 -07:00
Vern Paxson
07d211419a fix for setting object locations to avoid use-after-free situation 2024-09-27 13:23:06 -07:00
Vern Paxson
a68ffc5316 extended "-O allow-cond" to apply to both gen-C++ and gen-standalone-C++ 2024-09-27 13:23:06 -07:00
Vern Paxson
5c38f7637d -O gen-C++ fix for run-time warnings for "when" lambdas 2024-09-27 13:23:06 -07:00
Vern Paxson
d1c31927c4 fix to -O gen-C++ for recent AST profiling changes for identifying function parameters 2024-09-27 13:23:06 -07:00
Vern Paxson
2b64e3b05e fix to -O gen-C++ for dealing with "hidden" parameters 2024-09-27 13:23:06 -07:00
Vern Paxson
8025ee74ef tweak to prevent an incorrect warning for scripts compiled to C++ 2024-09-27 13:23:06 -07:00
Vern Paxson
ed905c33cd fixed overly narrow Spicy test for manipulating packet analyzers 2024-09-27 13:23:06 -07:00
Vern Paxson
11a7ef42ef fixed memory leak for recursive ZAM functions that exit via an exception 2024-09-27 13:23:06 -07:00
Vern Paxson
c165e05b83 remove unnecessary header include 2024-09-27 13:23:06 -07:00
Arne Welzel
c36ae2f1a3 Merge branch 'pr/3946'
* pr/3946:
  Honor no plaintext password logging for PASS cmd

Closes #3946 - squashed commits into a single one.
2024-09-27 16:52:44 +02:00
Martin van Hensbergen
0e926afbc1 Honor no plaintext password logging for PASS cmd 2024-09-27 16:45:06 +02:00
Arne Welzel
989730c241 Merge remote-tracking branch 'origin/topic/awelzel/cluster-backends-pre-work-v1'
* origin/topic/awelzel/cluster-backends-pre-work-v1:
  NEWS: Update
  scripts/base/cluster: Move active node management into node_down()
  logging/Manager: Extract another CreateWriter() helper
  logging/Manager: Extract path_func invocation into helper
  logging: Dedicated log flush timer
  all: Change to use Func::GetName()
  script_opt: Use Func::GetName()
  Func: Add std::string name accessors, deprecate const char* versions
  plugin/ComponentManager: Support lookup by EnumValPtr
2024-09-27 15:37:47 +02:00
Arne Welzel
77b0f7d7ba NEWS: Update 2024-09-27 15:32:09 +02:00
Arne Welzel
6bb7b9d726 scripts/base/cluster: Move active node management into node_down()
With the idea of an alternative cluster backend, we should
not maintain Cluster state within low-level Broker events.
2024-09-27 15:32:09 +02:00
Arne Welzel
78999d147d logging/Manager: Extract another CreateWriter() helper
For other cluster backends, CreateWriter() will use a logger's filter
configuration rather than receiving all configuration through CreateLog.
Extract a helper out from WriteToFilters() for reuse.
2024-09-27 15:32:09 +02:00
Arne Welzel
16cca62292 logging/Manager: Extract path_func invocation into helper 2024-09-27 15:32:09 +02:00
Arne Welzel
0d925e935e logging: Dedicated log flush timer
Log flushing is currently triggered based on the threading heartbeat timer
of WriterBackends and the hard-coded WRITE_BUFFER_SIZE 1000.

This change introduces a separate timer that is managed by the logger
manager instead of piggy-backing on the heartbeat timer, as well as a
const &redef for the buffer size.

This allows to modify the log flush frequency and batch size independently
of the threading heartbeat interval. Later, this will allow to re-use the
buffering and flushing logic of writer frontends for non-Broker cluster
backends, too.

One change here is that even frontends that do not have a backend will
be flushed regularly. This is wanted for non-Broker backends and should be
very cheap. Possibly, Broker can piggy back on this timer down the road, too,
rather than using its own script-level timer (see Broker::log_flush()).
2024-09-27 15:30:35 +02:00
Arne Welzel
77b9510c8a all: Change to use Func::GetName() 2024-09-27 15:11:17 +02:00
Arne Welzel
71e9c8d436 script_opt: Use Func::GetName() 2024-09-27 15:11:17 +02:00
Arne Welzel
89127722ea Func: Add std::string name accessors, deprecate const char* versions 2024-09-27 15:11:17 +02:00
Arne Welzel
bf2ba626b1 plugin/ComponentManager: Support lookup by EnumValPtr 2024-09-27 15:11:03 +02:00
Arne Welzel
4aca6290a7 Merge remote-tracking branch 'origin/topic/awelzel/debug-stream-env'
* origin/topic/awelzel/debug-stream-env:
  zeek-setup: Support enabling debug streams through env variable
2024-09-27 15:00:04 +02:00
Arne Welzel
780976bf91 zeek-setup: Support enabling debug streams through env variable
For debugging btests, it can be convenient to enable debug streams
by setting an environment variable rather than editing zeek invocations
and adding -B selectively.

Sample use case:

    $ export ZEEK_DEBUG_LOG_STREAMS=all
    $ btest -d core/failing-test.zeek
    $ less .tmp/core/failing-test/debug.log

This change makes Zeek's -B option and ZEEK_DEBUG_LOG_STREAMS are additive.
2024-09-27 14:27:36 +02:00
Benjamin Bannier
a27066e3fc Merge remote-tracking branch 'origin/topic/bbannier/ci-opensuse-leap-ps-dep' 2024-09-27 11:09:47 +02:00
Benjamin Bannier
2d1c865654 Install procps in OpenSuse Leap images
The tests `core.sigterm-regular` and `core.sigterm-stdin` rely on `ps`
to be present which is not the case anymore on OpenSuse Leap; install it
explicitly there.
2024-09-27 10:00:13 +02:00
Benjamin Bannier
96c091f7ac Merge remote-tracking branch 'origin/topic/bbannier/bump-spicy' 2024-09-27 09:20:18 +02:00
Benjamin Bannier
c8a4436552 Bump auxil/spicy to latest development snapshot 2024-09-27 08:10:57 +02:00
Tim Wojtulewicz
632b989e44 Update docs submodule [nomail] 2024-09-24 16:12:58 -07:00
Tim Wojtulewicz
702fb031a4 Merge remote-tracking branch 'origin/topic/awelzel/3936-pop3-and-redis'
* origin/topic/awelzel/3936-pop3-and-redis:
  pop3: Remove unused headers
  pop3: Prevent unbounded state growth
  btest/pop3: Add somewhat more elaborate testing
2024-09-23 11:00:06 -07:00
Arne Welzel
b22ec06568 Merge remote-tracking branch 'origin/topic/etyp/enum-port-sizeof'
* origin/topic/etyp/enum-port-sizeof:
  Add enum value negative check
  Fix port/enum values `SizeOf` not being a count
2024-09-18 19:10:59 +02:00
Arne Welzel
922cb4d51f pop3: Remove unused headers 2024-09-18 19:05:39 +02:00