Commit graph

17245 commits

Author SHA1 Message Date
Vern Paxson
b0e21b7e64 fix for ZAM identification of common subexpressions 2024-09-27 13:24:34 -07:00
Vern Paxson
e0e6c462ce "-O dump-final-ZAM" option similar to "dump-ZAM" only prints final version of functions 2024-09-27 13:24:30 -07:00
Vern Paxson
07d211419a fix for setting object locations to avoid use-after-free situation 2024-09-27 13:23:06 -07:00
Vern Paxson
a68ffc5316 extended "-O allow-cond" to apply to both gen-C++ and gen-standalone-C++ 2024-09-27 13:23:06 -07:00
Vern Paxson
5c38f7637d -O gen-C++ fix for run-time warnings for "when" lambdas 2024-09-27 13:23:06 -07:00
Vern Paxson
d1c31927c4 fix to -O gen-C++ for recent AST profiling changes for identifying function parameters 2024-09-27 13:23:06 -07:00
Vern Paxson
2b64e3b05e fix to -O gen-C++ for dealing with "hidden" parameters 2024-09-27 13:23:06 -07:00
Vern Paxson
8025ee74ef tweak to prevent an incorrect warning for scripts compiled to C++ 2024-09-27 13:23:06 -07:00
Vern Paxson
ed905c33cd fixed overly narrow Spicy test for manipulating packet analyzers 2024-09-27 13:23:06 -07:00
Vern Paxson
11a7ef42ef fixed memory leak for recursive ZAM functions that exit via an exception 2024-09-27 13:23:06 -07:00
Vern Paxson
c165e05b83 remove unnecessary header include 2024-09-27 13:23:06 -07:00
Arne Welzel
c36ae2f1a3 Merge branch 'pr/3946'
* pr/3946:
  Honor no plaintext password logging for PASS cmd

Closes #3946 - squashed commits into a single one.
2024-09-27 16:52:44 +02:00
Martin van Hensbergen
0e926afbc1 Honor no plaintext password logging for PASS cmd 2024-09-27 16:45:06 +02:00
Arne Welzel
989730c241 Merge remote-tracking branch 'origin/topic/awelzel/cluster-backends-pre-work-v1'
* origin/topic/awelzel/cluster-backends-pre-work-v1:
  NEWS: Update
  scripts/base/cluster: Move active node management into node_down()
  logging/Manager: Extract another CreateWriter() helper
  logging/Manager: Extract path_func invocation into helper
  logging: Dedicated log flush timer
  all: Change to use Func::GetName()
  script_opt: Use Func::GetName()
  Func: Add std::string name accessors, deprecate const char* versions
  plugin/ComponentManager: Support lookup by EnumValPtr
2024-09-27 15:37:47 +02:00
Arne Welzel
77b0f7d7ba NEWS: Update 2024-09-27 15:32:09 +02:00
Arne Welzel
6bb7b9d726 scripts/base/cluster: Move active node management into node_down()
With the idea of an alternative cluster backend, we should
not maintain Cluster state within low-level Broker events.
2024-09-27 15:32:09 +02:00
Arne Welzel
78999d147d logging/Manager: Extract another CreateWriter() helper
For other cluster backends, CreateWriter() will use a logger's filter
configuration rather than receiving all configuration through CreateLog.
Extract a helper out from WriteToFilters() for reuse.
2024-09-27 15:32:09 +02:00
Arne Welzel
16cca62292 logging/Manager: Extract path_func invocation into helper 2024-09-27 15:32:09 +02:00
Arne Welzel
0d925e935e logging: Dedicated log flush timer
Log flushing is currently triggered based on the threading heartbeat timer
of WriterBackends and the hard-coded WRITE_BUFFER_SIZE 1000.

This change introduces a separate timer that is managed by the logger
manager instead of piggy-backing on the heartbeat timer, as well as a
const &redef for the buffer size.

This allows to modify the log flush frequency and batch size independently
of the threading heartbeat interval. Later, this will allow to re-use the
buffering and flushing logic of writer frontends for non-Broker cluster
backends, too.

One change here is that even frontends that do not have a backend will
be flushed regularly. This is wanted for non-Broker backends and should be
very cheap. Possibly, Broker can piggy back on this timer down the road, too,
rather than using its own script-level timer (see Broker::log_flush()).
2024-09-27 15:30:35 +02:00
Arne Welzel
77b9510c8a all: Change to use Func::GetName() 2024-09-27 15:11:17 +02:00
Arne Welzel
71e9c8d436 script_opt: Use Func::GetName() 2024-09-27 15:11:17 +02:00
Arne Welzel
89127722ea Func: Add std::string name accessors, deprecate const char* versions 2024-09-27 15:11:17 +02:00
Arne Welzel
bf2ba626b1 plugin/ComponentManager: Support lookup by EnumValPtr 2024-09-27 15:11:03 +02:00
Arne Welzel
4aca6290a7 Merge remote-tracking branch 'origin/topic/awelzel/debug-stream-env'
* origin/topic/awelzel/debug-stream-env:
  zeek-setup: Support enabling debug streams through env variable
2024-09-27 15:00:04 +02:00
Arne Welzel
780976bf91 zeek-setup: Support enabling debug streams through env variable
For debugging btests, it can be convenient to enable debug streams
by setting an environment variable rather than editing zeek invocations
and adding -B selectively.

Sample use case:

    $ export ZEEK_DEBUG_LOG_STREAMS=all
    $ btest -d core/failing-test.zeek
    $ less .tmp/core/failing-test/debug.log

This change makes Zeek's -B option and ZEEK_DEBUG_LOG_STREAMS are additive.
2024-09-27 14:27:36 +02:00
Benjamin Bannier
a27066e3fc Merge remote-tracking branch 'origin/topic/bbannier/ci-opensuse-leap-ps-dep' 2024-09-27 11:09:47 +02:00
Benjamin Bannier
2d1c865654 Install procps in OpenSuse Leap images
The tests `core.sigterm-regular` and `core.sigterm-stdin` rely on `ps`
to be present which is not the case anymore on OpenSuse Leap; install it
explicitly there.
2024-09-27 10:00:13 +02:00
Benjamin Bannier
96c091f7ac Merge remote-tracking branch 'origin/topic/bbannier/bump-spicy' 2024-09-27 09:20:18 +02:00
Benjamin Bannier
c8a4436552 Bump auxil/spicy to latest development snapshot 2024-09-27 08:10:57 +02:00
Tim Wojtulewicz
632b989e44 Update docs submodule [nomail] 2024-09-24 16:12:58 -07:00
Tim Wojtulewicz
702fb031a4 Merge remote-tracking branch 'origin/topic/awelzel/3936-pop3-and-redis'
* origin/topic/awelzel/3936-pop3-and-redis:
  pop3: Remove unused headers
  pop3: Prevent unbounded state growth
  btest/pop3: Add somewhat more elaborate testing
2024-09-23 11:00:06 -07:00
Arne Welzel
b22ec06568 Merge remote-tracking branch 'origin/topic/etyp/enum-port-sizeof'
* origin/topic/etyp/enum-port-sizeof:
  Add enum value negative check
  Fix port/enum values `SizeOf` not being a count
2024-09-18 19:10:59 +02:00
Arne Welzel
922cb4d51f pop3: Remove unused headers 2024-09-18 19:05:39 +02:00
Arne Welzel
cf9fe91705 pop3: Prevent unbounded state growth
The cmds list may grow unbounded due to the POP3 analyzer being in
multiLine mode after seeing `AUTH` in a Redis connection, but never
a `.` terminator. This can easily be provoked by the Redis ping
command.

This adds two heuristics: 1) Forcefully process the oldest commands in
the cmds list and cap it at max_pending_commands. 2) Start raising
analyzer violations if the client has been using more than
max_unknown_client_commands commands (default 10).

Closes #3936
2024-09-18 19:05:39 +02:00
Arne Welzel
b4fdce8d5b btest/pop3: Add somewhat more elaborate testing
PCAP taken from here: https://tranalyzer.com/tutorial/pop and reference
added to Traces/README.
2024-09-18 17:31:13 +02:00
zeek-bot
2b21b101e1 Update doc submodule [nomail] [skip ci] 2024-09-18 00:13:17 +00:00
Arne Welzel
5a26a39d06 Merge remote-tracking branch 'origin/topic/awelzel/teredo-gtpv1-conn-removal-hook'
* origin/topic/awelzel/teredo-gtpv1-conn-removal-hook:
  btest: Update baselines for removal-hooks addition
  gtpv1: Replace connection_state_remove() with RemovalHook
  teredo: Replace connection_state_remove() with RemovalHook
  teredo: Move conn member from analyzer to encapsulation
2024-09-17 18:51:44 +02:00
Arne Welzel
f24bc1ee88 Merge remote-tracking branch 'origin/topic/awelzel/prom-callbacks-2'
* origin/topic/awelzel/prom-callbacks-2:
  Update broker submodule
  telemetry: Move callbacks to Zeek
  auxil/prometheus-cpp: Pin to 1.2.4
2024-09-17 18:51:12 +02:00
Arne Welzel
a5d93c4dec btest: Update baselines for removal-hooks addition
The removal_hooks field exists in bare mode (seems fine) and moved within the
connection record to earlier, so a bunch of baselines changed
2024-09-17 18:15:15 +02:00
Arne Welzel
9d4025804d gtpv1: Replace connection_state_remove() with RemovalHook 2024-09-17 18:15:09 +02:00
Arne Welzel
34956f4ca4 teredo: Replace connection_state_remove() with RemovalHook
Remove overhead of unconditionally calling remove_teredo_connection()
for *every* connection by installing a connection removal hook for only
when state was allocated.
2024-09-17 18:13:44 +02:00
Arne Welzel
6a930c1cf8 teredo: Move conn member from analyzer to encapsulation
There's only a single instance of the Teredo analyzer. Mutating the conn
member for every new packet and leaving it set after processing the
packet is confusing. Move conn into TeredoEncapsulation instead, or pass
it explicitly.
2024-09-17 18:13:33 +02:00
Evan Typanski
08348cd177 Add enum value negative check
There was one already at parse time, this adds a check later so that
cases like overflows or internal enums with negative values get caught.
2024-09-17 12:04:19 -04:00
Evan Typanski
d3dd8a155d Fix port/enum values SizeOf not being a count
Really, they both should be count. But, they were getting provided as an
integer. Port is easy since it is backed by an unsigned value. Enums
*should* be unsigned, but aren't. This doesn't address that, it just
takes the other name for this operator (absolute value) and makes the
enum value positive if it's negative.

This fixes a case where using the size of operator on enum/port values
in certain contexts (like the default parameter of a struct) would cause
an internal error.
2024-09-17 10:55:45 -04:00
Arne Welzel
617e7acd0f Update broker submodule
...to pin prometheus to 1.2.4
2024-09-12 13:06:02 +02:00
Tim Wojtulewicz
ba91de59b0 Merge remote-tracking branch 'mp/topic/mp-corelight/update-docker-env-syntax'
* mp/topic/mp-corelight/update-docker-env-syntax:
  Update Dockerfile to avoid LegacyKeyValueFormat warning
2024-09-12 09:18:56 +02:00
Arne Welzel
48dd89ef33 telemetry: Move callbacks to Zeek
Now that we run callbacks on the main loop, we can move callback support
for Counter and Gauge instances directly into Zeek and don't need to patch
prometheus-cpp anymore.
2024-09-12 09:12:24 +02:00
Tim Wojtulewicz
923ca7e817 Merge remote-tracking branch 'origin/topic/timw/strreplace-endless-loop'
* origin/topic/timw/strreplace-endless-loop:
  Fix potential endless loop in util::strreplace
2024-09-12 08:59:20 +02:00
Johanna Amann
899f7297d7 Merge remote-tracking branch 'origin/topic/timw/remove-negative-timestamp-test'
* origin/topic/timw/remove-negative-timestamp-test:
  Remove core.negative-time btest
2024-09-12 08:53:00 +02:00
Johanna Amann
a0ff0fb18b Merge remote-tracking branch 'origin/topic/johanna/reformat-spicy-ssl'
* origin/topic/johanna/reformat-spicy-ssl:
  Bump spicy-format to 0.16.2
  Bump spicy-format to 0.16.1
  Spicy SSL: reformat with new version of spicy format
2024-09-12 08:51:22 +02:00