zeek/testing/btest/scripts/base/frameworks/logging
Johanna Amann 0c875220e9 Default canonifier change to only remove first timestamp in line
In the past, we used a default canonifier, which removes everything that
looks like a timestamp from log files. The goal of this is to prevent
logs from changing, e.g., due to local system times ending up in log
files.

This, however, also has the side-effect of removing information that is
parsed from protocols which probably should be part of our tests.
There is at least one test (1999 certificates) where the entire test
output was essentially removed by the canonifier.

GH-4521 was similarly masked by this.

This commit changes the default canonifier, so that only the first
timestamp in a line is removed. This should skip timestamps that are
likely to change while keeping timestamps that are parsed
from protocol information.

A pass has been made over the tests, with some additional adjustments
for cases which require the old canonifier.

There are some cases in which we probably could go further and not
remove timestamps at all - that, however, seems like a follow-up
project.
2025-06-18 15:41:48 +01:00
..
delay testing/btest: Fix double commented @TEST- lines 2025-05-06 14:06:29 +02:00
sqlite testing/btest/*zeek: Comment all @TEST lines 2025-04-17 16:30:23 +02:00
adapt-filter.zeek GH-239: Rename bro to zeek, bro-config to zeek-config, and bro-path-dev to zeek-path-dev. 2019-05-01 21:43:45 +00:00
ascii-binary.zeek GH-239: Rename bro to zeek, bro-config to zeek-config, and bro-path-dev to zeek-path-dev. 2019-05-01 21:43:45 +00:00
ascii-double.zeek Canonifier improvements for the scripts.base.frameworks.logging.ascii-double test 2020-12-06 20:19:52 -08:00
ascii-empty.zeek GH-239: Rename bro to zeek, bro-config to zeek-config, and bro-path-dev to zeek-path-dev. 2019-05-01 21:43:45 +00:00
ascii-escape-binary.zeek Switch the TSV Zeek logs to be UTF8 by default. 2021-09-07 09:16:53 -07:00
ascii-escape-empty-str.zeek GH-239: Rename bro to zeek, bro-config to zeek-config, and bro-path-dev to zeek-path-dev. 2019-05-01 21:43:45 +00:00
ascii-escape-notset-str.zeek GH-239: Rename bro to zeek, bro-config to zeek-config, and bro-path-dev to zeek-path-dev. 2019-05-01 21:43:45 +00:00
ascii-escape-odd-url.zeek General btest cleanup 2020-08-11 11:26:22 -07:00
ascii-escape-set-separator.zeek GH-239: Rename bro to zeek, bro-config to zeek-config, and bro-path-dev to zeek-path-dev. 2019-05-01 21:43:45 +00:00
ascii-escape.zeek egrep reported as obsolete by opensuse-tumbleweed builds 2022-10-27 11:48:43 -07:00
ascii-gz-rotate.zeek GH-239: Rename bro to zeek, bro-config to zeek-config, and bro-path-dev to zeek-path-dev. 2019-05-01 21:43:45 +00:00
ascii-gz.zeek Adjust btests for OpenBSD portability 2019-11-08 17:32:44 -08:00
ascii-json-iso-timestamps.zeek GH-239: Rename bro to zeek, bro-config to zeek-config, and bro-path-dev to zeek-path-dev. 2019-05-01 21:43:45 +00:00
ascii-json-optional.zeek Add LogAscii::json_include_unset_fields flag to control unset field rendering 2021-12-08 17:29:07 -08:00
ascii-json-utf8.zeek Merge remote-tracking branch 'origin/topic/timw/cleaner-utf8' 2019-07-29 09:25:25 -07:00
ascii-json.zeek GH-606: Output nulls into json data if a field isn't set 2019-09-27 14:12:48 -07:00
ascii-line-like-comment.zeek GH-239: Rename bro to zeek, bro-config to zeek-config, and bro-path-dev to zeek-path-dev. 2019-05-01 21:43:45 +00:00
ascii-logdir.zeek Remove LogAscii::logdir (6.1 deprecation) 2023-06-14 10:07:22 -07:00
ascii-negative-ts.zeek logging/ascii/json: Make TS_MILLIS signed, add TS_MILLIS_UNSIGNED 2025-05-30 17:23:29 +02:00
ascii-options.zeek GH-239: Rename bro to zeek, bro-config to zeek-config, and bro-path-dev to zeek-path-dev. 2019-05-01 21:43:45 +00:00
ascii-timestamps.zeek GH-1450: Improve printing/logging of large double/interval/time values 2021-04-06 19:54:02 -07:00
ascii-tsv.zeek GH-239: Rename bro to zeek, bro-config to zeek-config, and bro-path-dev to zeek-path-dev. 2019-05-01 21:43:45 +00:00
ascii-utf8-enabled-control-characters.zeek Removed Policy Script for UTF-8 Logs 2019-07-30 11:19:06 -07:00
ascii-utf8-enabled-set-separator-escape.zeek Removed Policy Script for UTF-8 Logs 2019-07-30 11:19:06 -07:00
ascii-utf8-enabled.zeek Removed Policy Script for UTF-8 Logs 2019-07-30 11:19:06 -07:00
attr-extend.zeek GH-239: Rename bro to zeek, bro-config to zeek-config, and bro-path-dev to zeek-path-dev. 2019-05-01 21:43:45 +00:00
attr.zeek GH-239: Rename bro to zeek, bro-config to zeek-config, and bro-path-dev to zeek-path-dev. 2019-05-01 21:43:45 +00:00
disable-stream.zeek GH-239: Rename bro to zeek, bro-config to zeek-config, and bro-path-dev to zeek-path-dev. 2019-05-01 21:43:45 +00:00
empty-event.zeek GH-239: Rename bro to zeek, bro-config to zeek-config, and bro-path-dev to zeek-path-dev. 2019-05-01 21:43:45 +00:00
enable-stream.zeek GH-239: Rename bro to zeek, bro-config to zeek-config, and bro-path-dev to zeek-path-dev. 2019-05-01 21:43:45 +00:00
env-ext.test General btest cleanup 2020-08-11 11:26:22 -07:00
event-groups-integration.zeek logging: Add event_groups to Stream 2022-12-09 16:59:36 +01:00
events.zeek GH-239: Rename bro to zeek, bro-config to zeek-config, and bro-path-dev to zeek-path-dev. 2019-05-01 21:43:45 +00:00
exclude.zeek GH-239: Rename bro to zeek, bro-config to zeek-config, and bro-path-dev to zeek-path-dev. 2019-05-01 21:43:45 +00:00
field-extension-cluster-error.zeek Default canonifier change to only remove first timestamp in line 2025-06-18 15:41:48 +01:00
field-extension-cluster.zeek Default canonifier change to only remove first timestamp in line 2025-06-18 15:41:48 +01:00
field-extension-complex.zeek Default canonifier change to only remove first timestamp in line 2025-06-18 15:41:48 +01:00
field-extension-include-exclude.zeek Default canonifier change to only remove first timestamp in line 2025-06-18 15:41:48 +01:00
field-extension-invalid.zeek base/frameworks/spicy: Do not load base/misc/version 2023-10-24 13:15:21 +02:00
field-extension-optional.zeek Default canonifier change to only remove first timestamp in line 2025-06-18 15:41:48 +01:00
field-extension-table.zeek GH-239: Rename bro to zeek, bro-config to zeek-config, and bro-path-dev to zeek-path-dev. 2019-05-01 21:43:45 +00:00
field-extension.zeek Default canonifier change to only remove first timestamp in line 2025-06-18 15:41:48 +01:00
field-name-map.zeek GH-239: Rename bro to zeek, bro-config to zeek-config, and bro-path-dev to zeek-path-dev. 2019-05-01 21:43:45 +00:00
field-name-map2.zeek GH-239: Rename bro to zeek, bro-config to zeek-config, and bro-path-dev to zeek-path-dev. 2019-05-01 21:43:45 +00:00
file.zeek Remove deprecated open_log_file and log_file_name functions 2019-06-27 17:43:20 -07:00
hooks.zeek testing/btest/*zeek: Comment all @TEST lines 2025-04-17 16:30:23 +02:00
include.zeek GH-239: Rename bro to zeek, bro-config to zeek-config, and bro-path-dev to zeek-path-dev. 2019-05-01 21:43:45 +00:00
no-local.zeek GH-239: Rename bro to zeek, bro-config to zeek-config, and bro-path-dev to zeek-path-dev. 2019-05-01 21:43:45 +00:00
none-debug.zeek GH-239: Rename bro to zeek, bro-config to zeek-config, and bro-path-dev to zeek-path-dev. 2019-05-01 21:43:45 +00:00
path-func-column-demote.zeek Tighten local-nets filtering in the logging framework's path-func-column-demote test 2023-03-15 17:01:01 -07:00
path-func.zeek GH-239: Rename bro to zeek, bro-config to zeek-config, and bro-path-dev to zeek-path-dev. 2019-05-01 21:43:45 +00:00
pred.zeek Remove deprecated log filter predicates for 4.1 2021-01-27 10:52:40 -07:00
print-log-stdout.zeek Merge remote-tracking branch 'origin/topic/dev/print-to-log' 2019-12-02 13:47:09 -08:00
print-log.zeek Added a non boolean configuration and other changes as suggested by Jon 2019-11-26 21:53:21 -08:00
remove.zeek Remove unnecessary -B arguments from Zeek invocations in testsuite 2021-06-24 17:05:32 -07:00
rotate-ascii-logdir-leftover-log-rotation.zeek Remove LogAscii::logdir (6.1 deprecation) 2023-06-14 10:07:22 -07:00
rotate-custom-fmt-func-bad.zeek logging/Manager: Fix crash for rotation format function not returning 2023-04-13 09:23:51 +02:00
rotate-custom-fmt-func.zeek Default canonifier change to only remove first timestamp in line 2025-06-18 15:41:48 +01:00
rotate-custom.zeek Default canonifier change to only remove first timestamp in line 2025-06-18 15:41:48 +01:00
rotate-default-logdir-leftover-log-rotation-shadow-files.zeek logging: Introduce Log::default_logdir deprecate LogAscii::logdir and per writer logdir 2022-07-06 18:54:29 +02:00
rotate-default-logdir-leftover-log-rotation-stale-shadow-files.zeek logging: Introduce Log::default_logdir deprecate LogAscii::logdir and per writer logdir 2022-07-06 18:54:29 +02:00
rotate-default-logdir.zeek logging: Introduce Log::default_logdir deprecate LogAscii::logdir and per writer logdir 2022-07-06 18:54:29 +02:00
rotate-leftover-log-rotation-shadow-files-postprocessor-errors.zeek logging/Ascii: Fix abort() for non-existing postrotation functions 2025-06-16 14:55:49 +02:00
rotate-leftover-log-rotation-shadow-files.zeek btest/logging: Fly-by cleanup 2025-06-16 14:56:30 +02:00
rotate-pp-env.zeek logging: Support rotation_postprocessor_command_env 2023-04-17 13:10:14 +00:00
rotate-sanitize.zeek Merge remote-tracking branch 'origin/topic/robin/gh-54-sanitize' 2021-09-22 12:17:05 -07:00
rotate.zeek More bro-to-zeek renaming in the unit tests 2019-05-16 02:27:54 -05:00
scope_sep.zeek GH-239: Rename bro to zeek, bro-config to zeek-config, and bro-path-dev to zeek-path-dev. 2019-05-01 21:43:45 +00:00
scope_sep_and_field_name_map.zeek GH-239: Rename bro to zeek, bro-config to zeek-config, and bro-path-dev to zeek-path-dev. 2019-05-01 21:43:45 +00:00
stdout.zeek GH-239: Rename bro to zeek, bro-config to zeek-config, and bro-path-dev to zeek-path-dev. 2019-05-01 21:43:45 +00:00
telemetry.zeek Remove prefix column from telemetry.log 2024-06-04 14:14:58 -07:00
test-logging.zeek GH-239: Rename bro to zeek, bro-config to zeek-config, and bro-path-dev to zeek-path-dev. 2019-05-01 21:43:45 +00:00
types.zeek GH-239: Rename bro to zeek, bro-config to zeek-config, and bro-path-dev to zeek-path-dev. 2019-05-01 21:43:45 +00:00
unset-record.zeek GH-239: Rename bro to zeek, bro-config to zeek-config, and bro-path-dev to zeek-path-dev. 2019-05-01 21:43:45 +00:00
vec.zeek GH-239: Rename bro to zeek, bro-config to zeek-config, and bro-path-dev to zeek-path-dev. 2019-05-01 21:43:45 +00:00
writer-path-conflict.zeek General btest cleanup 2020-08-11 11:26:22 -07:00