zeek/testing/scripts
Johanna Amann 0c875220e9 Default canonifier change to only remove first timestamp in line
In the past, we used a default canonifier, which removes everything that
looks like a timestamp from log files. The goal of this is to prevent
logs from changing, e.g., due to local system times ending up in log
files.

This, however, also has the side-effect of removing information that is
parsed from protocols which probably should be part of our tests.
There is at least one test (1999 certificates) where the entire test
output was essentially removed by the canonifier.

GH-4521 was similarly masked by this.

This commit changes the default canonifier, so that only the first
timestamp in a line is removed. This should skip timestamps that are
likely to change while keeping timestamps that are parsed
from protocol information.

A pass has been made over the tests, with some additional adjustments
for cases which require the old canonifier.

There are some cases in which we probably could go further and not
remove timestamps at all - that, however, seems like a follow-up
project.
2025-06-18 15:41:48 +01:00
..
spicy spicy/diff-remove-timestamp: Fix missing -e 2024-01-31 14:06:32 +01:00
can-listen-tcp cluster/websocket: Deprecate $listen_host, introduce $listen_addr 2025-05-30 11:02:41 +02:00
coverage-calc Swap pre-commit yapf for ruff/ruff-format, fix findings 2024-12-11 11:08:37 -07:00
diff-canonifier Default canonifier change to only remove first timestamp in line 2025-06-18 15:41:48 +01:00
diff-canonifier-external Settle on analyzer.log for the dpd.log replacement 2025-06-03 17:33:36 +01:00
diff-canonifier-spicy Integrate the Spicy plugin into Zeek proper. 2023-05-16 10:17:45 +02:00
diff-clean-doctest testing/scripts: Use sed -E everywhere 2024-01-24 11:19:17 +01:00
diff-remove-abspath diff-remove-abspath: Add separate handling of Windows paths 2024-11-06 16:06:17 -07:00
diff-remove-fields Format shell scripts with shfmt. 2021-11-24 23:13:02 +01:00
diff-remove-file-ids Improve a few test canonifiers 2015-09-19 18:08:31 -05:00
diff-remove-first-timestamp Change x509 not_before/not_after to not be based on local timezone 2025-06-18 13:21:35 +01:00
diff-remove-fractions Format shell scripts with shfmt. 2021-11-24 23:13:02 +01:00
diff-remove-openclose-timestamps testing/scripts: Use sed -E everywhere 2024-01-24 11:19:17 +01:00
diff-remove-spicy-abspath testing/scripts: Use sed -E everywhere 2024-01-24 11:19:17 +01:00
diff-remove-timestamps testing/scripts: Use sed -E everywhere 2024-01-24 11:19:17 +01:00
diff-remove-timestamps-and-sort Fix formatting 2022-04-27 23:02:27 +02:00
diff-remove-uids Improve a few test canonifiers 2015-09-19 18:08:31 -05:00
diff-remove-x509-key-info Adjust x509 unit tests to work around OpenSSL 1.0 vs. 1.1 differences 2018-06-29 16:01:23 -05:00
diff-remove-x509-names Merge remote-tracking branch 'origin/master' into topic/dnthayer/ticket1467 2015-10-02 11:31:00 -07:00
diff-sort Format shell scripts with shfmt. 2021-11-24 23:13:02 +01:00
diff-sort-and-remove-abspath baseline & btest updates 2022-05-26 17:39:47 -07:00
diff-sort-set-elements Extend external test suite canonifier with set-sorting logic 2020-08-11 11:26:21 -07:00
external-ca-list.zeek Use .zeek file suffix in unit tests 2019-04-16 16:08:57 -07:00
external-ct-list.zeek External tests: add removed logs to CT list to prevent baseline changes 2025-04-28 16:42:52 +01:00
fake-sendmail Format shell scripts with shfmt. 2021-11-24 23:13:02 +01:00
file-analysis-test.zeek General btest cleanup 2020-08-11 11:26:22 -07:00
has-writer More bro-to-zeek renaming in the unit tests 2019-05-16 02:27:54 -05:00
have-af-packet Include in Jan's AF_PACKET plugin as builtin plugin 2022-10-13 13:29:27 +02:00
have-javascript Add experimental JavaScript support when libnode is available 2023-04-14 11:26:41 +02:00
have-redis Redis: Add btests for the redis backend 2025-03-18 09:32:34 -07:00
have-spicy Provide infrastructure to migrate legacy analyzers to Spicy. 2023-02-01 11:33:48 +01:00
have-spicy-ssl testing: Add have-spicy-ssl helper and update tests 2024-10-29 09:30:01 +01:00
have-tsan btest: Skip core.script-args under TSAN 2024-08-08 15:32:20 +02:00
have-zeromq cluster/backend/zeromq: Add ZeroMQ based cluster backend 2024-12-10 20:33:02 +01:00
hilti-ignore-cxx-errors Deprecate bro_int_t and bro_uint_t 2022-07-12 12:01:23 -07:00
httpd.py Swap pre-commit yapf for ruff/ruff-format, fix findings 2024-12-11 11:08:37 -07:00
rst-filter Going over initial parts of the scripting overview. 2013-08-30 16:23:17 -07:00
run-dnsmasq btest/dns_mgr: Update run-dnsmasq, use --host-record 2025-03-05 12:39:15 +01:00
run-redis-server Redis: Add support for sending AUTH commands during connection 2025-06-03 11:38:38 -07:00
snmp-test.zeek spelling: repetitions 2022-11-18 10:26:33 -05:00
spicy-version Integrate the Spicy plugin into Zeek proper. 2023-05-16 10:17:45 +02:00
update-external-repo-pointer.sh Format shell scripts with shfmt. 2021-11-24 23:13:02 +01:00
wait-for-file Merge remote-tracking branch 'origin/topic/etyp/centos-missing-file-test-fail' 2024-10-04 19:32:48 +02:00