In the past, we used a default canonifier, which removes everything that
looks like a timestamp from log files. The goal of this is to prevent
logs from changing, e.g., due to local system times ending up in log
files.
This, however, also has the side-effect of removing information that is
parsed from protocols which probably should be part of our tests.
There is at least one test (1999 certificates) where the entire test
output was essentially removed by the canonifier.
GH-4521 was similarly masked by this.
This commit changes the default canonifier, so that only the first
timestamp in a line is removed. This should skip timestamps that are
likely to change while keeping timestamps that are parsed
from protocol information.
A pass has been made over the tests, with some additional adjustments
for cases which require the old canonifier.
There are some cases in which we probably could go further and not
remove timestamps at all - that, however, seems like a follow-up
project.
All changes in this patch were performed automatically with `shfmt` with
configuration flags specified in `.pre-commit-config.yaml`.
In addition to fixing whitespace the roundtrip through shfmt's AST also
transforms command substitutions
`cmd`
# becomes
$(cmd)
and some redirects
>&2 echo "msg"
# becomes
echo >&2 "msg"
- The Makefiles now run btest with "-f diag.log" so that diangnostics
output will always be written into that file.
- Makefiles now hardcode path to btest to avoid picking up the wrong version
if in PATH.
- The canonifier scripts now live in testing/scripts, and they are
used from both btest/ and external/.
- There's a new diff-remove-uids scripts that removed connection UIDs
for diffing. The external/* tests now use that by default.
- Timestamp removal now has its own script: diff-remove-timestamps.
diff-canonifier calls that.
- All Makefile have a "brief" target that runs btest with -b.
- The higher-level directories have Makefile to call the subdirs.
