Integrate the Spicy plugin into Zeek proper.

This reflects the `spicy-plugin` code as of `d8c296b81cc2a11`.

In addition to moving the code into Zeek's source tree, this comes
with a couple small functional changes:

- `spicyz` no longer tries to infer if it's running from the build
  directory. Instead `ZEEK_SPICY_LIBRARY` can be set to a custom
  location. `zeek-set-path.sh` does that now.

- ZEEK_CONFIG can be set to change what `spicyz -z` print out. This is
  primarily for backwards compatibility.

Some further notes on specifics:

- We raise the minimum Spicy version to 1.8 (i.e., current `main`
  branch).

- Renamed the `compiler/` subdirectory to `spicyz` to avoid
  include-path conflicts with the Spicy headers.

- In `cmake/`, the corresponding PR brings a new/extended version of
  `FindZeek`, which Spicy analyzer packages need. We also now install
  some of the files that the Spicy plugin used to bring for testing,
  so that existing packages keep working.

- For now, this all remains backwards compatible with the current
  `zkg` analyzer templates so that they work with both external and
  integrated Spicy support. Later, once we don't need to support any
  external Spicy plugin versions anymore, we can clean up the
  templates as well.

- All the plugin's tests have moved into the standard test suite. They
  are skipped if configure with `--disable-spicy`.

This holds off on adapting the new code further to Zeek's coding
conventions, so that it remains easier to maintain it in parallel to
the (now legacy) external plugin. We'll make a pass over the
formatting for (presumable) Zeek 6.1.

testing/scripts/diff-canonifier-spicy

@@ -0,0 +1,6 @@
+#! /usr/bin/env bash
+#
+# Remove unstable pieces in Zeek logs produced by thge Spicy tests.
+
+$(dirname $0)/diff-remove-timestamps |
+    $(dirname $0)/diff-remove-abspath

testing/scripts/spicy-version

@@ -0,0 +1,13 @@
+#! /bin/sh
+#
+# Two usages:
+#     - Without argument, prints out the numerical Spicy version.
+#     - With a numerical Spicy version in $1, exit with true iff we have at least that version.
+
+version=$(spicy-config --version-number)
+
+if [ $# = 0 ]; then
+    echo "${version}"
+else
+    test "${version}" -ge "$1"
+fi

testing/scripts/spicy/README

@@ -0,0 +1,2 @@
+These are scripts kept for backwards compatibility with existing Spicy
+packages.

testing/scripts/spicy/canonify-zeek-log

@@ -0,0 +1,17 @@
+#! /usr/bin/env bash
+#
+# Remove unstable pieces in Zeek logs.
+
+# Get us "modern" regexps with sed.
+if [ $(uname) == "Linux" ]; then
+    sed="sed -r"
+else
+    sed="sed -E"
+fi
+
+# File IDs changed between Zeek 3.1 and 3.2.
+
+${sed} 's/^ *#(open|close).(19|20)..-..-..-..-..-..$/#\1 XXXX-XX-XX-XX-XX-XX/g' |
+    ${sed} 's/F[A-Za-z0-9]{14,17}/XXXXXXXXXXXXXXXXX/g' |
+    $(dirname $0)/diff-remove-timestamps |
+    $(dirname $0)/diff-remove-abspath

testing/scripts/spicy/canonify-zeek-log-sorted

@@ -0,0 +1,18 @@
+#! /usr/bin/env bash
+#
+# Remove unstable pieces in Zeek logs.
+
+# Get us "modern" regexps with sed.
+if [ $(uname) == "Linux" ]; then
+    sed="sed -r"
+else
+    sed="sed -E"
+fi
+
+# File IDs changed between Zeek 3.1 and 3.2.
+
+${sed} 's/^ *#(open|close).(19|20)..-..-..-..-..-..$/#\1 XXXX-XX-XX-XX-XX-XX/g' |
+    ${sed} 's/F[A-Za-z0-9]{15,17}/XXXXXXXXXXXXXXXXX/g' |
+    $(dirname $0)/diff-sort |
+    $(dirname $0)/diff-remove-timestamps |
+    $(dirname $0)/diff-remove-abspath

testing/scripts/spicy/diff-remove-abspath

@@ -0,0 +1,12 @@
+#! /usr/bin/env bash
+#
+# Replace absolute paths with the basename.
+
+if [ $(uname) == "Linux" ]; then
+    sed="sed -r"
+else
+    sed="sed -E"
+fi
+
+$sed 's#/+#/#g' |
+    $sed 's#/([^	 :/]{1,}/){1,}([^	 :/]{1,})#<...>/\2#g'

testing/scripts/spicy/diff-remove-timestamps

@@ -0,0 +1,12 @@
+#! /usr/bin/env bash
+#
+# Replace anything which looks like timestamps with XXXs (including the #start/end markers in logs).
+
+# Get us "modern" regexps with sed.
+if [ $(uname) == "Linux" ]; then
+    sed="sed -r"
+else
+    sed="sed -E"
+fi
+
+$sed -e 's/(^|[^0-9])([0-9]{9,10}\.[0-9]{1,8})/\1XXXXXXXXXX.XXXXXX/g' -e 's/^ *#(open|close).(19|20)..-..-..-..-..-../#\1 XXXX-XX-XX-XX-XX-XX/g'

testing/scripts/spicy/diff-sort

@@ -0,0 +1,19 @@
+#! /usr/bin/env bash
+#
+# A diff canonifier that sorts all lines but keeps all comments
+# at the top. It also adds a note at the beginning as a reminder
+# that the output has been sorted.
+
+if [ "$TMP" == "" ]; then
+    TMP=/tmp
+fi
+
+tmp=$TMP/$(basename $0).$$.tmp
+
+cat >$tmp
+
+echo "### NOTE: This file has been sorted with $(basename $0)."
+cat $tmp | grep ^#
+cat $tmp | grep -v ^# | sort -s
+
+rm -f $tmp

testing/scripts/spicy/run-zeek

@@ -0,0 +1,7 @@
+#! /bin/sh
+#
+# Wrapper around Zeek to set environment.
+
+base=$(cd $(dirname $0)/.. && pwd)
+
+LD_PRELOAD=${ZEEK_LD_PRELOAD} DYLD_INSERT_LIBRARIES=${ZEEK_LD_PRELOAD} ASAN_OPTIONS=detect_leaks=0:detect_odr_violation=0 zeek $@

testing/scripts/spicy/spicy-version

@@ -0,0 +1,14 @@
+#! /bin/sh
+#
+# Two usages:
+#     - Without argument, prints out the numerical Spicy version.
+#     - With a numerical Spicy version in $1, exit with true iff we have at least that version.
+
+base=${TEST_BASE-$(cd $(dirname $0)/.. && pwd)}
+nversion=$(spicy-config --version-number)
+
+if [ $# = 0 ]; then
+    echo "${nversion}"
+else
+    test "${nversion}" -ge "$1"
+fi

testing/scripts/spicy/zeek-version

@@ -0,0 +1,20 @@
+#! /bin/sh
+#
+# Two usages:
+#     - Without argument, prints out the numerical Zeek version.
+#     - With a numerical Zeek version in $1, exit with true iff we have at least that version.
+
+base=$(cd $(dirname $0)/.. && pwd)
+
+version=$(zeek-config --version)
+major=$(echo ${version} | cut -d . -f 1)
+minor=$(echo ${version} | cut -d . -f 2)
+patch=$(echo ${version} | cut -d . -f 3)
+
+nversion=$((${major} * 10000 + ${minor} * 100 + ${patch}))
+
+if [ $# = 0 ]; then
+    echo "${nversion}"
+else
+    test "${nversion}" -ge "$1"
+fi