zeek/testing/btest
Tim Wojtulewicz e3b22cd21f Merge remote-tracking branch 'origin/topic/vern/emphasize-IDPtrs'
* origin/topic/vern/emphasize-IDPtrs:
  fixup! fixup! shift much of the internal use of ID* identifier pointers over to IDPtr objects
  fixup! shift much of the internal use of ID* identifier pointers over to IDPtr objects
  fixup! shift much of the internal use of ID* identifier pointers over to IDPtr objects
  fixup! shift much of the internal use of ID* identifier pointers over to IDPtr objects
  annotate a number of BTests as unsuited for -O gen-C++ testing due to multiple Zeek runs
  BTest baseline updates for -O gen-C++ - all minor tweaks
  BTest updates for script optimization tracking of BiFs
  regression test for former ASAN issue with script optimization of lambdas
  shift much of the internal use of ID* identifier pointers over to IDPtr objects
  maintenance update for script optimization's knowledge of BiFs
  logger fix for interoperability with -O gen-C++ code
2025-09-03 15:33:43 -07:00
..
af_packet
Baseline Merge remote-tracking branch 'origin/topic/vern/emphasize-IDPtrs' 2025-09-03 15:33:43 -07:00
Baseline.cpp BTest baseline updates for -O gen-C++ - all minor tweaks 2025-09-03 11:19:38 -07:00
Baseline.zam Remove deprecations tagged for v8.1 2025-08-12 10:19:03 -07:00
bifs Revert "Move BinPAC, bifcl, af_packet, and gen_zam submodules into main zeek repo" 2025-08-15 15:11:22 -07:00
broker annotate a number of BTests as unsuited for -O gen-C++ testing due to multiple Zeek runs 2025-09-03 11:19:38 -07:00
cluster cluster/serializer/broker: Do not send empty metadata vectors around 2025-08-20 19:55:38 +02:00
core annotate a number of BTests as unsuited for -O gen-C++ testing due to multiple Zeek runs 2025-09-03 11:19:38 -07:00
coverage annotate a number of BTests as unsuited for -O gen-C++ testing due to multiple Zeek runs 2025-09-03 11:19:38 -07:00
dns_mgr zeek.bif: Introduce blocking_lookup_hostname() 2025-03-05 19:22:57 +01:00
doc testing/btest/*zeek: Comment all @TEST lines 2025-04-17 16:30:23 +02:00
Files btest/cluster/zeromq: Add tests for overload behavior 2025-07-29 11:23:53 +02:00
javascript testing/btest/*js: Comment all @TEST lines 2025-04-17 16:30:23 +02:00
language Merge remote-tracking branch 'origin/topic/vern/emphasize-IDPtrs' 2025-09-03 15:33:43 -07:00
misc
opt BTest updates for script optimization tracking of BiFs 2025-09-03 11:19:38 -07:00
plugins Remove deprecations tagged for v8.1 2025-08-12 10:19:03 -07:00
scripts annotate a number of BTests as unsuited for -O gen-C++ testing due to multiple Zeek runs 2025-09-03 11:19:38 -07:00
signatures testing/btest/*zeek: Comment all @TEST lines 2025-04-17 16:30:23 +02:00
spicy Spicy: Add functions to check if Zeek provides an analyzer of a given name. 2025-07-15 14:22:27 +02:00
supervisor testing/btest/*zeek: Comment all @TEST lines 2025-04-17 16:30:23 +02:00
telemetry
Traces Report PostgreSQL login success only after ReadyForQuery 2025-08-18 10:59:44 +08:00
.gitignore
btest.cfg testing/btest: Remove btest-bg-run sleep 1 2025-03-20 10:11:12 +01:00
Makefile
random.seed
README

This a test suite of small "unit tests" that verify individual pieces of Zeek
functionality.  They all utilize BTest, a simple framework/driver for
writing unit tests.  More information about BTest can be found at
https://github.com/zeek/btest

The test suite's BTest configuration is handled through the
``btest.cfg`` file.  Of particular interest is the "TestDirs" settings,
which specifies which directories BTest will recursively search for
test files.

Significant Subdirectories
==========================

* Baseline/
	Validated baselines for comparison against the output of each
	test on future runs. If the new output differs from the Baseline
	output, then the test fails.

* Traces/
	Packet captures utilized by the various BTest tests.

* scripts/
	This hierarchy of tests emulates the hierarchy of the Zeek scripts/
	directory.

* coverage/
	This collection of tests relates to checking whether we're covering
	everything we want to in terms of tests, documentation, and which
	scripts get loaded in different Zeek configurations.  These tests are
	more prone to fail as new Zeek scripts are developed and added to the
	distribution -- checking the individual test's comments is the best
	place to check for more details on what exactly the test is checking
	and hints on how to fix it when it fails.

Running Tests
=============

Either use the ``make all`` or ``make brief`` ``Makefile`` targets, or
run ``btest`` directly with desired options/arguments.  Examples:

* btest <no arguments>
	If you simply execute btest in this directory with no arguments,
	then all directories listed as "TestDirs" in btest.cfg will be
	searched recursively for test files.


* btest <btest options> test_directory
	You can specify a directory on the command line to run just the
	tests contained in that directory. This is useful if you wish to
	run all of a given type of test, without running all the tests
	there are. For example, "btest scripts" will run all of the Zeek
	script unit tests.


* btest <btest options> test_directory/test_file
	You can specify a single test file to run just that test. This
	is useful when testing a single failing test or when developing
	a new test.

Adding Tests
=============

See either the `BTest documentation
<https://github.com/zeek/btest>`_ or the existing unit
tests for examples of what they actually look like.  The essential
components of a new test include:

* A test file in one of the subdirectories listed in the ``TestDirs``
  of the ``btest.cfg`` file.

* If the unit test requires a known-good baseline output against which
  future tests will be compared (via ``btest-diff``), then that baseline
  output will need to live in the ``Baseline`` directory.  Manually
  adding that is possible, but it's easier to just use the ``-u`` or
  ``-U`` options of ``btest`` to do it for you (using ``btest -d`` on a
  test for which no baseline exists will show you the output so it can
  be verified first before adding/updating the baseline output).

If you create a new top-level testing directory for collecting related
tests, then you'll need to add it to the list of ``TestDirs`` in
``btest.cfg``. Do this only if your test really doesn't fit logically in
any of the extant directories.