zeek/testing/external
Jon Siwek e2dc0092f3 Merge branch 'ntp-rewrite' of https://github.com/mauropalumbo75/zeek
* 'ntp-rewrite' of https://github.com/mauropalumbo75/zeek: (25 commits)
  update tests baseline
  Apply requested changes: - file dpd.sig and TODO comments for signature protocol detection removed - missing doc field filled in events.bif - rename OpCode and ReqCode fields into op_code and req_code respectively - removed unnecessary child method in NTP.h/.cc - main.zeek and ntp-protocol.pac reformatted
  minor changes in the documentation
  fix some initializations
  fix wrong assignment of control key_id/crypto_checksum
  code clean up
  add extension fields parsing
  add extended mac field with 20 byte digest (+4 byte key id)
  update tests and add a new one for key_id and mac
  fix auth field (key_id and mac) in standard and control msg
  remove old NTP record in init-bare.zeek
  fix key_id and digest (WIP)
  fix wrong Assign with reference_id
  add tests for ntp protocol (finished)
  add tests for ntp protocol (WIP)
  fix problem with time vals
  add ntp records to init-bare.zeek
  update ntp analyzer to val_mgr
  extend and refact script-side of NTP analyzer
  extend and refactor several fields
  ...
2019-06-15 19:11:34 -07:00
..
Baseline Starting a small framework for doing regression testing with larger 2011-06-30 16:55:45 -07:00
scripts More bro-to-zeek renaming in the unit tests 2019-05-16 02:27:54 -05:00
.gitignore Update external test suite locations 2018-12-06 20:11:57 -06:00
commit-hash.zeek-testing Merge branch 'ntp-rewrite' of https://github.com/mauropalumbo75/zeek 2019-06-15 19:11:34 -07:00
commit-hash.zeek-testing-private Merge branch 'ntp-rewrite' of https://github.com/mauropalumbo75/zeek 2019-06-15 19:11:34 -07:00
Makefile Update external test suite locations 2018-12-06 20:11:57 -06:00
random.seed Merge remote-tracking branch 'origin/topic/johanna/bit-1612' 2016-07-14 16:26:17 -07:00
README More bro-to-zeek renaming in the unit tests 2019-05-16 02:27:54 -05:00
subdir-btest.cfg Rename all BRO-prefixed environment variables 2019-05-22 00:12:31 -05:00

Test Suite for Large Trace Files
================================

This test-suite runs more complex Zeek configurations on larger trace
files, and compares the results to a pre-established baseline. Due to
their size, both traces and baseline are not part of the main Zeek
repository but kept externally. In addition to the publically provided
files, one can also add a local set to the test-suite for running on
private traces.

Initialization
--------------

Before the test-suite can be run, one needs to download the necessary
files. Test and baselines are kept in git repositories, while any
traces are download directly. A ``Makefile`` is provided to get
everything that's needed initially:

.. console:

    > make init

If you need a proxy to download the traces, enter it into a file
``.proxy`` either in the top-level directory or inside one of the
repositories.

To later update to upstream changes:

.. console:

    > make pull

This updates the tests and the traces as necessary.


Running Tests
-------------

The easiest way to run all tests is simply typing ``make``. Doing so
will iterate through all git repositories found in the current
directory and run the tests in there. Output for failed tests will be
in files ``diag.log`` in the top-level repository directories.

Alternatively, one can also manually run all tests inside a single
test repository:

.. console:

    > cd zeek-testing
    > btest

All the standard ``btest`` options can be used to run individual
tests, get diagnostic output, etc.

Updating Baseline
-----------------

To update a test's baseline, first run ``btest`` in update mode:

.. console:

    > cd zeek-testing
    > btest -u tests/test-you-want-to-update

Then use ``git`` to commit the changes and push the changes upstream
as usual.

Adding a Local Repository
-------------------------

One can add local non-public set of tests (potentially using private
traces) by creating a git repository of a similar structure as the
public one.

If you already have such a private test repository that you want to
include into the test suite, clone it directly into ``<repo-name>``.

If you want to create a new private repository, there's a helper
script to set that up:

.. console:

    > ./scripts/create-new-repo <repo-name> <repo-url>

The first argument is the local name of the repository (it will be
cloned into ``<repo-name>``); and the second is the URL of the git
repository. The repository will be initialized with a few standard
directories as well as a skeleton test in ``<name>/tests``. You can
then edit files as needed. You add trace files by editing
``Traces/traces.cfg``; see the comments in there. For each trace, you
also need to calculate a checksum with ``md5sum`` and put it into
``<url>.md5sum``. The scripts use this to decide if they need to
redownload the trace. Accordingly, if you update a trace, make sure to
also recalculate its checksum. Note that the traces will be downloaded
to ``Traces/`` but must not be added to the git repostiory; there's a
``.gitignore`` installed to prevent that.