Commit graph

124 commits

Author SHA1 Message Date
Jon Siwek
ef5e9caaf4 Changes to how script coverage integrates with test suites.
- BRO_PROFILER_FILE now passes .X* templated filenames to mkstemp
  for generating unique coverage state files.  All test suites
  now use this so each Bro instance writes to a unique coverage file.
- Rearranging Makefile targets.  The general rule is that if the
  all/brief target fails out due to a test failure, then the dependent
  coverage target won't run, but can still be invoked directly later.
  (e.g. make brief || make coverage)
2012-03-01 13:00:44 -06:00
Robin Sommer
f6f9dd952f Fixed for Brofiler merge. 2012-01-26 20:55:24 -08:00
Jon Siwek
b6c3567ba4 Test coverage integration for external tests and complete suite.
coverage.log files are output to each testing dir and reflect
the number of times statement locations of Bro scripts are executed.
2012-01-12 11:58:13 -06:00
Robin Sommer
3d2dc5f5fc Merge remote-tracking branch 'origin/topic/script-reference'
* origin/topic/script-reference: (50 commits)
  A few updates for the FAQ.
  Fixing some doc warnings.
  Forgot to add protocol identifier support for TLS 1.2
  Finished SSL & syslog autodocs.
  Adding the draft SSL extension type next_protocol_negotiation.
  Fix some documentation errors.
  Tweaks.
  A set of script-reference polishing.
  fixed a couple typos in comments
  Add summary documentation to bif files.
  Add ssl and syslog script documentation
  Add Conn and DNS protocol script documentation. (fixes #731)
  Small updates to the default local.bro.
  Documentation updates for HTTP & IRC scripts.
  SSH&FTP Documentation updates.
  Fixing a warning from the documentation generation.
  This completes framework documentation package 4.
  Minor notice documentation tweaks.
  Fix some malformed Broxygen xref roles.
  Minor doc tweaks to init-bare.bro.
  ...

Conflicts:
	aux/broccoli
	aux/broctl
	src/bro.bif
	src/strings.bif

Includes:

    - Updated baselines for autodoc tests.
    - Now excluding stats.bro from external texts, it's not stable.
2012-01-10 14:00:44 -08:00
Jon Siwek
ff1768f857 Minor fixes to external test process. (addresses #298)
- Skip diffing of debug.log always.
- Skip diffing of reporter.log if it only contains an error about
  missing GeoIP support.
- Canonicalize X.509 Distinguished Name subjects since that can vary
  depending on installed OpenSSL version.
2011-12-20 11:47:39 -06:00
Robin Sommer
f1e132cd1a Adding missing script. 2011-12-07 10:28:56 -08:00
Robin Sommer
3c2fa085d4 Adapting diff-all so that it expands globs in both current and
baseline directory.

This way, it now spots if a Baseline file isn't produced anymore.

Closes #677.
2011-12-07 10:03:44 -08:00
Robin Sommer
4cdff8ffd8 Extending .gitignore other external test-suites as well. 2011-10-25 16:33:17 -07:00
Jon Siwek
357341c887 Changing some external testing scripts.
- The absolute path canonifier was overzealously canonifying relevant
  log fields, so it's no longer generally applied to diffing all
  baselines.  I don't think there's any logs that require local
  filesystem path names that aren't already tested by a unit test,
  but if any show up in the future, they can be canonified on a
  case-by-case basis.

- Removed some logs from being diff'd in the diff-all script
  because they're either already covered by a unit test
  (load_scripts.log) or because of difficulty/maintainenance
  tradeoff (prof.log).

Baselines for the external bro-testing repo still need updating.
2011-10-04 10:51:41 -05:00
Robin Sommer
9ee8a9f806 Testing/external scripts no longer compute MD5 checksums for SMTP
entities.

Before, whether they did depended on libmagic. To do that,
smpt/entities.bro gets a new option `never_calc_md5`.

Also restructuring the tests a bit so that load a common
testing-setup.bro scripts that can set a global configuration.
2011-09-15 15:42:10 -07:00
Robin Sommer
4bc18bd3f2 External tests no longer include the full content of mismatching files
in the diagnostics output.
2011-09-15 15:17:31 -07:00
Robin Sommer
6ffde1abc2 Updating baselines and testing helper script. 2011-09-12 12:12:47 -07:00
Robin Sommer
5e1ec1c50e Fix for proxy usage. 2011-09-09 11:56:44 -07:00
Robin Sommer
3a3f58d5df Bugfix for trace updater. 2011-09-08 19:21:52 -07:00
Robin Sommer
7a989c476d Test infrastructure fixes.
- A new helper script to btest-diff a bunch of files simulatiously.

- Fixes to btest.cfg.

- A few further script updates.
2011-09-08 18:50:51 -07:00
Robin Sommer
bd4a629bef Changes to the testing/external infrastructure.
The traces for external test-suites are no longer kept inside the
repositories themselves but downloaded separately via curl. This is
because git is pretty bad at dealing with large files.

See the README for more information.
2011-09-08 12:36:35 -07:00
Robin Sommer
0291083170 Spelling unification. 2011-08-01 21:03:06 -07:00
Robin Sommer
f95b645dd4 Updatin README. 2011-08-01 20:56:22 -07:00
Robin Sommer
0034eeb99a Merge remote-tracking branch 'origin/fastpath'
* origin/fastpath:
  Add git ignore for public trace testing repo path.

(Moved the gitignore entry into the testing/external directory.)
2011-07-12 09:21:33 -07:00
Robin Sommer
42f214b8d0 Test repository directory names no longer need to end in *.git. 2011-07-09 08:44:29 -07:00
Robin Sommer
c738701ffd Cleaning up some testing stuff.
- The Makefiles now run btest with "-f diag.log" so that diangnostics
  output will always be written into that file.

- Makefiles now hardcode path to btest to avoid picking up the wrong version
  if in PATH.

- The canonifier scripts now live in testing/scripts, and they are
  used from both btest/ and external/.

- There's a new diff-remove-uids scripts that removed connection UIDs
  for diffing. The external/* tests now use that by default.

- Timestamp removal now has its own script: diff-remove-timestamps.
  diff-canonifier calls that.

- All Makefile have a "brief" target that runs btest with -b.

- The higher-level directories have Makefile to call the subdirs.
2011-07-05 18:47:08 -07:00
Robin Sommer
80fcabdb47 Polishing the setup for external tests. 2011-06-30 22:54:25 -07:00
Robin Sommer
e24d5d548c Checkpoint. 2011-06-30 16:56:02 -07:00
Robin Sommer
6f44fffcea Starting a small framework for doing regression testing with larger
traces and more complex Bro configurations.

That's "Type 2" in the terminology of
http://www-new.bro-ids.org/devel/projects/testing.html

This leverages btest as well and will replace the set of scripts that
the old private test-suite is using. It can accomodate tests with both
public and private traces.

This is a checkpoint commmit, not much there yet in terms of actual
testing. Also still need to figure out a mechanmism to allow for
different baselines based on which branch we're testing.
2011-06-30 16:55:45 -07:00