Improved the travis-job test script so that it works outside of Travis.
This can be useful to test changes to the docker container config before
pushing to master, for testing changes on a branch, or for debugging
some problem with the tests running on Travis. This required the
following changes:
-The script no longer requires the TRAVIS env. variable to be set.
-Added an "all" command as a more convenient way to run all steps in a
build.
-Added a "coverity" command-line option to do a coverity build.
-Before building Bro or downloading coverity tools, do a cleanup from any
previous build.
Also other minor improvements (code comments, reorganization, etc.).
* origin/topic/jsiwek/openssl-1.1:
Update install instructions for OpenSSL 1.1 compat
Remove requestorName parameter of ocsp_request event
Adjust x509 unit tests to work around OpenSSL 1.0 vs. 1.1 differences
Fixes for OpenSSL 1.1 support
The advantage of using docker containers is to build and test Bro in
an environment that more closely resembles the environment where Bro
will actually be used. The docker containers currently used enable
testing Bro with all the major versions of gcc (versions 4 through 8),
as well as both python 2 and 3.
The "travis-job" script now takes a second parameter which specifies
a Linux distro to use (specifying "travis" will build and test bro
without using docker).
Improved readability of the output by adding more error checking
and better error and informational messages, and by moving the
unit test diag.log output to just before any external tests are run.
For pull request builds, skip the private tests instead of failing.
Prevent timeouts after 10 minutes of no output by not using the
btest "-b" option.
Decrease build time by not building unneeded components.
On Travis CI, btest was using 32 threads when running with "-j". Reducing
this to "-j 4" seems to make the tests run a bit faster overall and
somewhat reduces the number of test failures.
Remove some output from the Travis CI build log to make it easier
to read.
There is an issue with Travis CI when using encrypted environment
variables. In this case, when btest calls the python function
sys.stdout.isatty() it returns False, which causes btest to output one
line for each test (passed or failed). As a result, it is difficult to
see at a glance which tests failed. Switching to btest brief output to
address this issue.
I've moved .travis.job to testing/scripts/travis-job and adapted
.travis.yml accordingly.
* origin/topic/dnthayer/travis-ci-updates:
Fix information leak in the update-traces script
Add coverity scan and private testing to Travis CI
The OpenSSL on debian 8 refuses to validate one connection; everyone
else can. The easiest fix is to add the CA certificate to the root list.
Once debian has a newer version of OpenSSL, we should be able to remove
it again.
Fixed some test canonifier scripts to read from stdin instead of
from a filename specified as a cmd-line argument. This is needed
in order to be able to reliably use them in a pipeline with other
test canonifiers.
Also removed some unused test canonifier scripts.
* origin/topic/struck/BIT-1277:
[ADD] Added the feature to return 0 content to the python http test server and added functionality for post requests
[ADD] added baseline for the new active-http test and added a test to check for the content-length 0 fix.
[ADD] added baseline for the new exec test and added a test to check for the empty files fix.
[FIX] exec should write an empty string when file is empty instead of the filename
[FIX] Add files to result table even if the files are empty
BIT-1277 #merged
This supports parsing of SNMPv1 (RFC 1157), SNMPv2 (RFC 1901/3416), and
SNMPv2 (RFC 3412). An event is raised for each SNMP PDU type, though
there's not currently any event handlers for them and not a default
snmp.log either. However, simple presence of SNMP is currently visible
now in conn.log service field and known_services.log.
Slice ranges were not correctly determined for negative indices and also
off by one in general (included one more element at the end of the
substring than what actually matched the index range).
It's now equivalent to Python slice notation. Accessing a string at
a single index is also the same as Python except that an out-of-range
index returns an empty string instead of throwing an expection.
I'm merging in the documentation branch to avoid that it keeps getting
out of sync. We still need to work on this a bit more, I'll summarize
in a mail.
* origin/topic/documentation: (68 commits)
Going over initial parts of the scripting overview.
Adding lines in scripting/index.txt. No other change.
Updating submodule.
Pass over the Using Bro section.
Applying doc updates from BIT-1063. (Anthony Verez)
Breaking lines in using/*.
More doc reorg, and a light pass over the first 3 sections.
Starting to put a new structure in place.
Updating submodule(s).
Updating submodule(s).
Adding some temporary testing/demonstration to front page.
Switching btest to topic/robin/parts branch.
Basic cross-referencing UIDs between files, btests, and baselines.
A truly embarassing number of spelling corrections.
Using redirection with bro-cut. Include initial btests for this document.
Corrected a mis-typed RST include.
Initial start for "Using Bro".
Spelling corrections.
Include Notice Policy shortcuts in the Scripting User Manual.
Notice::policy hooks and tests.
...
Conflicts:
src/3rdparty
I made a light pass over the text. Switched the includes over to the
new btest-include and adapted the other TEXT-EXECs a bit.
Also includes more tweaking all over the Sphinx setup.
I edited the text little bit, reorganized the structure somewhat and
extended some parts. I've also simplified the tests a bit, using some
of the BTest tweaks commited in parallel.
* origin/topic/jsiwek/exec-module:
Exec module changes/fixes.
Coverage test fixes and whitespace/doc tweaks.
Update to make Dir::monitor watch inodes instead of file names.
Updates to use new input framework mechanism to execute command line programs.
Added Exec, Dir, and ActiveHTTP modules.
BIT-1046 #merged.
Conflicts:
magic
testing/btest/Baseline/coverage.default-load-baseline/canonified_loaded_scripts.log
- Several places were just using old variable names or not loading
scripts correctly after they'd been renamed/moved.
- Revert/adjust a change in how HTTP file handles are generated that
broke partial content responses.
- Turn some libmagic builtin checks back on; seems some are actually
useful (e.g. text detection seems to be a builtin). The rule going
forward probably will be only to turn off a builtin if we confirm it
causes issues.
- Removed some tests that are redundant or not necessary anymore because
the generic file analysis tests cover them.
- A couple FTP tests still fail that I think need an actual solution via
script changes.
- Give Dir::monitor() a param for the polling interval, so different
dirs can be monitored at different frequencies.
- Fix race in Exec::run() when reading extra output files produced by
a process -- it was possible for Exec::run() to return before all
extra output files had been fully read.
- Add test cases.
