The sequence of
- adding a new file
- deleting an existing one
- waiting for Zeek to notice the addition
- re-adding the pre-existing file
was prone to a race: it could happen that Zeek's directory observation would see
the new file in one round, and by the time the next round happens the removal
and re-addition had already happened, thus missing the change and failing the
test.
This avoids the race by placing the removal of the existing file before the
addition of the new one, ensuring that when Zeek notices the addition (and
pushes the test to the next round), it has also seen the removal, so the
re-addition cannot get lost.
When passing an empty string as a directory, the function would produce
filenames starting with a slash even when the given file_name is not an absolute
path. Defaulting to the root directory is likely never intended and might
conveivably be dangerous. The middle "/" is now skipped also if dir is an empty
string.
- Use `-b` most everywhere, it will save time.
- Start some intel tests upon the input file being fully read instead of
at an arbitrary time.
- Improve termination condition for some sumstats/cluster tests.
- Filter uninteresting output from some supervisor tests.
- Test for `notice_policy.log` is no longer needed.
The use of find_all() in extract_email_addrs_vec() extracted occurrences
to an intermediate set and thus lost any sense of ordering.
This changes extract_email_addrs_vec() to use find_all_ordered() and
return all occurrences of email addresses found in the argument,
included duplicates, with their order of occurrence preserved.
Haven't checked different build configurations yet, but all except
a few SumStats tests are stable for me now. The external tests
are also completely failing, but haven't looked at those yet.
I'd expect the portnum component to remain uninitialized in case the
given URI specifies no port number -- however, decompose_uri() raises an
error instead. Will address this in a subsequent commit.
* origin/topic/timw/595-json-perf:
Update COPYING.3rdparty
Use json::emplace to avoid some extra calls to operator[]
Use tessil/unordered-map instead of nlohmann/fifo-map to mitigate performance issues when logging JSON
* origin/topic/robin/gh-239:
Undo a change to btest.cfg from a recent commit
Updating submodule.
Fix zeek-wrapper
Update for renaming BroControl to ZeekControl.
Updating submodule.
GH-239: Rename bro to zeek, bro-config to zeek-config, and bro-path-dev to zeek-path-dev.
This also installs symlinks from "zeek" and "bro-config" to a wrapper
script that prints a deprecation warning.
The btests pass, but this is still WIP. broctl renaming is still
missing.
#239
* is_valid_ip() is now implemented as a BIF instead of in
base/utils/addrs
* The IPv4 and IPv6 regular expressions provided by base/utils/addrs
have been improved/corrected (previously they could possibly match
some invalid IPv4 decimals, or various "zero compressed" IPv6 strings
with too many hextets)
* extract_ip_addresses() should give better results as a result of
the above two points
These functions are now deprecated in favor of alternative versions that
return a vector of strings rather than a table of strings.
Deprecated functions:
- split: use split_string instead.
- split1: use split_string1 instead.
- split_all: use split_string_all instead.
- split_n: use split_string_n instead.
- cat_string_array: see join_string_vec instead.
- cat_string_array_n: see join_string_vec instead.
- join_string_array: see join_string_vec instead.
- sort_string_array: use sort instead instead.
- find_ip_addresses: use extract_ip_addresses instead.
Changed functions:
- has_valid_octets: uses a string_vec parameter instead of string_array.
Addresses BIT-924, BIT-757.
- I've changed/extended the URI record fields a bit:
- path is always the full path including the full file name
- if there's no path, the field still still be set set "/".
- file_name is the full name including extenstion, and
file_base and file_ext split it out.
- Adding a test exercising a bunch of URLs.
It was having trouble because the httpd.py script would start up
a webserver on ipv4 but on some platforms and with some versions
of curl "localhost" will attempt to connect to ::1.
* origin/fastpath:
Add check for curl command to active-http.test
I've moved the check from TEST-EXEC to TEST-REQUIRES. I think it's
fine to just skip the test if curl is not available.
Added a check if the curl command is available when running the
active-http.test so that it fails more quickly and with a clear error
message if it's not available.
- Do stream mode for commands done by exec module, it seems important
in some cases (e.g. ensure requested stdin is fully written).
- For cases where the raw input reader knows the child process has been
reaped, set the childpid member to a sentinel value to indicate such
so we don't later think we should kill it or wait on it anymore.
- More error checking on dup2/close calls. Set sentinel values when
closing ends of pipes to prevent double closing a fd.
- Signal flag not set when raw input reader's child exits as a result
of a signal. Left out a test for this -- might be portability issues
(e.g. Ubuntu seems to do things different regarding the exit code and
also is printing "Killed" to stderr where other platforms don't).
* origin/topic/jsiwek/exec-module:
Exec module changes/fixes.
Coverage test fixes and whitespace/doc tweaks.
Update to make Dir::monitor watch inodes instead of file names.
Updates to use new input framework mechanism to execute command line programs.
Added Exec, Dir, and ActiveHTTP modules.
BIT-1046 #merged.
Conflicts:
magic
testing/btest/Baseline/coverage.default-load-baseline/canonified_loaded_scripts.log
- Give Dir::monitor() a param for the polling interval, so different
dirs can be monitored at different frequencies.
- Fix race in Exec::run() when reading extra output files produced by
a process -- it was possible for Exec::run() to return before all
extra output files had been fully read.
- Add test cases.
