failure.
Once a writer/reader Do* method has returned false, no further ones
will be executed anymore. This is primarily a safety mechanism to make
it easier for writer/reader authors as otherwise they would often need
to track the failure state themselves (because with the now delayed
termination from the earlier commit, furhter messages can now still
arrive for a little bit).
If a thread command fails (like the input framework not finding a
file), that now (1) no longer hangs Bro, and (2) even allows for
propagating error messages back before the thread is stops.
(Actually, the thread doesn't really "stop"; the thread manager keeps
threads around independent of their success; but it no longer polls
them for input.)
Closes#858.
Small tweak: I added the "same writer" constraint to the loop
condition as well. Makes sense?
* origin/fastpath:
Change path conflicts between log filters to be auto-corrected.
There seems to be a race condition in capturing the external shell's
stdout output reliably. As far as I can tell, Bro's doing everything
correctly though, the log postprocessors gets executed as expected. So
I rewrote the test to capture the output in a separate file first, and
that seems to solve the test failures.
This change makes it so when differing logging filters on the same
stream attempt to write to the same writer/path combination, the path
of the filter doing the later write will be automatically adjusted so
that it does not conflict with the other. The path is adjusted by
appending "-N", where N is the smallest integer greater or equal to 2
required to resolve the path name conflict.
Addresses #842.
* origin/fastpath:
and remove superflous print.
add testcase for subrecords and events add missing binary testcase (Baseline is in master, testcase is missing for some reason) make error output for nonmatching event types much more verbose
Add more error handling for close() calls.
add testcase for subrecords to input framework tests
add missing binary testcase (Baseline is in master, testcase is missing for some reason)
make error output for nonmatching event types much more verbose
* origin/fastpath:
Fix complaint from valgrind about uninitialized memory usage.
Fix differing log filters of streams from writing to same writer/path.
Fix tests and error message for to_double BIF
Since WriterFrontend objects are looked up internally by writer type and
path, and they also expect to write consistent field arguments, it could
be the case that more than one filter of a given stream attempts to
write to the same path (derived either from $path or $path_func fields
of the filter) with the same writer type. This won't work, so now
WriterFrontend objects are bound to the filter that instantiated them so
that we can warn about other filters attempting to write to the
conflicting writer/path and the write can be skipped. Remote logs don't
appear to suffer the same issue due to pre-filtering.
Addresses #842.
* origin/fastpath:
Fix memory leak when processing a thread's input message fails.
add comparator functor to the info maps of readerbackend and readerwriteend.
Fix initialization of WriterFrontend names.
* origin/topic/bernhard/input-fixes:
fix problem with possible access to unititialized memory (thanks robin :) )
and just to be a little bit careful - add check if the field description is long enough. Otherwise there might possibly be an access of uninitialized memory, when someone reads a file that contains just #fields without any following field descriptions.
and like nearly always - forgot the baseline.
Input framework now accepts escaped ascii values as input.
make reading ascii logfiles work when the input separator is different from \t.
This is required, because after the recent changes the info map containst a
char* as key. Without the comparator the map will compare the char addresses
for all operations - which is not really what we want.
The string representation of the writer looked up based on the stream's
enum value instead of the writer's enum value, often causing this
component of the name to be "(null)" since a null pointer was returned
from the lookup.
* topic/robin/master-test: (60 commits)
Script fix for Linux.
Updating test base line.
Another small change to MsgThread API.
Bug fix for BasicThread.
make version_ok return true for TLSv12
Sed usage in canonifier script didn't work on non-Linux systems.
Changing HTTP DPD port 3138 to 3128.
Temporarily removing tuning/logs-to-elasticsearch.bro from the test-all-policy.
More documentation updates.
Revert "Fixing calc_next_rotate to use UTC based time functions."
Some documentation updates for elasticsearch plugin.
Give configure a --disable-perftools option.
Updating tests for the #start/#end change.
Further threading and API restructuring for logging and input frameworks.
Reworking forceful thread termination.
Moving the ASCII writer over to use UNIX I/O rather than stdio.
Further reworking the thread API.
Reworking thread termination logic.
If a thread doesn't terminate, we log that but not longer proceed (because it could hang later still).
Removing the thread kill functionality.
...
Instantiations of WriterInfo in RemoteSerializer::ProcessLogCreateWriter()
would leave the network_time member uninitialized which could later
cause localtime_r() calls in Ascii::Timestamp() to return a null pointer
due to the bizarre input and giving that to strftime() causes it to segfault.
Otherwise there might possibly be an access of uninitialized memory, when someone reads a file
that contains just #fields without any following field descriptions.
