* origin/topic/seth/reporter-to-stderr:
A couple of tests for printing reporter messages to STDERR.
Small improvements for printing reporter messages to STDERR.
Reporter warnings and error now print to stderr by default.
Closes#836.
An assertion would trigger in the case when a predicate refuses
a new entry and another entry with the same index elements was
already in the table. (I thought that code block was unreachable
... did not think of this case).
If a log filter attempts to write to a path for which a writer is
already instantiated due to remote logging, it will re-use the writer
as long as the fields of the filter and writer are compatible, else
the filter path will be auto-adjusted to not conflict with existing
writer's. Conflicts between two local filters are still always
auto-adjusted even if field types agree (since they could still
be semantically different).
Addresses #842.
There are now two FinishedRotation() methods, one that triggers
post-processing and one that doesn't. There's also insurance built in
against a writer not calling either (or both), in which case we abort
with an internal error.
This changes writer implementations to always respond to rotation
messages in their DoRotate() method, even for failure/no-op cases
with a new RotationFailedMessage. This informs the manager to
decrement its count of pending rotations.
Addresses #860.
* origin/fastpath:
Small (potential performance) improvement for logging framework.
Script-level rotation postprocessor fix.
update input framework documentation to reflect want_record change.
Fix crash when encountering an InterpreterException in a predicate in logging or input Framework.
make want_record=T the default for events
Inputframework: did not contain any error handling for this case.
Logging framework: tried to catch the interpreter-exception. However the exception already was caught
by the call-function and not propagated. Instead, call returns a 0-pointer in this case, which
lead to a segmentation fault.
failure.
Once a writer/reader Do* method has returned false, no further ones
will be executed anymore. This is primarily a safety mechanism to make
it easier for writer/reader authors as otherwise they would often need
to track the failure state themselves (because with the now delayed
termination from the earlier commit, furhter messages can now still
arrive for a little bit).
If a thread command fails (like the input framework not finding a
file), that now (1) no longer hangs Bro, and (2) even allows for
propagating error messages back before the thread is stops.
(Actually, the thread doesn't really "stop"; the thread manager keeps
threads around independent of their success; but it no longer polls
them for input.)
Closes#858.
Small tweak: I added the "same writer" constraint to the loop
condition as well. Makes sense?
* origin/fastpath:
Change path conflicts between log filters to be auto-corrected.
This change makes it so when differing logging filters on the same
stream attempt to write to the same writer/path combination, the path
of the filter doing the later write will be automatically adjusted so
that it does not conflict with the other. The path is adjusted by
appending "-N", where N is the smallest integer greater or equal to 2
required to resolve the path name conflict.
Addresses #842.
* origin/fastpath:
and remove superflous print.
add testcase for subrecords and events add missing binary testcase (Baseline is in master, testcase is missing for some reason) make error output for nonmatching event types much more verbose
Add more error handling for close() calls.
add testcase for subrecords to input framework tests
add missing binary testcase (Baseline is in master, testcase is missing for some reason)
make error output for nonmatching event types much more verbose
Since WriterFrontend objects are looked up internally by writer type and
path, and they also expect to write consistent field arguments, it could
be the case that more than one filter of a given stream attempts to
write to the same path (derived either from $path or $path_func fields
of the filter) with the same writer type. This won't work, so now
WriterFrontend objects are bound to the filter that instantiated them so
that we can warn about other filters attempting to write to the
conflicting writer/path and the write can be skipped. Remote logs don't
appear to suffer the same issue due to pre-filtering.
Addresses #842.
* origin/fastpath:
Fix memory leak when processing a thread's input message fails.
add comparator functor to the info maps of readerbackend and readerwriteend.
Fix initialization of WriterFrontend names.
* origin/topic/bernhard/input-fixes:
fix problem with possible access to unititialized memory (thanks robin :) )
and just to be a little bit careful - add check if the field description is long enough. Otherwise there might possibly be an access of uninitialized memory, when someone reads a file that contains just #fields without any following field descriptions.
and like nearly always - forgot the baseline.
Input framework now accepts escaped ascii values as input.
make reading ascii logfiles work when the input separator is different from \t.
This is required, because after the recent changes the info map containst a
char* as key. Without the comparator the map will compare the char addresses
for all operations - which is not really what we want.
The string representation of the writer looked up based on the stream's
enum value instead of the writer's enum value, often causing this
component of the name to be "(null)" since a null pointer was returned
from the lookup.
* topic/robin/master-test: (60 commits)
Script fix for Linux.
Updating test base line.
Another small change to MsgThread API.
Bug fix for BasicThread.
make version_ok return true for TLSv12
Sed usage in canonifier script didn't work on non-Linux systems.
Changing HTTP DPD port 3138 to 3128.
Temporarily removing tuning/logs-to-elasticsearch.bro from the test-all-policy.
More documentation updates.
Revert "Fixing calc_next_rotate to use UTC based time functions."
Some documentation updates for elasticsearch plugin.
Give configure a --disable-perftools option.
Updating tests for the #start/#end change.
Further threading and API restructuring for logging and input frameworks.
Reworking forceful thread termination.
Moving the ASCII writer over to use UNIX I/O rather than stdio.
Further reworking the thread API.
Reworking thread termination logic.
If a thread doesn't terminate, we log that but not longer proceed (because it could hang later still).
Removing the thread kill functionality.
...
