The BroFile ctor now wraps /dev/std{in,out,err} string arguments
into the actual FILE* provided by stdio.h because use of the former
directly isn't POSIX compliant and led to subtle differences that
broke unit tests on certain platforms (e.g. OS X redirection of stderr
behavior started differing from Linux). The BroFile (un)serialization
methods already did this kind of logic, so adding it in the ctor also
should make things more consistent.
Some of the reporter-related unit tests looked like they were missing
output because of this, and the coverage test for bare-mode errors
needed tweaking to branch on whether or not libcurl was available
(since the error output differs when elasticsearch isn't there).
- Some baselines for tests in "leaks" group were outdated.
- Changed a few of the cluster/communication tests to terminate
more explicitly instead of relying on btest-bg-wait to kill
processes. This makes the tests finish faster in the success case
and makes the reason for failing clearer in the that case.
When using --enable-debug, values keep track of the last identifier
to which they were bound by storing a ref'd ID pointer. This could
lead to some circular dependencies in which an ID is never reclaimed
because the Val is bound to the ID and the ID is bound to the Val, with
both holding references to each other.
There might be more cases where this feature of --enable-debug caused
a leak, but it showed up in particular when running the
core.leaks.remote unit test due to the internal
SendID("peer_description") call during the handshake between remote
processes. Other tests showed the send_id() BIF leaked more generally.
Tracking the ID last bound to a Val through just the identifier string
instead of a ref'd ID pointer fixes the leak.
* origin/topic/seth/reporter-to-stderr:
A couple of tests for printing reporter messages to STDERR.
Small improvements for printing reporter messages to STDERR.
Reporter warnings and error now print to stderr by default.
Closes#836.
* origin/fastpath:
fix little sneaky bug in input framework with an edge case.
small bug in test script. Still worked, because the internal type checking let this through...
- All 5 intelligence tests pass.
- Some initial memory optimizations done.
- More work needs done to reduce duplicate data in memory.
- Input framework integration.
- Define files to read in the "Bro intelligence format" in Intel::read_files.
- Cluster transparency.
- DNS Zones are a fully supported data type.
- Queries for Intel::DOMAIN values will automatically check in DNS_ZONE intelligence.
An assertion would trigger in the case when a predicate refuses
a new entry and another entry with the same index elements was
already in the table. (I thought that code block was unreachable
... did not think of this case).
If a log filter attempts to write to a path for which a writer is
already instantiated due to remote logging, it will re-use the writer
as long as the fields of the filter and writer are compatible, else
the filter path will be auto-adjusted to not conflict with existing
writer's. Conflicts between two local filters are still always
auto-adjusted even if field types agree (since they could still
be semantically different).
Addresses #842.
There are now two FinishedRotation() methods, one that triggers
post-processing and one that doesn't. There's also insurance built in
against a writer not calling either (or both), in which case we abort
with an internal error.
This changes writer implementations to always respond to rotation
messages in their DoRotate() method, even for failure/no-op cases
with a new RotationFailedMessage. This informs the manager to
decrement its count of pending rotations.
Addresses #860.
Also reenabling the logs-to-elasticsearch.bro script in
test-all-policy.bro, that seems to work now.
* origin/fastpath:
Reworked how the logs-to-elasticsearch scripts works to stop abusing the logging framework.
* origin/fastpath:
Small (potential performance) improvement for logging framework.
Script-level rotation postprocessor fix.
update input framework documentation to reflect want_record change.
Fix crash when encountering an InterpreterException in a predicate in logging or input Framework.
make want_record=T the default for events
Inputframework: did not contain any error handling for this case.
Logging framework: tried to catch the interpreter-exception. However the exception already was caught
by the call-function and not propagated. Instead, call returns a 0-pointer in this case, which
lead to a segmentation fault.
