* origin/topic/dina/modbus:
put some make-up on Modbus analyser
Modbus analyser, added support: FC=20,21
Modbus analyzer,added support: FC=1,2,15,24
Modbus analyzer, current support: FC=3,4,5,6,7,16,22,23
I cleaned up the code a bit, mainly layout style.
I did not include the *.bro scripts for now, but a test script
../testing/btest/scripts/base/protocols/modbus/events.bro that prints
out the value for each event.
Merged the Modbus traces from the ics repository into a single trace
as input for the test. They currently trigger 20 of the 34 events.
Addresses #870.
Added tests for the conditional operator, operator precedence,
modules ("module" and "export" keywords, and the "::" operator), and
for the "copy" keyword.
Also improved tests of max/min values of int, count, and double constants.
* remotes/origin/topic/bernhard/input-warn-on-invalid-numbers:
...and another small change to error handling -> now errors in single lines do not kill processing, but simply ignore the line, log it, and continue.
Ok, this one was a little bit sneaky.
ok, this one might really be a bit too big for 2.1
* origin/fastpath:
Ok, this one is not really necessary for 2.1 and more of a nice-to-have
another small bug found while searching for something else...
Fix two little bugs:
sorry. the patch for the set_separator.
make set_separators different from , work for input framework.
Bug found bei Keith & Seth: input framework was not handling counts and ints out of 32-bit-range correctly.
Before this patch, empty values were not hashed at all. Which had the unfortunate side-effect
that e.g. the lines
TEST -
and
- TEST
have the same hash values. On re-reads that means that the change will
be ignored.
This is probably pretty academic, but this patch changes it and adds a testcase.
Output of the reread test changes due to re-ordering of the output (probably
due to the fact that the internal hash values are changed and thus transferred
in a different order)
Escaped ,'s in sets and vectors were unescaped before tokenization
Handling of zero-length-strings as last element in a set was broken (sets ending with a ,).
Hashing of lines just containing zero-length-strings was broken (now a \0 is appended to each
string before it is hashed - giving us a hash of something for a line just consisting of \0s.
This also allows to differentiate between vectors with varying numbers of zero-length-strings).
Added tests of all built-in Bro data types (including different
representations of constant values, and max./min. values), keywords, and
operators (including special properties of certain operators, such as
short-circuit evaluation and associativity).
* origin/fastpath:
Update documentation for builtin types
Adding an identifier to the SMTP blocklist notices for duplicate suppression.
Doc fixes for signature 'eval' conditions.
Remove orphaned unit tests.
Add type checking for signature 'eval' condition functions.
Otherwise functions could be called with a mismatching argument list
and cause a crash at run-time. The incorrect function type is now
reported at parse-time.
* origin/topic/bernhard/input-allow_invalid_types:
to be sure - add a small assertion
add an option to the input framework that allows the user to chose to not die upon encountering files/functions.
That's the last feature for 2.1!
If the metrics break interval happened to occur between first and second
worker starting up and getting connected to the cluster, the test
would fail because the second worker didn't get a chance to connect and
send data. The test now waits for the cluster setup to complete
before workers send metrics data.
This one would fail intermittently in the cases where log files were
opened or closed on a different second of the time of day from each
other since the "out" baseline contains only a single "#open" and
"#close" tag (indicating all logs opened/closed on same second of time
of day). Piping aggregated log output through the timestamp canonifier
before `uniq` makes it so "#open" and "#close" tags for different
seconds of the time of day are reduced to a single one.
The BroFile ctor now wraps /dev/std{in,out,err} string arguments
into the actual FILE* provided by stdio.h because use of the former
directly isn't POSIX compliant and led to subtle differences that
broke unit tests on certain platforms (e.g. OS X redirection of stderr
behavior started differing from Linux). The BroFile (un)serialization
methods already did this kind of logic, so adding it in the ctor also
should make things more consistent.
Some of the reporter-related unit tests looked like they were missing
output because of this, and the coverage test for bare-mode errors
needed tweaking to branch on whether or not libcurl was available
(since the error output differs when elasticsearch isn't there).
- Some baselines for tests in "leaks" group were outdated.
- Changed a few of the cluster/communication tests to terminate
more explicitly instead of relying on btest-bg-wait to kill
processes. This makes the tests finish faster in the success case
and makes the reason for failing clearer in the that case.
* origin/topic/seth/reporter-to-stderr:
A couple of tests for printing reporter messages to STDERR.
Small improvements for printing reporter messages to STDERR.
Reporter warnings and error now print to stderr by default.
Closes#836.
- All 5 intelligence tests pass.
- Some initial memory optimizations done.
- More work needs done to reduce duplicate data in memory.
- Input framework integration.
- Define files to read in the "Bro intelligence format" in Intel::read_files.
- Cluster transparency.
- DNS Zones are a fully supported data type.
- Queries for Intel::DOMAIN values will automatically check in DNS_ZONE intelligence.
An assertion would trigger in the case when a predicate refuses
a new entry and another entry with the same index elements was
already in the table. (I thought that code block was unreachable
... did not think of this case).
* origin/fastpath:
Small (potential performance) improvement for logging framework.
Script-level rotation postprocessor fix.
update input framework documentation to reflect want_record change.
Fix crash when encountering an InterpreterException in a predicate in logging or input Framework.
make want_record=T the default for events
Small tweak: I added the "same writer" constraint to the loop
condition as well. Makes sense?
* origin/fastpath:
Change path conflicts between log filters to be auto-corrected.
There seems to be a race condition in capturing the external shell's
stdout output reliably. As far as I can tell, Bro's doing everything
correctly though, the log postprocessors gets executed as expected. So
I rewrote the test to capture the output in a separate file first, and
that seems to solve the test failures.
This change makes it so when differing logging filters on the same
stream attempt to write to the same writer/path combination, the path
of the filter doing the later write will be automatically adjusted so
that it does not conflict with the other. The path is adjusted by
appending "-N", where N is the smallest integer greater or equal to 2
required to resolve the path name conflict.
Addresses #842.
