- Give Dir::monitor() a param for the polling interval, so different
dirs can be monitored at different frequencies.
- Fix race in Exec::run() when reading extra output files produced by
a process -- it was possible for Exec::run() to return before all
extra output files had been fully read.
- Add test cases.
- This caused us to lose signatures for POP3 and Bittorrent. These will
need discovered in the repository again when we add scripts
for those analyzers.
Closes#1030.
* origin/topic/seth/packet-filter-updates:
Missed a test fix.
Updating test baselines.
Updates for the PacketFilter framework to simplify it.
Last test update for PacketFilter framework.
Several final fixes for PacketFilter framework.
Packet filter framework checkpoint.
Checkpoint on the packet filter framework.
Initial rework of packet filter framework.
Closes#1021.
* origin/topic/bernhard/input-update:
this event handler fails the unused-event-handlers test because it is a bit of a special case.
...and fix the event ordering issue. Dispatch != QueueEvent
add Terminate to input framework to prevent potential shutdown race-conditions.
fix warning.
fix stderr test. ls behaves differently on errors on linux...
small fixes.
linux does not have strnstr
and close only fds that are currently open (the logging framework really did not like that :) )
A bunch of more changes for the raw reader
make reading from stdout and stderr simultaneously work.
allow sending data to stdin of child process
Streaming reads from external commands work without blocking anything.
replace popen with fork and exec.
change raw reader to use basic c io instead of fdstream encapsulation class.
Adding one todo back in as that's something we indeed still need to do.
* origin/topic/jsiwek/plugins-cleanup:
Fix various documentation/typos; remove a few superfluous things.
http.log now has files taken from request and response bodies in
different fields for each, and can now track multiple files per body.
That is, the "extraction_file" field is now "extracted_request_files"
and "extracted_response_files".
It was mostly redundant when logged, but still can be useful to
inspect at runtime. In the future, a better field for logging
will be available which will be similar to the "service" field
for connection records (there's not any file-format-specific
analyzers that would currently make use of such a thing).
- Remove script-layer data input interface (will be managed directly
by input framework later).
- Only track files internally by file id hash. Chance of collision
too small to justify also tracking unique file string.
- Each transaction ID within a connection is now maintained as
a queue of DNS::Info logging records.
- New function added to the queue.bro script to support
peeking at the new gettable item in the queue without removing it.
Thanks to git this merge was less troublesome that I was afraid it
would be. Not all tests pass yet though (and file hashes have changed
unfortunately).
Conflicts:
cmake
doc/scripts/DocSourcesList.cmake
scripts/base/init-bare.bro
scripts/base/protocols/ftp/main.bro
scripts/base/protocols/irc/dcc-send.bro
scripts/test-all-policy.bro
src/AnalyzerTags.h
src/CMakeLists.txt
src/analyzer/Analyzer.cc
src/analyzer/protocol/file/File.cc
src/analyzer/protocol/file/File.h
src/analyzer/protocol/http/HTTP.cc
src/analyzer/protocol/http/HTTP.h
src/analyzer/protocol/mime/MIME.cc
src/event.bif
src/main.cc
src/util-config.h.in
testing/btest/Baseline/coverage.bare-load-baseline/canonified_loaded_scripts.log
testing/btest/Baseline/coverage.default-load-baseline/canonified_loaded_scripts.log
testing/btest/Baseline/istate.events-ssl/receiver.http.log
testing/btest/Baseline/istate.events-ssl/sender.http.log
testing/btest/Baseline/istate.events/receiver.http.log
testing/btest/Baseline/istate.events/sender.http.log
* origin/topic/bernhard/thread-cleanup:
and just to be really sure - always make threads go through OnWaitForStop
hopefully finally fix last interesting race-condition
it is apparently getting a bit late for changes at important code...
spoke to soon (forgot to comment in line again).
Change thread shutdown again to also work with input framework.
Changing semantics of thread stop methods.
Support for cleaning up threads that have terminated.
Closes#1003.
* origin/topic/bernhard/metrics-samples:
finishing touches, make test more robust, rename function in last again
change names of data structures after talking with seth
make last plugin nicer and samplify sqli detector
add tests for sampler
reservoir sampler. untested.
Closes#997.
* topic/robin/sqlite-merge: (25 commits)
Fix to make sqlite test consistent, and updating coverage baselines
Avoid a CMake warning about 3rdparty looking like a number.
Fixing linker error.
and there is no has-reader.
make sqlite3 executable required and add test-cases for errors
Renaming src/external -> src/3rdparty
fix a few small rough edges (mostly comments that do no longer apply)
fix bug in input-manager regarding enums that a writer reads without 0-terminating the string
actually make sqlite work again (tests passed because the writer was not actually defined because of the define.)
add sqlite distribution.
fix warnings, update baselines, handle rotation
add sqlite tests and fix small vector/set escaping bugs
fix small bug with vectors and sets.
make work with newer AsciiFormatter.
start adding a different text for empty records for the sqlite writer.
no, you will never guess from where I copied this file...
make sqlite support more or less work for logging and input
make sqlite-writer more stable.
make it compile with new version of AsciiInputOutput
and adapt to AsciiInputOutput - seems to work...
...
Conflicts:
scripts/base/frameworks/input/__load__.bro
src/CMakeLists.txt
src/input.bif
src/input/Manager.cc
src/main.cc
src/types.bif
testing/btest/Baseline/coverage.bare-load-baseline/canonified_loaded_scripts.log
testing/btest/Baseline/coverage.default-load-baseline/canonified_loaded_scripts.log
* origin/topic/bernhard/sqlite:
fix a few small rough edges (mostly comments that do no longer apply)
fix bug in input-manager regarding enums that a writer reads without 0-terminating the string
actually make sqlite work again (tests passed because the writer was not actually defined because of the define.)
add sqlite distribution.
fix warnings, update baselines, handle rotation
add sqlite tests and fix small vector/set escaping bugs
fix small bug with vectors and sets.
make work with newer AsciiFormatter.
start adding a different text for empty records for the sqlite writer.
no, you will never guess from where I copied this file...
make sqlite support more or less work for logging and input
make sqlite-writer more stable.
make it compile with new version of AsciiInputOutput
and adapt to AsciiInputOutput - seems to work...
make it compile
add SQLite reader.
...adapt to new api...
now the writer supports tables and vectors.
basic sqlite writer seems to work.
