Closes#997.
* topic/robin/sqlite-merge: (25 commits)
Fix to make sqlite test consistent, and updating coverage baselines
Avoid a CMake warning about 3rdparty looking like a number.
Fixing linker error.
and there is no has-reader.
make sqlite3 executable required and add test-cases for errors
Renaming src/external -> src/3rdparty
fix a few small rough edges (mostly comments that do no longer apply)
fix bug in input-manager regarding enums that a writer reads without 0-terminating the string
actually make sqlite work again (tests passed because the writer was not actually defined because of the define.)
add sqlite distribution.
fix warnings, update baselines, handle rotation
add sqlite tests and fix small vector/set escaping bugs
fix small bug with vectors and sets.
make work with newer AsciiFormatter.
start adding a different text for empty records for the sqlite writer.
no, you will never guess from where I copied this file...
make sqlite support more or less work for logging and input
make sqlite-writer more stable.
make it compile with new version of AsciiInputOutput
and adapt to AsciiInputOutput - seems to work...
...
Conflicts:
scripts/base/frameworks/input/__load__.bro
src/CMakeLists.txt
src/input.bif
src/input/Manager.cc
src/main.cc
src/types.bif
testing/btest/Baseline/coverage.bare-load-baseline/canonified_loaded_scripts.log
testing/btest/Baseline/coverage.default-load-baseline/canonified_loaded_scripts.log
* origin/topic/bernhard/sqlite:
fix a few small rough edges (mostly comments that do no longer apply)
fix bug in input-manager regarding enums that a writer reads without 0-terminating the string
actually make sqlite work again (tests passed because the writer was not actually defined because of the define.)
add sqlite distribution.
fix warnings, update baselines, handle rotation
add sqlite tests and fix small vector/set escaping bugs
fix small bug with vectors and sets.
make work with newer AsciiFormatter.
start adding a different text for empty records for the sqlite writer.
no, you will never guess from where I copied this file...
make sqlite support more or less work for logging and input
make sqlite-writer more stable.
make it compile with new version of AsciiInputOutput
and adapt to AsciiInputOutput - seems to work...
make it compile
add SQLite reader.
...adapt to new api...
now the writer supports tables and vectors.
basic sqlite writer seems to work.
* send end_of_data event for all kind of streams
* send process_finished event containing exit code of child process for executed programs
* move raw-tests to separate directory
* expose name of input stream to readers
* better handling of some error cases in raw reader
* new force_kill option for raw reader which SIGKILLs progesses on exit
The ordering of events how they arrive in the main loop is a bit peculiar at the moment.
The process_finished event arrives in scriptland before all of the other events, even though
it should be sent last. I have not yet fully figured that out.
* origin/topic/bernhard/input-logging-commmon-functions:
add the last of Robins suggestions (separate info-struct for constructors).
port memory leak fix from master
harmonize function naming
move AsciiInputOutput over to threading
and thinking about it, ascii-io doesn't need the separator
change constructors
and factor stuff out the input framework too.
factor out ascii input/output.
std::string accessors to escape_sequence functionality
intermediate commit - it has been over a month since I touched this...
I cleaned up the AsciiInputOutput class somewhat, including renaming
it to AsciiFormatter, renaming some of its methods, and turning the
static methods into members for consistency.
Closes#929.
Sadly there also seems to be another deadlock issue which I am currently
not really able to figure out - on shutdown sometimes (too often) the main
thread + all sqlite threads wait for semaphores or mutexes.
more cases.
It will now not only fire after table-reads have been completed,
but also after the last event of a whole-file-read (or whole-db-read, etc.).
The interface also has been extended a bit to allow readers to
directly fire the event should they so choose. This allows the
event to be fired in direct table-setting/event-sending modes,
which was previously not possible.
* origin/topic/bernhard/input-allow_invalid_types:
to be sure - add a small assertion
add an option to the input framework that allows the user to chose to not die upon encountering files/functions.
That's the last feature for 2.1!
* origin/topic/bernhard/reader-info:
fix small bug - now configuration actually is passed.
add mode to readerinfo - no need to have it separately everywhere anymore.
introduce reader-info struct analogous to writer-info.
Introduce support for a table of key/value pairs with further configuration options, with the same userinterface as in the logging interface.
make writer-info work when debugging is enabled
Conflicts:
testing/btest/Baseline/scripts.base.frameworks.input.event/out
testing/btest/Baseline/scripts.base.frameworks.input.executeraw/out
testing/btest/Baseline/scripts.base.frameworks.input.raw/out
testing/btest/Baseline/scripts.base.frameworks.input.rereadraw/out
testing/btest/Baseline/scripts.base.frameworks.input.tableevent/out
Closes#841.
compiles, not really tested.
basic test works 70% of the time, coredumps in the other 30 - but was not easy to debug on a first glance (most interestingly the crash happens in the logging framework - I wonder how that works).
Other tests are not adjusted to the new interface yet.
* change internal reader interface again
* remove some quite embarassing bugs that must have been in the interface for rather long
* add different read methods to script & internal interface (like normal, streaming, etc). Not implemented in ascii reader yet.
But: there are still a few places where I am sure that there are race conditions & memory leaks & I do not really like the current interface & I have to add a few more messages between the front and backend.
But - it works :)
The only snag is... with the default output format of the log-file writer, the input reader cannot tell if a table or set is empty or uninitialized (both cases use the same character by default). In this case, by default it is assumed that the field/vector is uninitalized.
