- Do stream mode for commands done by exec module, it seems important
in some cases (e.g. ensure requested stdin is fully written).
- For cases where the raw input reader knows the child process has been
reaped, set the childpid member to a sentinel value to indicate such
so we don't later think we should kill it or wait on it anymore.
- More error checking on dup2/close calls. Set sentinel values when
closing ends of pipes to prevent double closing a fd.
- Signal flag not set when raw input reader's child exits as a result
of a signal. Left out a test for this -- might be portability issues
(e.g. Ubuntu seems to do things different regarding the exit code and
also is printing "Killed" to stderr where other platforms don't).
logging: table does not contain all required columns (when extending
data structures)
input: table does not contain all required columns (when extending
data structure), wrong sql statement
* send end_of_data event for all kind of streams
* send process_finished event containing exit code of child process for executed programs
* move raw-tests to separate directory
* expose name of input stream to readers
* better handling of some error cases in raw reader
* new force_kill option for raw reader which SIGKILLs progesses on exit
The ordering of events how they arrive in the main loop is a bit peculiar at the moment.
The process_finished event arrives in scriptland before all of the other events, even though
it should be sent last. I have not yet fully figured that out.
Sadly there also seems to be another deadlock issue which I am currently
not really able to figure out - on shutdown sometimes (too often) the main
thread + all sqlite threads wait for semaphores or mutexes.
more cases.
It will now not only fire after table-reads have been completed,
but also after the last event of a whole-file-read (or whole-db-read, etc.).
The interface also has been extended a bit to allow readers to
directly fire the event should they so choose. This allows the
event to be fired in direct table-setting/event-sending modes,
which was previously not possible.
without a final \0 - which means that strings read by the input framework are
unusable by basically all internal functions (like to_count).
the basic test now also checks this.
Thanks at Sheharbano for noticing this.
* remotes/origin/topic/bernhard/input-warn-on-invalid-numbers:
...and another small change to error handling -> now errors in single lines do not kill processing, but simply ignore the line, log it, and continue.
Ok, this one was a little bit sneaky.
ok, this one might really be a bit too big for 2.1
* origin/fastpath:
Ok, this one is not really necessary for 2.1 and more of a nice-to-have
another small bug found while searching for something else...
Fix two little bugs:
sorry. the patch for the set_separator.
make set_separators different from , work for input framework.
Bug found bei Keith & Seth: input framework was not handling counts and ints out of 32-bit-range correctly.
Before this patch, empty values were not hashed at all. Which had the unfortunate side-effect
that e.g. the lines
TEST -
and
- TEST
have the same hash values. On re-reads that means that the change will
be ignored.
This is probably pretty academic, but this patch changes it and adds a testcase.
Output of the reread test changes due to re-ordering of the output (probably
due to the fact that the internal hash values are changed and thus transferred
in a different order)
Escaped ,'s in sets and vectors were unescaped before tokenization
Handling of zero-length-strings as last element in a set was broken (sets ending with a ,).
Hashing of lines just containing zero-length-strings was broken (now a \0 is appended to each
string before it is hashed - giving us a hash of something for a line just consisting of \0s.
This also allows to differentiate between vectors with varying numbers of zero-length-strings).
* origin/topic/bernhard/input-allow_invalid_types:
to be sure - add a small assertion
add an option to the input framework that allows the user to chose to not die upon encountering files/functions.
That's the last feature for 2.1!
An assertion would trigger in the case when a predicate refuses
a new entry and another entry with the same index elements was
already in the table. (I thought that code block was unreachable
... did not think of this case).
* origin/fastpath:
Small (potential performance) improvement for logging framework.
Script-level rotation postprocessor fix.
update input framework documentation to reflect want_record change.
Fix crash when encountering an InterpreterException in a predicate in logging or input Framework.
make want_record=T the default for events
add missing binary testcase (Baseline is in master, testcase is missing for some reason)
make error output for nonmatching event types much more verbose
Generally tried to make them more reliable and execute quicker.
They all now load the listen script as a trick to make sure input
sources are fully read, but also terminate() at appropriate times
so that they don't take more time than needed. They're also all
serialized with the 'comm' group so listening on a port doesn't
interfere with the communication tests.
This lead to hanging bro's because pclose apparently can wait for eternity
if things go wrong. And there probably are a couple of other problems with this approach.
Adjust twotables testcase - now it is faster. Shorten the output
-- because of threading, the results did not always come out in
the same order (it depends on which thread manages to sneak in
the results into the queue earlier).
