Commit graph

49 commits

Author SHA1 Message Date
Bernhard Amann
8875953751 A bunch of more changes for the raw reader
* send end_of_data event for all kind of streams
* send process_finished event containing exit code of child process for executed programs
* move raw-tests to separate directory
* expose name of input stream to readers
* better handling of some error cases in raw reader
* new force_kill option for raw reader which SIGKILLs progesses on exit

The ordering of events how they arrive in the main loop is a bit peculiar at the moment.
The process_finished event arrives in scriptland before all of the other events, even though
it should be sent last. I have not yet fully figured that out.
2013-03-18 21:49:16 -07:00
Robin Sommer
762c034ec2 Merge remote-tracking branch 'origin/topic/bernhard/input-logging-commmon-functions'
* origin/topic/bernhard/input-logging-commmon-functions:
  add the last of Robins suggestions (separate info-struct for constructors).
  port memory leak fix from master
  harmonize function naming
  move AsciiInputOutput over to threading
  and thinking about it, ascii-io doesn't need the separator
  change constructors
  and factor stuff out the input framework too.
  factor out ascii input/output.
  std::string accessors to escape_sequence functionality
  intermediate commit - it has been over a month since I touched this...

I cleaned up the AsciiInputOutput class somewhat, including renaming
it to AsciiFormatter, renaming some of its methods, and turning the
static methods into members for consistency.

Closes #929.
2013-01-23 16:51:54 -08:00
Bernhard Amann
9b2265877d and factor stuff out the input framework too. 2012-12-03 13:41:19 -08:00
Robin Sommer
503412e472 Merge remote-tracking branch 'origin/topic/bernhard/input-end-of-data'
* origin/topic/bernhard/input-end-of-data:
  rename the update_finished event to end_of_data and make it fire in more cases.

Closes #894.
2012-10-12 09:48:58 -07:00
Bernhard Amann
a6d87fcab7 rename the update_finished event to end_of_data and make it fire in
more cases.

It will now not only fire after table-reads have been completed,
but also after the last event of a whole-file-read (or whole-db-read, etc.).

The interface also has been extended a bit to allow readers to
directly fire the event should they so choose. This allows the
event to be fired in direct table-setting/event-sending modes,
which was previously not possible.
2012-10-10 11:51:20 -07:00
Robin Sommer
42355bf6f1 Merge remote-tracking branch 'origin/topic/bernhard/input-allow_invalid_types'
* origin/topic/bernhard/input-allow_invalid_types:
  to be sure - add a small assertion
  add an option to the input framework that allows the user to chose to not die upon encountering files/functions.

That's the last feature for 2.1!
2012-08-22 16:11:28 -07:00
Bernhard Amann
b53be21750 add an option to the input framework that allows the user
to chose to not die upon encountering files/functions.

I am not entirely sure if I like the approach I took for
this, it is a bit... hacky.
2012-08-22 06:46:45 -07:00
Bernhard Amann
ec224ada06 single-line documentation addition to main input framework script. 2012-08-21 22:17:28 -07:00
Bernhard Amann
a3798070da update input framework documentation to reflect want_record change. 2012-07-27 07:33:04 -07:00
Bernhard Amann
76ea182387 make want_record=T the default for events 2012-07-26 21:13:49 -07:00
Robin Sommer
06d2fd52bd Merge remote-tracking branch 'origin/topic/bernhard/reader-info'
* origin/topic/bernhard/reader-info:
  fix small bug - now configuration actually is passed.
  add mode to readerinfo - no need to have it separately everywhere anymore.
  introduce reader-info struct analogous to writer-info.
  Introduce support for a table of key/value pairs with further configuration options, with the same userinterface as in the logging interface.
  make writer-info work when debugging is enabled

Conflicts:
	testing/btest/Baseline/scripts.base.frameworks.input.event/out
	testing/btest/Baseline/scripts.base.frameworks.input.executeraw/out
	testing/btest/Baseline/scripts.base.frameworks.input.raw/out
	testing/btest/Baseline/scripts.base.frameworks.input.rereadraw/out
	testing/btest/Baseline/scripts.base.frameworks.input.tableevent/out

Closes #841.
2012-07-02 15:33:08 -07:00
Bernhard Amann
f820ee9f5c Introduce support for a table of key/value pairs with further configuration options,
with the same userinterface as in the logging interface.

Not really tested, but tests still work.
2012-06-28 16:16:48 -07:00
Robin Sommer
fc907c0090 A set of input framework refactoring, cleanup, and polishing. 2012-05-30 16:38:08 -07:00
Robin Sommer
b37f9e38f6 Input framework merge in progress. 2012-05-25 16:16:53 -07:00
Bernhard Amann
658b188dff filters have been called streams for eternity. And I always was too
lazy to change it everywhere...

Fix that.
2012-05-25 16:11:30 -07:00
Bernhard Amann
2e452dc29f remove last remnants of autostart, which has been removed for quite a while. 2012-05-25 10:49:17 -07:00
Bernhard Amann
1170a87769 make benchmark reader hartbeat inverval aware
fix small memleak on tablereader destruction
make timespread better configureable
2012-03-30 09:19:01 -07:00
Bernhard Amann
b47620e501 add a couple more configuration options 2012-03-30 09:18:44 -07:00
Bernhard Amann
ed5374b6d7 and another option for the benchmark reader (constant addition of lines) 2012-03-30 09:17:35 -07:00
Bernhard Amann
579a10d060 make benchmark reader more configureable 2012-03-30 09:17:25 -07:00
Bernhard Amann
355b85fcd7 most of the stuff we should need for benchmarking. next: search memory leaks, after 1.5million simulated
inputs we are leaking about 1Gb of ram...
2012-03-30 09:08:08 -07:00
Bernhard Amann
3286d013c9 forgot two files. 2012-03-16 23:45:10 -07:00
Bernhard Amann
57ffe1be77 completely change interface again.
compiles, not really tested.

basic test works 70% of the time, coredumps in the other 30 - but was not easy to debug on a first glance (most interestingly the crash happens in the logging framework - I wonder how that works).
Other tests are not adjusted to the new interface yet.
2012-03-15 18:41:51 -07:00
Bernhard Amann
b4e6971aab Add regular debugging output for interesting operations (stream/filter operations) to input framework (this was way overdue) 2012-03-14 14:45:53 -07:00
Bernhard Amann
92555badd4 cleanup, more sanity tests, a little bit more documentation 2012-03-11 20:43:26 -07:00
Bernhard Amann
7e5f733826 raw input reader for seth, which can simply read a file into string-events given a line separator. 2012-02-22 09:44:45 -08:00
Bernhard Amann
91943c2655 * rework script interface, add autostart stream flag that starts up a stream automatically when first filter has been added ( probably the most common use case )
* change internal reader interface again
* remove some quite embarassing bugs that must have been in the interface for rather long
* add different read methods to script & internal interface (like normal, streaming, etc). Not implemented in ascii reader yet.
2012-02-16 15:03:20 -08:00
Bernhard Amann
84883348ec interface documentation.
to a big part stolen from the logging framework
2012-02-16 11:27:10 -08:00
Bernhard Amann
88233efb2c It works. Even including all unit tests.
But: there are still a few places where I am sure that there are race conditions & memory leaks & I do not really like the current interface & I have to add a few more messages between the front and backend.

But - it works :)
2012-02-13 22:29:55 -08:00
Bernhard Amann
a14ec02d3b change empty field defenition like in logging framework 2011-12-19 12:43:25 -08:00
Bernhard Amann
a68e6b9fa4 allow sets to be read from files, convenience function for reading a file once,
bug in destructor that could lead to a segfault.
2011-11-29 15:05:09 -08:00
Bernhard Amann
2a6387129c documentation 2011-11-29 11:25:11 -08:00
Bernhard Amann
4975584e01 change Log enum to Input enum. 2011-11-28 13:45:00 -08:00
Bernhard Amann
3c40f00a53 make filters pointers (for inheritance) 2011-11-22 16:09:13 -08:00
Bernhard Amann
53af0544cc re-enable table events 2011-11-21 19:03:35 -08:00
Bernhard Amann
18591b53d4 rename filter to tablefilter in preparation of event filters... 2011-11-21 15:20:52 -08:00
Bernhard Amann
b3f01915fb compiles with basic new filter framework - but crashes on use. 2011-11-20 12:07:50 -08:00
Bernhard Amann
e2c521fc4e start reworking input framework...
does not compile at the moment, but there are a few uncommitted changes that will be reverted in the next commit.
2011-11-18 10:49:20 -08:00
Bernhard Amann
4dd95fcf3c support for uninitialized fields & empty sets and tables.
The only snag is... with the default output format of the log-file writer, the input reader cannot tell if a table or set is empty or uninitialized (both cases use the same character by default). In this case, by default it is assumed that the field/vector is uninitalized.
2011-11-16 23:51:51 -08:00
Bernhard Amann
4fef1e3f8c set & entry separator configuration (with the restriction that they have to be exactly one character long) 2011-11-16 22:47:28 -08:00
Bernhard Amann
cde8153c18 switch to set if record or simple value is desired. 2011-11-15 08:36:03 -08:00
Bernhard Amann
1d39eaf32d small fixes, less leakiness 2011-11-04 15:03:40 -07:00
Bernhard Amann
2aa0f6da57 beautify script calls, track filters 2011-11-04 14:33:34 -07:00
Bernhard Amann
2e3874331d support for filters and little event fix 2011-11-04 12:41:10 -07:00
Bernhard Amann
86730c13dd more complex types... 2011-11-02 15:36:35 -07:00
Bernhard Amann
b245d4168a yay, basic table assignment. 2011-11-02 15:36:35 -07:00
Bernhard Amann
5b0c307f87 very basic input to event working... 2011-11-02 15:36:34 -07:00
Bernhard Amann
3654060246 compiles. sill doesn't do much. 2011-11-02 15:36:34 -07:00
Bernhard Amann
9c8b0dec3b event from c++ to script works (at last...) 2011-11-02 15:36:33 -07:00