Commit graph

18 commits

Author SHA1 Message Date
Robin Sommer
87e10b5f97 Further threading and API restructuring for logging and input
frameworks.

There were a number of cases that weren't thread-safe. In particular,
we don't use std::string anymore for anything that's passed between
threads (but instead plain old const char*, with manual memmory
managmenet).

This is still a check-point commit, I'll do more testing.
2012-07-19 22:28:30 -07:00
Robin Sommer
f73eb3b086 Reworking thread termination logic.
Turns out the finish methods weren't called correctly, caused by a
mess up with method names which all sounded too similar and the wrong
one ended up being called. I've reworked this by changing the
thread/writer/reader interfaces, which actually also simplifies them
by getting rid of the requirement for writer backends to call their
parent methods (i.e., less opportunity for errors).

This commit also includes the following (because I noticed the problem
above when working on some of these):

     - The ASCII log writer now includes "#start <timestamp>" and
      "#end <timestamp> lines in the each file. The latter supersedes
      Bernhard's "EOF" patch.

      This required a number of tests updates. The standard canonifier
      removes the timestamps, but some tests compare files directly,
      which doesn't work if they aren't printing out the same
      timestamps (like the comm tests).

     - The above required yet another change to the writer API to
       network_time to methods.

     - Renamed ASCII logger "header" options to "meta".

     - Fixes #763 "Escape # when first character in log file line".

All btests pass for me on Linux FC15. Will try MacOS next.
2012-07-19 21:21:53 -07:00
Robin Sommer
06d2fd52bd Merge remote-tracking branch 'origin/topic/bernhard/reader-info'
* origin/topic/bernhard/reader-info:
  fix small bug - now configuration actually is passed.
  add mode to readerinfo - no need to have it separately everywhere anymore.
  introduce reader-info struct analogous to writer-info.
  Introduce support for a table of key/value pairs with further configuration options, with the same userinterface as in the logging interface.
  make writer-info work when debugging is enabled

Conflicts:
	testing/btest/Baseline/scripts.base.frameworks.input.event/out
	testing/btest/Baseline/scripts.base.frameworks.input.executeraw/out
	testing/btest/Baseline/scripts.base.frameworks.input.raw/out
	testing/btest/Baseline/scripts.base.frameworks.input.rereadraw/out
	testing/btest/Baseline/scripts.base.frameworks.input.tableevent/out

Closes #841.
2012-07-02 15:33:08 -07:00
Bernhard Amann
7f83f157fc add mode to readerinfo - no need to have it separately everywhere anymore.
Disable remoteserialization of readerinfo - in contrast to the logging framework this is not needed here (I think).
2012-07-02 10:41:02 -07:00
Bernhard Amann
3559a39d59 introduce reader-info struct analogous to writer-info.
All tests still pass.
2012-07-02 10:03:28 -07:00
Bernhard Amann
f820ee9f5c Introduce support for a table of key/value pairs with further configuration options,
with the same userinterface as in the logging interface.

Not really tested, but tests still work.
2012-06-28 16:16:48 -07:00
Bernhard Amann
852de4700c automatically delete disabled input streams 2012-06-07 14:12:42 -07:00
Robin Sommer
c0e0481af2 Merge branch 'topic/robin/input-threads-merge'
* topic/robin/input-threads-merge: (130 commits)
  And now it even compiles after my earlier changes.
  A set of input framework refactoring, cleanup, and polishing.
  another small memory leak in ascii reader:
  and another small memory leak when using streaming reads.
  fix another memory lead (when updating tables).
  Input framework merge in progress.
  filters have been called streams for eternity. And I always was too lazy to change it everywhere...
  reactivate network_time check in threading manager. previously this line made all input framework tests fail - it works now. Some of the other recent changes of the threading manager must have fixed that problem.
  fix up the executeraw test - now it works for the first time and does not always fail
  baselines for the autostart removal.
  remove last remnants of autostart, which has been removed for quite a while.
  make input framework source (hopefully) adhere to the usual indentation style. No functional changes.
  fix two memory leaks which occured when one used filters.
  update description to current interface.
  rename a couple of structures and make the names in manager fit the api more.
  fix memory leak in tables and vectors that are read into tables
  fix missing get call for heart beat in benchmark reader.
  fix heart_beat_interval -- initialization in constructor does not work anymore (probably due to change in init ordering?)
  fix memory leak for tables... nearly completely.
  fix a couple more leaks. But - still leaking quite a lot with tables.
  ...
2012-05-30 19:12:18 -07:00
Robin Sommer
fc907c0090 A set of input framework refactoring, cleanup, and polishing. 2012-05-30 16:38:08 -07:00
Robin Sommer
b37f9e38f6 Input framework merge in progress. 2012-05-25 16:16:53 -07:00
Bernhard Amann
2034c10e97 make input framework source (hopefully) adhere to the usual indentation
style. No functional changes.
2012-05-25 10:33:22 -07:00
Bernhard Amann
3b82d69eb3 Merge remote-tracking branch 'origin/master' into topic/bernhard/input-threads
Conflicts:
	src/CMakeLists.txt
	testing/btest/Baseline/coverage.bare-load-baseline/canonified_loaded_scripts.log
	testing/btest/Baseline/coverage.default-load-baseline/canonified_loaded_scripts.log
2012-05-18 15:26:36 -07:00
Bernhard Amann
57ffe1be77 completely change interface again.
compiles, not really tested.

basic test works 70% of the time, coredumps in the other 30 - but was not easy to debug on a first glance (most interestingly the crash happens in the logging framework - I wonder how that works).
Other tests are not adjusted to the new interface yet.
2012-03-15 18:41:51 -07:00
Bernhard Amann
91943c2655 * rework script interface, add autostart stream flag that starts up a stream automatically when first filter has been added ( probably the most common use case )
* change internal reader interface again
* remove some quite embarassing bugs that must have been in the interface for rather long
* add different read methods to script & internal interface (like normal, streaming, etc). Not implemented in ascii reader yet.
2012-02-16 15:03:20 -08:00
Bernhard Amann
a850cc5992 make filter removal and stream closure asynchronous. 2012-02-15 15:14:04 -08:00
Bernhard Amann
88233efb2c It works. Even including all unit tests.
But: there are still a few places where I am sure that there are race conditions & memory leaks & I do not really like the current interface & I have to add a few more messages between the front and backend.

But - it works :)
2012-02-13 22:29:55 -08:00
Bernhard Amann
8385d5bb2d it compiles :)
But that's all, not tested, don't expect it to do anything but crash.
2012-02-06 17:37:02 -08:00
Bernhard Amann
f6c6387c52 Merge branch 'topic/bernhard/input' into topic/bernhard/input-threads
most stuff is inplace, logging framework needs a few changes merged before continuing here...

Conflicts:
	src/CMakeLists.txt
	src/LogMgr.h
	src/logging/Manager.cc
	src/main.cc
2012-02-06 10:54:07 -08:00