Commit graph

25 commits

Author SHA1 Message Date
Robin Sommer
87e10b5f97 Further threading and API restructuring for logging and input
frameworks.

There were a number of cases that weren't thread-safe. In particular,
we don't use std::string anymore for anything that's passed between
threads (but instead plain old const char*, with manual memmory
managmenet).

This is still a check-point commit, I'll do more testing.
2012-07-19 22:28:30 -07:00
Robin Sommer
f73eb3b086 Reworking thread termination logic.
Turns out the finish methods weren't called correctly, caused by a
mess up with method names which all sounded too similar and the wrong
one ended up being called. I've reworked this by changing the
thread/writer/reader interfaces, which actually also simplifies them
by getting rid of the requirement for writer backends to call their
parent methods (i.e., less opportunity for errors).

This commit also includes the following (because I noticed the problem
above when working on some of these):

     - The ASCII log writer now includes "#start <timestamp>" and
      "#end <timestamp> lines in the each file. The latter supersedes
      Bernhard's "EOF" patch.

      This required a number of tests updates. The standard canonifier
      removes the timestamps, but some tests compare files directly,
      which doesn't work if they aren't printing out the same
      timestamps (like the comm tests).

     - The above required yet another change to the writer API to
       network_time to methods.

     - Renamed ASCII logger "header" options to "meta".

     - Fixes #763 "Escape # when first character in log file line".

All btests pass for me on Linux FC15. Will try MacOS next.
2012-07-19 21:21:53 -07:00
Robin Sommer
06d2fd52bd Merge remote-tracking branch 'origin/topic/bernhard/reader-info'
* origin/topic/bernhard/reader-info:
  fix small bug - now configuration actually is passed.
  add mode to readerinfo - no need to have it separately everywhere anymore.
  introduce reader-info struct analogous to writer-info.
  Introduce support for a table of key/value pairs with further configuration options, with the same userinterface as in the logging interface.
  make writer-info work when debugging is enabled

Conflicts:
	testing/btest/Baseline/scripts.base.frameworks.input.event/out
	testing/btest/Baseline/scripts.base.frameworks.input.executeraw/out
	testing/btest/Baseline/scripts.base.frameworks.input.raw/out
	testing/btest/Baseline/scripts.base.frameworks.input.rereadraw/out
	testing/btest/Baseline/scripts.base.frameworks.input.tableevent/out

Closes #841.
2012-07-02 15:33:08 -07:00
Bernhard Amann
7f83f157fc add mode to readerinfo - no need to have it separately everywhere anymore.
Disable remoteserialization of readerinfo - in contrast to the logging framework this is not needed here (I think).
2012-07-02 10:41:02 -07:00
Bernhard Amann
3559a39d59 introduce reader-info struct analogous to writer-info.
All tests still pass.
2012-07-02 10:03:28 -07:00
Bernhard Amann
f820ee9f5c Introduce support for a table of key/value pairs with further configuration options,
with the same userinterface as in the logging interface.

Not really tested, but tests still work.
2012-06-28 16:16:48 -07:00
Robin Sommer
464732bfce Merge remote-tracking branch 'origin/topic/bernhard/input-threads-merge'
* origin/topic/bernhard/input-threads-merge:
  disable streaming reads from executed commands.
  automatically delete disabled input streams
  small documentation fixes
  Documentation
2012-06-15 15:39:58 -07:00
Bernhard Amann
85e29a9471 Documentation 2012-05-31 14:26:25 -07:00
Robin Sommer
fc907c0090 A set of input framework refactoring, cleanup, and polishing. 2012-05-30 16:38:08 -07:00
Robin Sommer
b37f9e38f6 Input framework merge in progress. 2012-05-25 16:16:53 -07:00
Bernhard Amann
658b188dff filters have been called streams for eternity. And I always was too
lazy to change it everywhere...

Fix that.
2012-05-25 16:11:30 -07:00
Bernhard Amann
2e452dc29f remove last remnants of autostart, which has been removed for quite a while. 2012-05-25 10:49:17 -07:00
Bernhard Amann
3b82d69eb3 Merge remote-tracking branch 'origin/master' into topic/bernhard/input-threads
Conflicts:
	src/CMakeLists.txt
	testing/btest/Baseline/coverage.bare-load-baseline/canonified_loaded_scripts.log
	testing/btest/Baseline/coverage.default-load-baseline/canonified_loaded_scripts.log
2012-05-18 15:26:36 -07:00
Bernhard Amann
f73de0bc8c fix small memory leak (field description given to readers was never freed). 2012-03-22 15:11:42 -07:00
Bernhard Amann
57ffe1be77 completely change interface again.
compiles, not really tested.

basic test works 70% of the time, coredumps in the other 30 - but was not easy to debug on a first glance (most interestingly the crash happens in the logging framework - I wonder how that works).
Other tests are not adjusted to the new interface yet.
2012-03-15 18:41:51 -07:00
Bernhard Amann
7076c64a5e Merge remote-tracking branch 'origin/topic/robin/log-threads' into topic/bernhard/input-threads
(and move a little bit of functionality from ascii reader to backend)

Conflicts:
	src/threading/Manager.cc
2012-03-07 13:42:49 -08:00
Bernhard Amann
417542f283 Merge branch 'topic/bernhard/log-threads' into topic/bernhard/input-threads
Seems to work -- all test pass.
But there are thread-safety issues at the moment, because the constructors of IPAddr and IPPrefix are not thread-safe, but needed by workers.

Conflicts:
	src/logging/Manager.cc
2012-02-27 22:59:08 -08:00
Bernhard Amann
531189b5fd try to make ascii reader a little bit more robust to failure - mainly ignore messages after a reader has disabled itself 2012-02-22 08:56:45 -08:00
Bernhard Amann
91943c2655 * rework script interface, add autostart stream flag that starts up a stream automatically when first filter has been added ( probably the most common use case )
* change internal reader interface again
* remove some quite embarassing bugs that must have been in the interface for rather long
* add different read methods to script & internal interface (like normal, streaming, etc). Not implemented in ascii reader yet.
2012-02-16 15:03:20 -08:00
Bernhard Amann
84883348ec interface documentation.
to a big part stolen from the logging framework
2012-02-16 11:27:10 -08:00
Bernhard Amann
a850cc5992 make filter removal and stream closure asynchronous. 2012-02-15 15:14:04 -08:00
Bernhard Amann
88233efb2c It works. Even including all unit tests.
But: there are still a few places where I am sure that there are race conditions & memory leaks & I do not really like the current interface & I have to add a few more messages between the front and backend.

But - it works :)
2012-02-13 22:29:55 -08:00
Bernhard Amann
8385d5bb2d it compiles :)
But that's all, not tested, don't expect it to do anything but crash.
2012-02-06 17:37:02 -08:00
Bernhard Amann
833e724400 way less compile errors. 2012-02-06 16:14:39 -08:00
Bernhard Amann
f6c6387c52 Merge branch 'topic/bernhard/input' into topic/bernhard/input-threads
most stuff is inplace, logging framework needs a few changes merged before continuing here...

Conflicts:
	src/CMakeLists.txt
	src/LogMgr.h
	src/logging/Manager.cc
	src/main.cc
2012-02-06 10:54:07 -08:00