Threaded logging framework.

This is based on Gilbert's code but I ended up refactoring it quite a
bit. That's why I didn't do a direct merge but started with a new
branch and copied things over to adapt. It looks quite a bit different
now as I tried to generalize things a bit more to also support the
Input Framework.

The larger changes code are:

    - Moved all logging code into subdirectory src/logging/. Code
      here is in namespace "logging".

    - Moved all threading code into subdirectory src/threading/. Code
      here is in namespace "threading".

    - Introduced a central thread manager that tracks threads and is
      in charge of termination and (eventually) statistics.

    - Refactored logging independent threading code into base classes
      BasicThread and MsgThread. The former encapsulates all the
      pthread code with simple start/stop methods and provides a
      single Run() method to override.

      The latter is derived from BasicThread and adds bi-directional
      message passing between main and child threads. The hope is that
      the Input Framework can reuse this part quite directly.

    - A log writer is now split into a general WriterFrontend
      (LogEmissary in Gilbert's code) and a type-specific
      WriterBackend. Specific writers are implemented by deriving from
      the latter. (The plugin interface is almost unchanged compared
      to the 2.0 version.).

      Frontend and backend communicate via MsgThread's message
      passing.

    - MsgThread (and thus WriterBackend) has a Heartbeat() method that
      a thread can override to execute code on a regular basis. It's
      triggered roughly once a second by the main thread.

    - Integration into "the rest of Bro". Threads can send messages to
      the reporter and do debugging output; they are hooked into the
      I/O loop for sending messages back; and there's a new debugging
      stream "threading" that logs, well, threading activity.

This all seems to work for the most part, but it's not done yet.

TODO list:

    - Not all tests pass yet. In particular, diffs for the external
      tests seem to indicate some memory problem (no crashes, just an
      occasional weird character).

    - Only tested in --enable-debug mode.

    - Only tested on Linux.

    - Needs leak check.

    - Each log write is currently a single inter-thread message. Bring
      Gilbert's bulk writes back.

    - Code needs further cleanup.

    - Document the class API.

    - Document the internal structure of the logging framework.

    - Check for robustness: live traffic, aborting, signals, etc.

    - Add thread statistics to profile.log (most of the code is there).

    - Customize the OS-visible thread names on platforms that support it.

src/threading/Manager.cc

@@ -0,0 +1,104 @@
+
+#include "Manager.h"
+
+using namespace threading;
+
+Manager::Manager()
+	{
+	DBG_LOG(DBG_THREADING, "Creating thread manager ...");
+
+	did_process = false;
+	next_beat = 0;
+	}
+
+Manager::~Manager()
+	{
+	if ( all_threads.size() )
+		Terminate();
+	}
+
+void Manager::Terminate()
+	{
+	DBG_LOG(DBG_THREADING, "Terminating thread manager ...");
+
+	// First process remaining thread output for the message threads.
+	do Process(); while ( did_process );
+
+	// Signal all to stop.
+	for ( all_thread_list::iterator i = all_threads.begin(); i != all_threads.end(); i++ )
+		(*i)->Stop();
+
+	// Then join them all.
+	for ( all_thread_list::iterator i = all_threads.begin(); i != all_threads.end(); i++ )
+		{
+		(*i)->Join();
+		delete *i;
+		}
+
+	all_threads.clear();
+	msg_threads.clear();
+	}
+
+void Manager::AddThread(BasicThread* thread)
+	{
+	DBG_LOG(DBG_THREADING, "Adding thread %s ...", thread->Name().c_str());
+	all_threads.push_back(thread);
+	}
+
+void Manager::AddMsgThread(MsgThread* thread)
+	{
+	DBG_LOG(DBG_THREADING, "%s is a MsgThread ...", thread->Name().c_str());
+	msg_threads.push_back(thread);
+	}
+
+void Manager::GetFds(int* read, int* write, int* except)
+	{
+	}
+
+double Manager::NextTimestamp(double* network_time)
+	{
+	if ( did_process || ! next_beat == 0 )
+		// If we had something to process last time (or haven't had a
+		// chance to check yet), we want to check for more asap.
+		return timer_mgr->Time();
+
+	// Else we assume we don't have much to do at all and wait for the next heart beat.
+	return next_beat;
+	}
+
+void Manager::Process()
+	{
+	bool do_beat = (next_beat == 0 || network_time >= next_beat);
+
+	did_process = false;
+
+	for ( msg_thread_list::iterator i = msg_threads.begin(); i != msg_threads.end(); i++ )
+		{
+		MsgThread* t = *i;
+
+		if ( do_beat )
+			t->Heartbeat();
+
+		if ( ! t->HasOut() )
+			continue;
+
+		Message* msg = t->RetrieveOut();
+
+		if ( msg->Process() )
+			did_process = true;
+
+		else
+			{
+			string s = msg->Name() + " failed, terminating thread";
+			reporter->Error(s.c_str());
+			t->Stop();
+			}
+
+		delete msg;
+		}
+
+	if ( do_beat )
+		next_beat = network_time + HEART_BEAT_INTERVAL;
+	}
+
+