Threaded logging framework.

This is based on Gilbert's code but I ended up refactoring it quite a
bit. That's why I didn't do a direct merge but started with a new
branch and copied things over to adapt. It looks quite a bit different
now as I tried to generalize things a bit more to also support the
Input Framework.

The larger changes code are:

    - Moved all logging code into subdirectory src/logging/. Code
      here is in namespace "logging".

    - Moved all threading code into subdirectory src/threading/. Code
      here is in namespace "threading".

    - Introduced a central thread manager that tracks threads and is
      in charge of termination and (eventually) statistics.

    - Refactored logging independent threading code into base classes
      BasicThread and MsgThread. The former encapsulates all the
      pthread code with simple start/stop methods and provides a
      single Run() method to override.

      The latter is derived from BasicThread and adds bi-directional
      message passing between main and child threads. The hope is that
      the Input Framework can reuse this part quite directly.

    - A log writer is now split into a general WriterFrontend
      (LogEmissary in Gilbert's code) and a type-specific
      WriterBackend. Specific writers are implemented by deriving from
      the latter. (The plugin interface is almost unchanged compared
      to the 2.0 version.).

      Frontend and backend communicate via MsgThread's message
      passing.

    - MsgThread (and thus WriterBackend) has a Heartbeat() method that
      a thread can override to execute code on a regular basis. It's
      triggered roughly once a second by the main thread.

    - Integration into "the rest of Bro". Threads can send messages to
      the reporter and do debugging output; they are hooked into the
      I/O loop for sending messages back; and there's a new debugging
      stream "threading" that logs, well, threading activity.

This all seems to work for the most part, but it's not done yet.

TODO list:

    - Not all tests pass yet. In particular, diffs for the external
      tests seem to indicate some memory problem (no crashes, just an
      occasional weird character).

    - Only tested in --enable-debug mode.

    - Only tested on Linux.

    - Needs leak check.

    - Each log write is currently a single inter-thread message. Bring
      Gilbert's bulk writes back.

    - Code needs further cleanup.

    - Document the class API.

    - Document the internal structure of the logging framework.

    - Check for robustness: live traffic, aborting, signals, etc.

    - Add thread statistics to profile.log (most of the code is there).

    - Customize the OS-visible thread names on platforms that support it.

src/logging/writers/Ascii.h

@@ -0,0 +1,64 @@
+// See the file "COPYING" in the main distribution directory for copyright.
+//
+// Log writer for delimiter-separated ASCII logs.
+
+#ifndef LOGGING_WRITER_ASCII_H
+#define LOGGING_WRITER_ASCII_H
+
+#include "../WriterBackend.h"
+
+namespace logging { namespace writer {
+
+class Ascii : public WriterBackend {
+public:
+	Ascii();
+	~Ascii();
+
+	static WriterBackend* Instantiate()	{ return new Ascii; }
+	static string LogExt();
+
+protected:
+	virtual bool DoInit(string path, int num_fields,
+			    const Field* const * fields);
+	virtual bool DoWrite(int num_fields, const Field* const * fields,
+			     Value** vals);
+	virtual bool DoSetBuf(bool enabled);
+	virtual bool DoRotate(WriterFrontend* writer, string rotated_path,
+			      double open, double close, bool terminating);
+	virtual bool DoFlush();
+	virtual bool DoFinish();
+
+private:
+	bool IsSpecial(string path) 	{ return path.find("/dev/") == 0; }
+	bool DoWriteOne(ODesc* desc, Value* val, const Field* field);
+	bool WriteHeaderField(const string& key, const string& value);
+
+	FILE* file;
+	string fname;
+	ODesc desc;
+
+	// Options set from the script-level.
+	bool output_to_stdout;
+	bool include_header;
+
+	char* separator;
+	int separator_len;
+
+	char* set_separator;
+	int set_separator_len;
+
+	char* empty_field;
+	int empty_field_len;
+
+	char* unset_field;
+	int unset_field_len;
+
+	char* header_prefix;
+	int header_prefix_len;
+};
+
+}
+}
+
+
+#endif