Minor adjustments to plugin code/docs.

Mostly whitespace/typos.
Moved some Plugin methods out from public access.

CHANGES

@@ -1,4 +1,72 @@
 
+2.3-83 | 2014-07-30 16:26:11 -0500
+
+  * Minor adjustments to plugin code/docs. (Jon Siwek)
+
+  * Dynamic plugin support. (Robin Sommer)
+
+    - An overview of main functionality is in doc/devel/plugins.rst.
+
+    - This removes the old Plugin macro magic, and hence touches all the
+      existing analyzers to move them to the new API.
+
+    - The plugin API changed to generally use std::strings instead of
+      const char*.
+
+    - There are a number of invocations of PLUGIN_HOOK_
+      {VOID,WITH_RESULT} across the code base, which allow plugins to
+      hook into the processing at those locations. These are macros to
+      make sure the overhead remains as low as possible when no plugin
+      actually defines a hook (i.e., the normal case). See
+      src/plugin/Manager.h for the macros' definition.
+
+    - There's one hook which could be potentially expensive: plugins can
+      be notified if a BroObj they are interested in gets destroyed. But
+      I didn't see a performance impact in my tests (with no such hook
+      defined), and the memory usage doesn't change due to field
+      alignment.
+
+    - Adds a few new accessor methods to various classes to allow
+      plugins to get to that information.
+
+    - network_time cannot be just assigned to anymore, there's now
+      function net_update_time() for that.
+
+    - Redos how builtin variables are initialized, so that it
+      works for plugins as well. No more init_net_var(), but instead
+      bifcl-generated code that registers them.
+
+    - same_type() gets an optional extra argument allowing record type
+      comparision to ignore if field names don't match.
+
+    - There are various changes for adjusting to the now dynamic
+      generation of analyzer instances.
+
+    - The file analysis API gets unified further with the protocol
+      analyzer API (assigning IDs to analyzers; adding Init()/Done()
+      methods; adding subtypes).
+
+    - Adding a new command line option -Q that prints some basic
+      execution time stats. Seems generally useful, and I'm planing
+      to provide a plugin hook for measuring custom stuff.
+
+    - I'm not yet happy with the current conventions for the C++
+      namespaces that plugins are in. I'm planing to clean that up later
+      though, as I have some more branches relying on the current scheme
+      and it will be easier to clean things up once everything is in.
+
+    - There's a new piece of functionality for the file analysis
+      framework: activate analyzers by MIME type. Pieces going in there:
+
+      - File::register_for_mime_type(tag: Analyzer::Tag, mt: string):
+        Associates a file analyzer with a MIME type.
+
+      - File::add_analyzers_for_mime_type(f: fa_file, mtype: string):
+        Activates all analyzers registered for a MIME type for the file.
+
+      - The default file_new() handler calls
+        File::add_analyzers_for_mime_type() with the file's MIME type.
+
 2.3-20 | 2014-07-22 17:41:02 -0700
 
   * Updating submodule(s).

VERSION

@@ -1 +1 @@
-2.3-20
+2.3-83

doc/devel/plugins.rst

@@ -73,7 +73,7 @@ there as follows::
             *p  = (*p - b + 13) % 26 + b;
             }
 
-        return new StringVal(strlen(rot13), rot13);
+        return new StringVal(new BroString(1, rot13, strlen(rot13)));
         %}
 
 The syntax of this file is just like any other ``*.bif`` file; we
@@ -200,7 +200,7 @@ directory.
     activated. See below for more information on activating plugins.
 
 ``lib/bif/``
-    Directory with auto-generated Bro scripts that declare the plugins
+    Directory with auto-generated Bro scripts that declare the plugin's
     bif elements. The files here are produced by ``bifcl``.
 
 By convention, a plugin should put its custom scripts into sub folders
@@ -229,9 +229,9 @@ install``).
 ``make install`` copies over the ``lib`` and ``scripts`` directories,
 as well as the ``__bro_plugin__`` magic file and the ``README`` (which
 you should customize). One can add further CMake ``install`` rules to
-install additional files if neeed.
+install additional files if needed.
 
-``init-plugin`` will never override existing files, so it's safe to
+``init-plugin`` will never overwrite existing files, so it's safe to
 rerun in an existing plugin directory; it only put files in place that
 don't exist yet. That also provides a convenient way to revert a file
 back to what ``init-plugin`` created originally: just delete it and
@@ -420,7 +420,7 @@ At runtime, one then activates a plugin's debugging output with ``-B
 plugin-<name>``, where ``<name>`` is the name of the plugin as
 returned by its ``Configure()`` method, yet with the
 namespace-separator ``::`` replaced with a simple dash. Example: If
-the plugin is called ``Bro::Demo``, use ``-B plugin-Bro-Dome``. As
+the plugin is called ``Bro::Demo``, use ``-B plugin-Bro-Demo``. As
 usual, the debugging output will be recorded to ``debug.log`` if Bro's
 compiled in debug mode.
 

scripts/base/frameworks/files/main.bro

@@ -150,8 +150,8 @@ export {
 	##          for the file isn't currently active or the *args*
 	##          were invalid for the analyzer type.
 	global add_analyzer: function(f: fa_file,
-				      tag: Files::Tag,
+	                              tag: Files::Tag,
-				      args: AnalyzerArgs &default=AnalyzerArgs()): bool;
+	                              args: AnalyzerArgs &default=AnalyzerArgs()): bool;
 
 	## Adds all analyzers associated with a give MIME type to the analysis of
 	## a file.  Note that analyzers added via MIME types cannot take further
@@ -248,13 +248,13 @@ export {
 
 	## Registers a MIME type for an analyzer. If a future file with this type is seen,
 	## the analyzer will be automatically assigned to parsing it. The function *adds*
-	## to all MIME types already registered, it doesn't replace them.   
+	## to all MIME types already registered, it doesn't replace them.
 	##
 	## tag: The tag of the analyzer.
 	##
 	## mt: The MIME type in the form "foo/bar" (case-insensitive).
 	##
-	## Returns: True if the MIME type was successfully registered.  
+	## Returns: True if the MIME type was successfully registered.
 	global register_for_mime_type: function(tag: Analyzer::Tag, mt: string) : bool;
 
 	## Returns a set of all MIME types currently registered for a specific analyzer.

src/file_analysis/Analyzer.h

@@ -31,13 +31,13 @@ public:
 	 * Initializes the analyzer before input processing starts.
 	 */
 	virtual void Init()
-		{ };
+		{ }
 
 	/**
 	 * Finishes the analyzer's operation after all input has been parsed.
 	 */
 	virtual void Done()
-		{ };
+		{ }
 
 	/**
 	 * Subclasses may override this metod to receive file data non-sequentially.

src/main.cc

@@ -962,9 +962,6 @@ int main(int argc, char** argv)
 		}
 
 	reporter->InitOptions();
-
-	init_general_global_var();
-
 	broxygen_mgr->GenerateDocs();
 
 	if ( user_pcap_filter )

src/plugin/Component.h

@@ -76,7 +76,7 @@ public:
 
 protected:
 	/**
-	 * Adds type specific information to the outout of Describe().
+	 * Adds type specific information to the output of Describe().
 	 *
 	 * The default version does nothing.
 	 *

src/plugin/ComponentManager.h

@@ -161,7 +161,7 @@ EnumType* ComponentManager<T, C>::GetTagEnumType() const
 template <class T, class C>
 const std::string& ComponentManager<T, C>::GetComponentName(T tag) const
 	{
-	static const std::string& error = "<error>";
+	static const std::string error = "<error>";
 
 	if ( ! tag )
 		return error;

src/plugin/Manager.cc

@@ -144,7 +144,7 @@ bool Manager::ActivateDynamicPluginInternal(const std::string& name, bool ok_if_
 
 		reporter->Error("plugin %s is not available", name.c_str());
 		return false;
-                }
+		}
 
 	if ( m->second == "" )
 		// Already activated.

src/plugin/Manager.h

@@ -73,7 +73,7 @@ public:
 	void SearchDynamicPlugins(const std::string& dir);
 
 	/**
-	 * Activates a plugin that SearchPlugins() has previously discovered.
+	 * Activates a plugin that SearchDynamicPlugins() has previously discovered.
 	 * Activating a plugin involves loading its dynamic module, making its
 	 * bifs available, and adding its script paths to BROPATH.
 	 *
@@ -86,8 +86,8 @@ public:
 	bool ActivateDynamicPlugin(const std::string& name);
 
 	/**
-	 * Activates plugins that SearchPlugins() has previously discovered. The
+	 * Activates plugins that SearchDynamicPlugins() has previously discovered.
-	 * effect is the same all calling \a ActivePlugin(name) for each plugin.
+	 * The effect is the same all calling \a ActivePlugin(name) for each plugin.
 	 *
 	 * @param all If true, activates all plugins that are found. If false,
 	 * activates only those that should always be activated unconditionally,
@@ -218,7 +218,7 @@ public:
 	// Hook entry functions.
 
 	/**
-	 * Hook that gives plugins a chance to take over loading an input input
+	 * Hook that gives plugins a chance to take over loading an input
 	 * file. This method must be called between InitPreScript() and
 	 * InitPostScript() for each input file Bro is about to load, either
 	 * given on the command line or via @load script directives. The hook can
@@ -288,7 +288,7 @@ public:
 	 * Internal method that registers a bif file's init function for a
 	 * plugin.
 	 *
-	 * @param plugin The plugin to reguster the function for.
+	 * @param plugin The plugin to register the function for.
 	 *
 	 * @param c The init function to register.
 	 */

src/plugin/Plugin.h

@@ -302,7 +302,7 @@ typedef std::list<HookArgument> HookArgumentList;
  *
  * A plugin needs to explicitly register all the functionality it provides.
  * For components, it needs to call AddComponent(); for BiFs AddBifItem();
- * and for hooks EnableHook() and then also implemennt the corresponding
+ * and for hooks EnableHook() and then also implement the corresponding
  * virtual methods.
  *
  */
@@ -346,7 +346,7 @@ public:
 
 	/**
 	 * For dynamic plugins, returns the base directory from which it was
-	 * loaded. For static plugins, returns null.
+	 * loaded. For static plugins, returns an empty string.
 	 **/
 	const std::string& PluginDirectory() const;
 
@@ -377,40 +377,6 @@ public:
 	 */
 	bif_item_list BifItems() const;
 
-	/**
-	 * A function called when the plugin is instantiated to query basic
-	 * configuration parameters.
-	 *
-	 * The plugin must override this method and return a suitably
-	 * initialized configuration object.
-	 *
-	 * @return A configuration describing the plugin.
-	 */
-	virtual Configuration Configure() = 0;
-
-	/**
-	 * First-stage initialization of the plugin called early during Bro's
-	 * startup, before scripts are parsed. This can be overridden by
-	 * derived classes; they must however call the parent's
-	 * implementation.
-	 */
-	virtual void InitPreScript();
-
-	/**
-	 * Second-stage initialization of the plugin called late during Bro's
-	 * startup, after scripts are parsed. This can be overridden by
-	 * derived classes; they must however call the parent's
-	 * implementation.
-	 */
-	virtual void InitPostScript();
-
-	/**
-	 * Finalizer method that derived classes can override for performing
-	 * custom tasks at shutdown.  This can be overridden by derived
-	 * classes; they must however call the parent's implementation.
-	 */
-	virtual void Done();
-
 	/**
 	 * Returns a textual description of the plugin.
 	 *
@@ -445,7 +411,7 @@ public:
 	 * will normally be a Bro script, but it passes through the plugin
 	 * system as well to load files with other extensions as supported by
 	 * any of the current plugins. In other words, calling this method is
-	 * similar to given a file on the command line. Note that the file
+	 * similar to giving a file on the command line. Note that the file
 	 * may be only queued for now, and actually loaded later.
 	 *
 	 * This method must not be called after InitPostScript().
@@ -461,6 +427,29 @@ public:
 protected:
 	friend class Manager;
 
+	/**
+	 * First-stage initialization of the plugin called early during Bro's
+	 * startup, before scripts are parsed. This can be overridden by
+	 * derived classes; they must however call the parent's
+	 * implementation.
+	 */
+	virtual void InitPreScript();
+
+	/**
+	 * Second-stage initialization of the plugin called late during Bro's
+	 * startup, after scripts are parsed. This can be overridden by
+	 * derived classes; they must however call the parent's
+	 * implementation.
+	 */
+	virtual void InitPostScript();
+
+	/**
+	 * Finalizer method that derived classes can override for performing
+	 * custom tasks at shutdown.  This can be overridden by derived
+	 * classes; they must however call the parent's implementation.
+	 */
+	virtual void Done();
+
 	/**
 	 * Registers and activates a component.
 	 *
@@ -471,7 +460,7 @@ protected:
 	/**
 	 * Enables a hook. The corresponding virtual method will now be
 	 * called as Bro's processing proceeds. Note that enabling hooks can
-	 * have performance impaxct as many trigger frequently inside Bro's
+	 * have performance impact as many trigger frequently inside Bro's
 	 * main processing path.
 	 *
 	 * Note that while hooks may be enabled/disabled dynamically at any
@@ -557,16 +546,16 @@ protected:
 	 * from executing it). In the latter case it must provide a matching
 	 * return value.
 	 *
-	 * The default implementation does never handle the call in any way.
+	 * The default implementation never handles the call in any way.
 	 *
 	 * @param func The function being called.
 	 *
 	 * @param args The function arguments. The method can modify the list
-	 * in place long as it ensures matching types and correct reference
+	 * in place as long as it ensures matching types and correct reference
 	 * counting.
 	 *
 	 * @return If the plugin handled the call, a Val with +1 reference
-	 * count containomg the result value to pass back to the interpreter
+	 * count containixnmg the result value to pass back to the interpreter
 	 * (for void functions and events any \a Val is fine; it will be
 	 * ignored; best to use a \c TYPE_ANY). If the plugin did not handle
 	 * the call, it must return null.
@@ -581,15 +570,14 @@ protected:
 	 * inspect the event, or take it over (i.e., prevent the interpreter
 	 * from queuing it itself).
 	 *
-	 * The default implementation does never handle the queuing in any
+	 * The default implementation never handles the queuing in any way.
-	 * way.
 	 *
-	 * @param event The even to be queued. The method can modify it in in
+	 * @param event The event to be queued. The method can modify it in
-	 * place long as it ensures matching types and correct reference
+	 * place as long as it ensures matching types and correct reference
 	 * counting.
 	 *
 	 * @return True if the plugin took charge of the event; in that case
-	 * it must have assumed ownership of the event and the intpreter will
+	 * it must have assumed ownership of the event and the interpreter will
 	 * not do anything further with it. False otherwise.
 	 */
 	virtual bool HookQueueEvent(Event* event);
@@ -609,7 +597,7 @@ protected:
 	virtual void HookUpdateNetworkTime(double network_time);
 
 	/**
-	 * Hook for destruction of objects registerd with
+	 * Hook for destruction of objects registered with
 	 * RequestBroObjDtor(). When Bro's reference counting triggers the
 	 * objects destructor to run, this method will be run. It may also
 	 * run for other objects that this plugin has not registered for.
@@ -652,6 +640,18 @@ protected:
 	virtual void MetaHookPost(HookType hook, const HookArgumentList& args, HookArgument result);
 
 private:
+
+	/**
+	 * A function called when the plugin is instantiated to query basic
+	 * configuration parameters.
+	 *
+	 * The plugin must override this method and return a suitably
+	 * initialized configuration object.
+	 *
+	 * @return A configuration describing the plugin.
+	 */
+	virtual Configuration Configure() = 0;
+
 	/**
 	 * Intializes the plugin's internal configuration. Called by the
 	 * manager before anything else.

src/scan.l

@@ -319,7 +319,7 @@ when	return TOK_WHEN;
 
 @load-plugin{WS}{ID} {
 	const char* plugin = skip_whitespace(yytext + 12);
-    plugin_mgr->ActivateDynamicPlugin(plugin);
+	plugin_mgr->ActivateDynamicPlugin(plugin);
 }
 
 @unload{WS}{FILE}	{
@@ -715,7 +715,7 @@ void add_input_file_at_front(const char* file)
 	if ( ! filename )
 		(void) load_files(file);
 	else
-        	input_files.insert(copy_string(file));
+		input_files.insert(copy_string(file));
 	}
 
 void add_to_name_list(char* s, char delim, name_list& nl)

src/util.h

@@ -178,7 +178,7 @@ bool is_file(const std::string& path);
 // Replaces all occurences of *o* in *s* with *n*.
 extern std::string strreplace(const std::string& s, const std::string& o, const std::string& n);
 
-// Remove all leading and trainling white space from string.
+// Remove all leading and trailing white space from string.
 extern std::string strstrip(std::string s);
 
 extern uint8 shared_hmac_md5_key[16];