Merge remote-tracking branch 'origin/master' into topic/dnthayer/lang-docs

CHANGES

@@ -1,4 +1,101 @@
 
+2.1-195 | 2012-12-03 14:50:33 -0800
+
+  * Catching out-of-memory in patricia tree code. (Bill Parker)
+    
+2.1-194 | 2012-12-03 14:36:26 -0800
+
+  * Renaming ASCII writer filter option 'only_single_header_row' to
+    'tsv'. Also clarifying usage. Closes #912. (Robin Sommer)
+
+2.1-193 | 2012-12-03 14:11:14 -0800
+
+  * Fix a set of bugs with table/set attributes. (Jon Siwek)
+
+    - Identifiers that are initialized with set()/table() constructor
+      expressions now inherit attributes from the expression.  Before,
+      statements like
+
+         const i: set[string] = set() &redef;
+
+      associated the attribute with the set() constructor, but not the
+      "i" identifier, preventing redefinition.  Addresses #866.
+
+    - Allow &default attribute to apply to tables initialized as empty
+      (via either "{ }" or "table()") or if the expression supplied to it
+      can evaluate to a type that's promotable to the same yield type as
+      the table.
+
+2.1-191 | 2012-12-03 14:08:56 -0800
+
+  * Add test of record() constructor to table initializer unit test.
+    (Jon Siwek)
+
+  * Fix table(), set(), vector() constructors in table initializer
+    lists. Also adds type checking of yield values to table()
+    constructor and fixes the type checking of yield values in
+    vector() constructor. Addresses #5. (Jon Siwek)
+
+2.1-188 | 2012-12-03 14:04:29 -0800
+
+  * Hook functions now callable with "hook" expression (i.e., hook is
+    no longer a statement). The return value of the call is an
+    implicit boolean value of T if all hook handlers ran, or F if one
+    hook handler exited as a result of a break statement and
+    potentially prevented other handlers from running.
+
+    Scripts don't need to declare hooks with an explicit return type of bool
+    (internally, that's assumed), and any values given to (optional) return
+    statements in handler definitions are just ignored.
+
+    Addresses #918. (Jon Siwek)
+
+  * Clarification in hook documentation. (Jon Siwek)
+
+2.1-184 | 2012-12-03 13:59:50 -0800
+
+  * Slightly fix up file name extraction from Content-Disposition
+    headers. (Seth Hall)
+
+  * Adding -b flag to bro in unit tests so they run faster.
+
+  * Fixed a DNS attribute issue. Reported by Matt Thompson. (Seth
+    Hall)
+
+  * Adding NEWS placeholder for hooks and CSV mode. (Robin Sommer)
+    
+2.1-178 | 2012-11-23 19:35:32 -0800
+
+  * The ASCII writer now supports a new filter config option
+    "only_single_header_row" that turns the output into CSV format
+    when set to "T". (Carsten Langer)
+
+  * Add new function flavor called a "hook". This new flavor of
+    function behaves like a "synchronous event". See
+    doc/scripts/builtins.rst more details on usage. (Jon Siwek)
+
+  * Improve auto-generated enum documentation. The names of enum types
+    are tracked so that variables holding a value of a given enum type
+    can generate a reference to it instead of just listing the type as
+    a generic "enum". (Jon Siwek)
+
+2.1-171 | 2012-11-23 18:24:15 -0800
+
+  * Fix ambiguity between composite table index and record ctor
+    expressions. If a table type is "global t = table[conn_id, bool]
+    of count", then checking membership like "[c$id, is_orig] in t"
+    now works. Addresses #80. (Jon Siwek)
+
+2.1-169 | 2012-11-23 18:21:32 -0800
+
+  * Fix some warnings from sphinx when building docs. (Jon Siwek)
+
+2.1-167 | 2012-11-14 13:19:17 -0800
+
+  * Add a new BIF "bytestring_to_double" for converting from a binary
+    representation of a double. Addresses #908. (Carsten Langer/Daniel
+    Thayer)
+
 2.1-162 | 2012-11-13 17:29:00 -0800
 
   * Fix modbus register array parsing. (Jon Siwek)

NEWS

@@ -17,6 +17,11 @@ New Functionality
 
 - ssl.log now also records the subject client and issuer certificates.
 
+- Hooks: TODO: Briefly summarize the documention from
+  doc/scripts/builtins.rst here.
+
+- The ASCII writer can now output CSV files on a per filter basis.
+
 Changed Functionality
 ~~~~~~~~~~~~~~~~~~~~~
 

VERSION

@@ -1 +1 @@
-2.1-162
+2.1-195

aux/broccoli

@@ -1 +1 @@
-Subproject commit 907210ce1470724fb386f939cc1b10a4caa2ae39
+Subproject commit a8846fc5b004ffe4e3d00e826d0077ba19518192

doc/scripts/builtins.rst

@@ -524,10 +524,6 @@ The Bro scripting language supports the following built-in types.
     Writing to files like this for logging usually isn't recommended, for better
     logging support see :doc:`/logging`.
 
-.. bro:type:: func
-
-    See :bro:type:`function`.
-
 .. bro:type:: function
 
     Function types in Bro are declared using::
@@ -611,6 +607,87 @@ The Bro scripting language supports the following built-in types.
     identifier and the body of each will be executed in turn.  Ordering
     of execution can be influenced with :bro:attr:`&priority`.
 
+.. bro:type:: hook
+
+    A hook is another flavor of function that shares characteristics of
+    both a :bro:type:`function` and a :bro:type:`event`.  They are like
+    events in that many handler bodies can be defined for the same hook
+    identifier and the order of execution can be enforced with
+    :bro:attr:`&priority`.  They are more like functions in the way they
+    are invoked/called, because, unlike events, their execution is
+    immediate and they do not get scheduled through an event queue.
+    Also, a unique feature of a hook is that a given hook handler body
+    can short-circuit the execution of remaining hook handlers simply by
+    exiting from the body as a result of a ``break`` statement (as
+    opposed to a ``return`` or just reaching the end of the body).
+
+    A hook type is declared like::
+
+        hook( argument* )
+
+    where *argument* is a (possibly empty) comma-separated list of
+    arguments.  For example:
+
+    .. code:: bro
+
+        global myhook: hook(s: string)
+
+    Here ``myhook`` is the hook type identifier and no hook handler
+    bodies have been defined for it yet.  To define some hook handler
+    bodies the syntax looks like:
+
+    .. code:: bro
+
+        hook myhook(s: string) &priority=10
+            {
+            print "priority 10 myhook handler", s;
+            s = "bye";
+            }
+
+        hook myhook(s: string)
+            {
+            print "break out of myhook handling", s;
+            break;
+            }
+
+        hook myhook(s: string) &priority=-5
+            {
+            print "not going to happen", s;
+            }
+
+    Note that the first (forward) declaration of ``myhook`` as a hook
+    type isn't strictly required.  Argument types must match for all
+    hook handlers and any forward declaration of a given hook.
+
+    To invoke immediate execution of all hook handler bodies, they
+    are called similarly to a function, except preceded by the ``hook``
+    keyword:
+
+    .. code:: bro
+
+        hook myhook("hi");
+
+    or
+
+    .. code:: bro
+
+        if ( hook myhook("hi") )
+            print "all handlers ran";
+
+    And the output would look like::
+
+        priority 10 myhook handler, hi
+        break out of myhook handling, bye
+
+    Note how the modification to arguments can be seen by remaining
+    hook handlers.
+
+    The return value of a hook call is an implicit :bro:type:`bool`
+    value with ``T`` meaning that all handlers for the hook were
+    executed and ``F`` meaning that only some of the handlers may have
+    executed due to one handler body exiting as a result of a ``break``
+    statement.
+
 Attributes
 ----------
 

scripts/base/frameworks/intel/main.bro

@@ -59,7 +59,7 @@ export {
 	};
 	
 	## Enum to represent where data came from when it was discovered.
-	## The convenction is to prefix the name with "IN_".
+	## The convention is to prefix the name with ``IN_``.
 	type Where: enum {
 		## A catchall value to represent data of unknown provenance.
 		IN_ANYWHERE,

scripts/base/frameworks/logging/writers/ascii.bro

@@ -1,5 +1,16 @@
 ##! Interface for the ASCII log writer.  Redefinable options are available
 ##! to tweak the output format of ASCII logs.
+##!
+##! The ASCII writer supports currently one writer-specific filter option via
+##! ``config``: setting ``tsv`` to the string ``T`` turns the output into into
+##! "tab-separated-value" mode where only a single header row with the column names
+##! is printed out as meta information, with no "# fields" prepended; no other meta
+##! data gets included in that mode.   
+##! 
+##! Example filter using this::
+##! 
+##!    local my_filter: Log::Filter = [$name = "my-filter", $writer = Log::WRITER_ASCII, $config = table(["tsv"] = "T")]; 
+##!
 
 module LogAscii;
 

scripts/base/protocols/dns/main.bro

@@ -67,7 +67,7 @@ export {
 		ready:         bool            &default=F;
 		## The total number of resource records in a reply message's answer
 		## section.
-		total_answers: count           &default=0;
+		total_answers: count           &optional;
 		## The total number of resource records in a reply message's answer,
 		## authority, and additional sections.
 		total_replies: count           &optional;
@@ -231,6 +231,7 @@ event DNS::do_reply(c: connection, msg: dns_msg, ans: dns_answer, reply: string)
 		Log::write(DNS::LOG, c$dns);
 		# This record is logged and no longer pending.
 		delete c$dns_state$pending[c$dns$trans_id];
+		delete c$dns;
 		}
 	}
 

scripts/base/utils/files.bro

@@ -19,7 +19,7 @@ function generate_extraction_filename(prefix: string, c: connection, suffix: str
 ## the filename.
 function extract_filename_from_content_disposition(data: string): string
 	{
-	local filename = sub(data, /^.*[fF][iI][lL][eE][nN][aA][mM][eE]=/, "");
+	local filename = sub(data, /^.*[fF][iI][lL][eE][nN][aA][mM][eE][[:blank:]]*=[[:blank:]]*/, "");
 	# Remove quotes around the filename if they are there.
 	if ( /^\"/ in filename )
 		filename =  split_n(filename, /\"/, F, 2)[2];

scripts/policy/integration/collective-intel/README

@@ -1,6 +1,4 @@
-Collective Intelligence Framework Integration
+The scripts in this module are for deeper integration with the
-=============================================
+Collective Intelligence Framework (CIF) since Bro's Intel framework
-
+doesn't natively behave the same as CIF nor does it store and maintain
-The scripts in this module are for deeper integration with the Collective Intelligence
+the same data in all cases.
-Framework (CIF) since Bro's Intel framework doesn't natively behave the same as CIF nor
-does it store and maintain the same data in all cases.

scripts/policy/protocols/modbus/track-memmap.bro

@@ -2,11 +2,12 @@
 ##! changes as they are discovered.
 ##!
 ##! .. todo: Not all register reads and write functions are being supported yet.
-module Modbus;
 
 @load base/protocols/modbus
 @load base/utils/directions-and-hosts
 
+module Modbus;
+
 export {
 	redef enum Log::ID += { Modbus::REGISTER_CHANGE_LOG };
 

src/Attr.cc

@@ -71,7 +71,9 @@ void Attr::DescribeReST(ODesc* d) const
 
 		else if ( expr->Type()->Tag() == TYPE_FUNC )
 			{
-			d->Add(":bro:type:`func`");
+			d->Add(":bro:type:`");
+			d->Add(expr->Type()->AsFuncType()->FlavorString());
+			d->Add("`");
 			}
 
 		else
@@ -258,6 +260,11 @@ void Attributes::CheckAttr(Attr* a)
 				// Ok.
 				break;
 
+			if ( type->Tag() == TYPE_TABLE &&
+			     type->AsTableType()->IsUnspecifiedTable() )
+				// Ok.
+				break;
+
 			a->AttrExpr()->Error("&default value has inconsistent type", type);
 			}
 
@@ -288,6 +295,11 @@ void Attributes::CheckAttr(Attr* a)
 					// Ok.
 					break;
 
+				Expr* e = a->AttrExpr();
+				if ( check_and_promote_expr(e, ytype) )
+					// Ok.
+					break;
+
 				Error("&default value has inconsistent type 2");
 				}
 
@@ -401,13 +413,13 @@ void Attributes::CheckAttr(Attr* a)
 
 	case ATTR_GROUP:
 		if ( type->Tag() != TYPE_FUNC ||
-		     ! type->AsFuncType()->IsEvent() )
+		     type->AsFuncType()->Flavor() != FUNC_FLAVOR_EVENT )
 			Error("&group only applicable to events");
 		break;
 
 	case ATTR_ERROR_HANDLER:
 		if ( type->Tag() != TYPE_FUNC ||
-		     ! type->AsFuncType()->IsEvent() )
+		     type->AsFuncType()->Flavor() != FUNC_FLAVOR_EVENT )
 			Error("&error_handler only applicable to events");
 		break;
 

src/BroDoc.cc

@@ -271,6 +271,7 @@ void BroDoc::WriteInterface(const char* heading, char underline,
 	WriteBroDocObjList(state_vars, isPublic, "State Variables", sub, isShort);
 	WriteBroDocObjList(types, isPublic, "Types", sub, isShort);
 	WriteBroDocObjList(events, isPublic, "Events", sub, isShort);
+	WriteBroDocObjList(hooks, isPublic, "Hooks", sub, isShort);
 	WriteBroDocObjList(functions, isPublic, "Functions", sub, isShort);
 	WriteBroDocObjList(redefs, isPublic, "Redefinitions", sub, isShort);
 	}

src/BroDoc.h

@@ -179,6 +179,30 @@ public:
 		all.push_back(o);
 		}
 
+	/**
+	 * Schedules documentation of a hook declared by the script.
+	 * @param o A pointer to a BroDocObj which contains the internal
+	 *        Bro language representation of the script hook and
+	 *        also any associated comments about it.
+	 */
+	void AddHook(const BroDocObj* o)
+		{
+		hooks.push_back(o);
+		all.push_back(o);
+		}
+
+	/**
+	 * Schedules documentation of a hook handler declared by the script.
+	 * @param o A pointer to a BroDocObj which contains the internal
+	 *        Bro language representation of the script hook handler and
+	 *        also any associated comments about it.
+	 */
+	void AddHookHandler(const BroDocObj* o)
+		{
+		hook_handlers.push_back(o);
+		all.push_back(o);
+		}
+
 	/**
 	 * Schedules documentation of a function declared by the script.
 	 * @param o A pointer to a BroDocObj which contains the internal
@@ -241,6 +265,8 @@ protected:
 	BroDocObjList notices;
 	BroDocObjList events;
 	BroDocObjList event_handlers;
+	BroDocObjList hooks;
+	BroDocObjList hook_handlers;
 	BroDocObjMap functions;
 	BroDocObjList redefs;
 

src/Expr.cc

@@ -2663,7 +2663,7 @@ void AssignExpr::EvalIntoAggregate(const BroType* t, Val* aggr, Frame* f) const
 	TableVal* tv = aggr->AsTableVal();
 
 	Val* index = op1->Eval(f);
-	Val* v = op2->Eval(f);
+	Val* v = check_and_promote(op2->Eval(f), t->YieldType(), 1);
 	if ( ! index || ! v )
 		return;
 
@@ -3386,7 +3386,14 @@ Val* TableConstructorExpr::InitVal(const BroType* t, Val* aggr) const
 	if ( IsError() )
 		return 0;
 
-	return op->InitVal(t, aggr);
+	TableType* tt = Type()->AsTableType();
+	TableVal* tval = aggr ? aggr->AsTableVal() : new TableVal(tt, attrs);
+	const expr_list& exprs = op->AsListExpr()->Exprs();
+
+	loop_over_list(exprs, i)
+		exprs[i]->EvalIntoAggregate(t, tval, 0);
+
+	return tval;
 	}
 
 void TableConstructorExpr::ExprDescribe(ODesc* d) const
@@ -3438,7 +3445,7 @@ Val* SetConstructorExpr::Eval(Frame* f) const
 	if ( IsError() )
 		return 0;
 
-	TableVal* aggr = new TableVal(type->AsTableType(), 0);
+	TableVal* aggr = new TableVal(type->AsTableType(), attrs);
 	const expr_list& exprs = op->AsListExpr()->Exprs();
 
 	loop_over_list(exprs, i)
@@ -3456,7 +3463,26 @@ Val* SetConstructorExpr::InitVal(const BroType* t, Val* aggr) const
 	if ( IsError() )
 		return 0;
 
-	return op->InitVal(t, aggr);
+	const BroType* index_type = t->AsTableType()->Indices();
+	TableType* tt = Type()->AsTableType();
+	TableVal* tval = aggr ? aggr->AsTableVal() : new TableVal(tt, attrs);
+	const expr_list& exprs = op->AsListExpr()->Exprs();
+
+	loop_over_list(exprs, i)
+		{
+		Expr* e = exprs[i];
+		Val* element = check_and_promote(e->Eval(0), index_type, 1);
+
+		if ( ! element || ! tval->Assign(element, 0) )
+			{
+			Error(fmt("initialization type mismatch in set"), e);
+			return 0;
+			}
+
+		Unref(element);
+		}
+
+	return tval;
 	}
 
 void SetConstructorExpr::ExprDescribe(ODesc* d) const
@@ -3536,14 +3562,14 @@ Val* VectorConstructorExpr::InitVal(const BroType* t, Val* aggr) const
 	if ( IsError() )
 		return 0;
 
-	VectorVal* vec = aggr->AsVectorVal();
+	VectorType* vt = Type()->AsVectorType();
-	const BroType* vt = vec->Type()->AsVectorType()->YieldType();
+	VectorVal* vec = aggr ? aggr->AsVectorVal() : new VectorVal(vt);
 	const expr_list& exprs = op->AsListExpr()->Exprs();
 
 	loop_over_list(exprs, i)
 		{
 		Expr* e = exprs[i];
-		Val* v = check_and_promote(e->Eval(0), vt, 1);
+		Val* v = check_and_promote(e->Eval(0), t->YieldType(), 1);
 
 		if ( ! v || ! vec->Assign(i, v, e) )
 			{
@@ -4374,7 +4400,7 @@ bool InExpr::DoUnserialize(UnserialInfo* info)
 	return true;
 	}
 
-CallExpr::CallExpr(Expr* arg_func, ListExpr* arg_args)
+CallExpr::CallExpr(Expr* arg_func, ListExpr* arg_args, bool in_hook)
 : Expr(EXPR_CALL)
 	{
 	func = arg_func;
@@ -4394,6 +4420,13 @@ CallExpr::CallExpr(Expr* arg_func, ListExpr* arg_args)
 		return;
 		}
 
+	if ( func_type->AsFuncType()->Flavor() == FUNC_FLAVOR_HOOK && ! in_hook )
+		{
+		func->Error("hook cannot be called directly, use hook operator");
+		SetError();
+		return;
+		}
+
 	if ( ! func_type->MatchesIndex(args) )
 		SetError("argument type mismatch in function call");
 	else
@@ -4402,8 +4435,28 @@ CallExpr::CallExpr(Expr* arg_func, ListExpr* arg_args)
 
 		if ( ! yield )
 			{
-			Error("event called in expression");
+			switch ( func_type->AsFuncType()->Flavor() ) {
-			SetError();
+
+			case FUNC_FLAVOR_FUNCTION:
+				Error("function has no yield type");
+				SetError();
+				break;
+
+			case FUNC_FLAVOR_EVENT:
+				Error("event called in expression, use event statement instead");
+				SetError();
+				break;
+
+			case FUNC_FLAVOR_HOOK:
+				Error("hook has no yield type");
+				SetError();
+				break;
+
+			default:
+				Error("invalid function flavor");
+				SetError();
+				break;
+			}
 			}
 		else
 			SetType(yield->Ref());

src/Expr.h

@@ -747,6 +747,8 @@ public:
 	TableConstructorExpr(ListExpr* constructor_list, attr_list* attrs);
 	~TableConstructorExpr()	{ Unref(attrs); }
 
+	Attributes* Attrs() { return attrs; }
+
 	Val* Eval(Frame* f) const;
 
 protected:
@@ -767,6 +769,8 @@ public:
 	SetConstructorExpr(ListExpr* constructor_list, attr_list* attrs);
 	~SetConstructorExpr()	{ Unref(attrs); }
 
+	Attributes* Attrs() { return attrs; }
+
 	Val* Eval(Frame* f) const;
 
 protected:
@@ -959,7 +963,7 @@ protected:
 
 class CallExpr : public Expr {
 public:
-	CallExpr(Expr* func, ListExpr* args);
+	CallExpr(Expr* func, ListExpr* args, bool in_hook = false);
 	~CallExpr();
 
 	Expr* Func() const	{ return func; }

src/Func.cc

@@ -284,8 +284,8 @@ Val* BroFunc::Call(val_list* args, Frame* parent) const
 #endif
 	if ( ! bodies.size() ) 
 		{
-		// Can only happen for events.
+		// Can only happen for events and hooks.
-		assert(IsEvent());
+		assert(Flavor() == FUNC_FLAVOR_EVENT || Flavor() == FUNC_FLAVOR_HOOK);
 		loop_over_list(*args, i)
 			Unref((*args)[i]);
 		return 0 ;
@@ -309,7 +309,7 @@ Val* BroFunc::Call(val_list* args, Frame* parent) const
 		DescribeDebug(&d, args);
 
 		g_trace_state.LogTrace("%s called: %s\n",
-			IsEvent() ? "event" : "function", d.Description());
+			FType()->FlavorString().c_str(), d.Description());
 		}
 
 	loop_over_list(*args, i)
@@ -348,11 +348,32 @@ Val* BroFunc::Call(val_list* args, Frame* parent) const
 			parent->SetDelayed();
 			break;
 			}
+
+		if ( Flavor() == FUNC_FLAVOR_HOOK )
+			{
+			// Ignore any return values of hook bodies, final return value
+			// depends on whether a body returns as a result of break statement.
+			Unref(result);
+			result = 0;
+
+			if ( flow == FLOW_BREAK )
+				{
+				// Short-circuit execution of remaining hook handler bodies.
+				result = new Val(false, TYPE_BOOL);
+				break;
+				}
+			}
+		}
+
+	if ( Flavor() == FUNC_FLAVOR_HOOK )
+		{
+		if ( ! result )
+			result = new Val(true, TYPE_BOOL);
 		}
 
 	// Warn if the function returns something, but we returned from
 	// the function without an explicit return, or without a value.
-	if ( FType()->YieldType() && FType()->YieldType()->Tag() != TYPE_VOID &&
+	else if ( FType()->YieldType() && FType()->YieldType()->Tag() != TYPE_VOID &&
 	     (flow != FLOW_RETURN /* we fell off the end */ ||
 	      ! result /* explicit return with no result */) &&
 	     ! f->HasDelayed() )
@@ -380,7 +401,7 @@ void BroFunc::AddBody(Stmt* new_body, id_list* new_inits, int new_frame_size,
 
 	new_body = AddInits(new_body, new_inits);
 
-	if ( ! IsEvent() )
+	if ( Flavor() == FUNC_FLAVOR_FUNCTION )
 		{
 		// For functions, we replace the old body with the new one.
 		assert(bodies.size() <= 1);

src/Func.h

@@ -25,7 +25,7 @@ public:
 	virtual ~Func();
 
 	virtual int IsPure() const = 0;
-	int IsEvent() const	{ return FType()->IsEvent(); }
+	function_flavor Flavor() const	{ return FType()->Flavor(); }
 
 	struct Body {
 		Stmt* stmts;

src/ID.cc

@@ -107,7 +107,8 @@ void ID::SetVal(Val* v, Opcode op, bool arg_weak_ref)
 #endif
 
 	if ( type && val &&
-	     type->Tag() == TYPE_FUNC && type->AsFuncType()->IsEvent() )
+	     type->Tag() == TYPE_FUNC &&
+	     type->AsFuncType()->Flavor() == FUNC_FLAVOR_EVENT )
 		{
 		EventHandler* handler = event_registry->Lookup(name);
 		if ( ! handler )
@@ -657,11 +658,19 @@ void ID::DescribeReSTShort(ODesc* d) const
 				break;
 
 			case TYPE_FUNC:
-				d->Add(type->AsFuncType()->IsEvent() ? "event" : type_name(t));
+				d->Add(type->AsFuncType()->FlavorString());
+				break;
+
+			case TYPE_ENUM:
+				if ( is_type )
+					d->Add(type_name(t));
+				else
+					d->Add(type->AsEnumType()->Name().c_str());
 				break;
 
 			default:
 				d->Add(type_name(t));
+				break;
 			}
 			}
 

src/Serializer.cc

@@ -389,23 +389,35 @@ bool Serializer::UnserializeCall(UnserialInfo* info)
 		{
 		if ( info->print )
 			fprintf(info->print, "%s [%.06f] %s(%s)\n",
-				functype->IsEvent() ? "Event" : "Function call",
+				functype->FlavorString().c_str(),
 				time, name, types ? d.Description() : "<ignored>");
 
-		if ( functype->IsEvent() )
+		switch ( functype->Flavor() ) {
+
+		case FUNC_FLAVOR_EVENT:
 			{
 			EventHandler* handler = event_registry->Lookup(name);
 			assert(handler);
+
 			if ( ! info->ignore_callbacks )
 				GotEvent(name, time, handler, args);
+
+			break;
 			}
-		else
+
+		case FUNC_FLAVOR_FUNCTION:
+		case FUNC_FLAVOR_HOOK:
 			if ( ! info->ignore_callbacks )
 				GotFunctionCall(name, time, id->ID_Val()->AsFunc(), args);
+			break;
+
+		default:
+			reporter->InternalError("invalid function flavor");
+			break;
+		}
 
 		if ( info->ignore_callbacks )
 			delete_vals(args);
-
 		}
 	else
 		delete_vals(args);

src/Serializer.h

@@ -125,7 +125,7 @@ protected:
 
 	// This will be increased whenever there is an incompatible change
 	// in the data format.
-	static const uint32 DATA_FORMAT_VERSION = 22;
+	static const uint32 DATA_FORMAT_VERSION = 23;
 
 	ChunkedIO* io;
 

src/Stmt.cc

@@ -1332,7 +1332,10 @@ ReturnStmt::ReturnStmt(Expr* arg_e) : ExprStmt(STMT_RETURN, arg_e)
 		}
 
 	else if ( ! e )
-		Error("return statement needs expression");
+		{
+		if ( ft->Flavor() != FUNC_FLAVOR_HOOK )
+			Error("return statement needs expression");
+		}
 
 	else
 		(void) check_and_promote_expr(e, yt);

src/Type.cc

@@ -651,12 +651,12 @@ bool SetType::DoUnserialize(UnserialInfo* info)
 	return true;
 	}
 
-FuncType::FuncType(RecordType* arg_args, BroType* arg_yield, int arg_is_event)
+FuncType::FuncType(RecordType* arg_args, BroType* arg_yield, function_flavor arg_flavor)
 : BroType(TYPE_FUNC)
 	{
 	args = arg_args;
 	yield = arg_yield;
-	is_event = arg_is_event;
+	flavor = arg_flavor;
 
 	arg_types = new TypeList();
 
@@ -664,6 +664,25 @@ FuncType::FuncType(RecordType* arg_args, BroType* arg_yield, int arg_is_event)
 		arg_types->Append(args->FieldType(i)->Ref());
 	}
 
+string FuncType::FlavorString() const
+	{
+	switch ( flavor ) {
+
+	case FUNC_FLAVOR_FUNCTION:
+		return "function";
+
+	case FUNC_FLAVOR_EVENT:
+		return "event";
+
+	case FUNC_FLAVOR_HOOK:
+		return "hook";
+
+	default:
+		reporter->InternalError("Invalid function flavor");
+		return "invalid_func_flavor";
+	}
+	}
+
 FuncType::~FuncType()
 	{
 	Unref(arg_types);
@@ -698,7 +717,7 @@ void FuncType::Describe(ODesc* d) const
 	{
 	if ( d->IsReadable() )
 		{
-		d->Add(is_event ? "event" : "function");
+		d->Add(FlavorString());
 		d->Add("(");
 		args->DescribeFields(d);
 		d->Add(")");
@@ -712,7 +731,7 @@ void FuncType::Describe(ODesc* d) const
 	else
 		{
 		d->Add(int(Tag()));
-		d->Add(is_event);
+		d->Add(flavor);
 		d->Add(yield != 0);
 		args->DescribeFields(d);
 		if ( yield )
@@ -723,7 +742,7 @@ void FuncType::Describe(ODesc* d) const
 void FuncType::DescribeReST(ODesc* d) const
 	{
 	d->Add(":bro:type:`");
-	d->Add(is_event ? "event" : "function");
+	d->Add(FlavorString());
 	d->Add("`");
 	d->Add(" (");
 	args->DescribeFieldsReST(d, true);
@@ -755,9 +774,30 @@ bool FuncType::DoSerialize(SerialInfo* info) const
 
 	SERIALIZE_OPTIONAL(yield);
 
+	int ser_flavor = 0;
+
+	switch ( flavor ) {
+
+	case FUNC_FLAVOR_FUNCTION:
+		ser_flavor = 0;
+		break;
+
+	case FUNC_FLAVOR_EVENT:
+		ser_flavor = 1;
+		break;
+
+	case FUNC_FLAVOR_HOOK:
+		ser_flavor = 2;
+		break;
+
+	default:
+		reporter->InternalError("Invalid function flavor serialization");
+		break;
+	}
+
 	return args->Serialize(info) &&
 		arg_types->Serialize(info) &&
-		SERIALIZE(is_event);
+		SERIALIZE(ser_flavor);
 	}
 
 bool FuncType::DoUnserialize(UnserialInfo* info)
@@ -774,7 +814,27 @@ bool FuncType::DoUnserialize(UnserialInfo* info)
 	if ( ! arg_types )
 		return false;
 
-	return UNSERIALIZE(&is_event);
+	int ser_flavor = 0;
+
+	if ( ! UNSERIALIZE(&ser_flavor) )
+		return false;
+
+	switch ( ser_flavor ) {
+	case 0:
+		flavor = FUNC_FLAVOR_FUNCTION;
+		break;
+	case 1:
+		flavor = FUNC_FLAVOR_EVENT;
+		break;
+	case 2:
+		flavor = FUNC_FLAVOR_HOOK;
+		break;
+	default:
+		reporter->InternalError("Invalid function flavor unserialization");
+		break;
+	}
+
+	return true;
 	}
 
 TypeDecl::TypeDecl(BroType* t, const char* i, attr_list* arg_attrs, bool in_record)
@@ -1202,9 +1262,10 @@ bool FileType::DoUnserialize(UnserialInfo* info)
 	return yield != 0;
 	}
 
-EnumType::EnumType()
+EnumType::EnumType(const string& arg_name)
 : BroType(TYPE_ENUM)
 	{
+	name = arg_name;
 	counter = 0;
 	}
 
@@ -1327,6 +1388,13 @@ const char* EnumType::Lookup(bro_int_t value)
 	return 0;
 	}
 
+void EnumType::DescribeReST(ODesc* d) const
+	{
+	d->Add(":bro:type:`");
+	d->Add(name.c_str());
+	d->Add("`");
+	}
+
 void CommentedEnumType::DescribeReST(ODesc* d) const
 	{
 	// create temporary, reverse name map so that enums can be documented
@@ -1595,7 +1663,7 @@ int same_type(const BroType* t1, const BroType* t2, int is_init)
 		const FuncType* ft1 = (const FuncType*) t1;
 		const FuncType* ft2 = (const FuncType*) t2;
 
-		if ( ft1->IsEvent() != ft2->IsEvent() )
+		if ( ft1->Flavor() != ft2->Flavor() )
 			return 0;
 
 		if ( t1->YieldType() || t2->YieldType() )
@@ -1882,7 +1950,7 @@ BroType* merge_types(const BroType* t1, const BroType* t2)
 		BroType* yield = t1->YieldType() ?
 			merge_types(t1->YieldType(), t2->YieldType()) : 0;
 
-		return new FuncType(args->AsRecordType(), yield, ft1->IsEvent());
+		return new FuncType(args->AsRecordType(), yield, ft1->Flavor());
 		}
 
 	case TYPE_RECORD:

src/Type.h

@@ -35,7 +35,11 @@ typedef enum {
 #define NUM_TYPES (int(TYPE_ERROR) + 1)
 } TypeTag;
 
-typedef enum { FUNC_FLAVOR_FUNCTION, FUNC_FLAVOR_EVENT } function_flavor;
+typedef enum {
+	FUNC_FLAVOR_FUNCTION,
+	FUNC_FLAVOR_EVENT,
+	FUNC_FLAVOR_HOOK
+} function_flavor;
 
 typedef enum {
 	TYPE_INTERNAL_VOID,
@@ -350,18 +354,19 @@ protected:
 
 class FuncType : public BroType {
 public:
-	FuncType(RecordType* args, BroType* yield, int is_event);
+	FuncType(RecordType* args, BroType* yield, function_flavor f);
 
 	~FuncType();
 
 	RecordType* Args() const	{ return args; }
 	BroType* YieldType();
 	void SetYieldType(BroType* arg_yield)	{ yield = arg_yield; }
-	int IsEvent() const		{ return is_event; }
+	function_flavor Flavor() const { return flavor; }
+	string FlavorString() const;
 
-	// Used to convert a function type to an event type.
+	// Used to convert a function type to an event or hook type.
-	void ClearYieldType()
+	void ClearYieldType(function_flavor arg_flav)
-		{ Unref(yield); yield = 0; is_event = 1; }
+		{ Unref(yield); yield = 0; flavor = arg_flav; }
 
 	int MatchesIndex(ListExpr*& index) const;
 	int CheckArgs(const type_list* args) const;
@@ -374,14 +379,13 @@ public:
 	void DescribeReST(ODesc* d) const;
 
 protected:
-	FuncType()	{ args = 0; arg_types = 0; yield = 0; return_value = 0; }
+	FuncType()	{ args = 0; arg_types = 0; yield = 0; flavor = FUNC_FLAVOR_FUNCTION; }
 	DECLARE_SERIAL(FuncType)
 
 	RecordType* args;
 	TypeList* arg_types;
 	BroType* yield;
-	int is_event;
+	function_flavor flavor;
-	ID* return_value;
 };
 
 class TypeType : public BroType {
@@ -497,7 +501,7 @@ protected:
 
 class EnumType : public BroType {
 public:
-	EnumType();
+	EnumType(const string& arg_name);
 	~EnumType();
 
 	// The value of this name is next internal counter value, starting
@@ -513,7 +517,12 @@ public:
 	bro_int_t Lookup(const string& module_name, const char* name);
 	const char* Lookup(bro_int_t value); // Returns 0 if not found
 
+	string Name() const { return name; }
+
+	void DescribeReST(ODesc* d) const;
+
 protected:
+	EnumType() { counter = 0; }
 	DECLARE_SERIAL(EnumType)
 
 	virtual void AddNameInternal(const string& module_name,
@@ -529,11 +538,14 @@ protected:
 	// as a flag to prevent mixing of auto-increment and explicit
 	// enumerator specifications.
 	bro_int_t counter;
+
+	// The name of the enum type is stored for documentation purposes.
+	string name;
 };
 
 class CommentedEnumType: public EnumType {
 public:
-	CommentedEnumType() {}
+	CommentedEnumType(const string& arg_name) : EnumType(arg_name) {}
 	~CommentedEnumType();
 
 	void DescribeReST(ODesc* d) const;

src/Val.cc

@@ -3117,6 +3117,9 @@ void VectorVal::ValDescribe(ODesc* d) const
 
 Val* check_and_promote(Val* v, const BroType* t, int is_init)
 	{
+	if ( ! v )
+		return 0;
+
 	BroType* vt = v->Type();
 
 	vt = flatten_type(vt);

src/Var.cc

@@ -109,6 +109,36 @@ static void make_var(ID* id, BroType* t, init_class c, Expr* init,
 	if ( attr )
 		id->AddAttrs(new Attributes(attr, t, false));
 
+	if ( init )
+		{
+		switch ( init->Tag() ) {
+		case EXPR_TABLE_CONSTRUCTOR:
+			{
+			TableConstructorExpr* ctor = (TableConstructorExpr*) init;
+			if ( ctor->Attrs() )
+				{
+				::Ref(ctor->Attrs());
+				id->AddAttrs(ctor->Attrs());
+				}
+			}
+			break;
+
+		case EXPR_SET_CONSTRUCTOR:
+			{
+			SetConstructorExpr* ctor = (SetConstructorExpr*) init;
+			if ( ctor->Attrs() )
+				{
+				::Ref(ctor->Attrs());
+				id->AddAttrs(ctor->Attrs());
+				}
+			}
+			break;
+
+		default:
+			break;
+		}
+		}
+
 	if ( id->FindAttr(ATTR_PERSISTENT) || id->FindAttr(ATTR_SYNCHRONIZED) )
 		{
 		if ( dt == VAR_CONST )
@@ -171,7 +201,9 @@ static void make_var(ID* id, BroType* t, init_class c, Expr* init,
 
 	id->UpdateValAttrs();
 
-	if ( t && t->Tag() == TYPE_FUNC && t->AsFuncType()->IsEvent() )
+	if ( t && t->Tag() == TYPE_FUNC &&
+	     (t->AsFuncType()->Flavor() == FUNC_FLAVOR_EVENT ||
+	      t->AsFuncType()->Flavor() == FUNC_FLAVOR_HOOK) )
 		{
 		// For events, add a function value (without any body) here so that
 		// we can later access the ID even if no implementations have been
@@ -258,7 +290,7 @@ void add_type(ID* id, BroType* t, attr_list* attr, int /* is_event */)
 		case TYPE_FUNC:
 			tnew = new FuncType(t->AsFuncType()->Args(),
 			                    t->AsFuncType()->YieldType(),
-			                    t->AsFuncType()->IsEvent());
+			                    t->AsFuncType()->Flavor());
 			break;
 		default:
 			SerializationFormat* form = new BinarySerializationFormat();
@@ -299,7 +331,7 @@ void begin_func(ID* id, const char* module_name, function_flavor flavor,
 		if ( yt && yt->Tag() != TYPE_VOID )
 			id->Error("event cannot yield a value", t);
 
-		t->ClearYieldType();
+		t->ClearYieldType(flavor);
 		}
 
 	if ( id->Type() )
@@ -313,21 +345,29 @@ void begin_func(ID* id, const char* module_name, function_flavor flavor,
 
 	if ( id->HasVal() )
 		{
-		int id_is_event = id->ID_Val()->AsFunc()->IsEvent();
+		function_flavor id_flavor = id->ID_Val()->AsFunc()->Flavor();
 
-		if ( id_is_event != (flavor == FUNC_FLAVOR_EVENT) )
+		if ( id_flavor != flavor )
-			id->Error("inconsistency between event and function", t);
+			id->Error("inconsistent function flavor", t);
-		if ( id_is_event )
+
-			{
+		switch ( id_flavor ) {
+
+		case FUNC_FLAVOR_EVENT:
+		case FUNC_FLAVOR_HOOK:
 			if ( is_redef )
 				// Clear out value so it will be replaced.
 				id->SetVal(0);
-			}
+			break;
-		else
+
-			{
+		case FUNC_FLAVOR_FUNCTION:
 			if ( ! id->IsRedefinable() )
 				id->Error("already defined");
-			}
+			break;
+
+		default:
+			reporter->InternalError("invalid function flavor");
+			break;
+		}
 		}
 	else
 		id->SetType(t);

src/bro.bif

@@ -2701,6 +2701,27 @@ function to_port%(s: string%): port
     return new PortVal(port, TRANSPORT_UNKNOWN);
 	%}
 
+## Converts a string of bytes (in network byte order) to a :bro:type:`double`.
+##
+## s: A string of bytes containing the binary representation of a double value.
+##
+## Returns: The double value contained in *s*, or 0 if the conversion
+##          failed.
+##
+function bytestring_to_double%(s: string%): double
+	%{
+	if ( s->Len() != sizeof(double) )
+		{
+		builtin_error("bad conversion to double");
+		return new Val(0.0, TYPE_DOUBLE);
+		}
+
+	// See #908 for a discussion of portability.
+	double d;
+	memcpy(&d, s->Bytes(), sizeof(double));
+	return new Val(ntohd(d), TYPE_DOUBLE);
+	%}
+
 ## Converts a reverse pointer name to an address. For example,
 ## ``1.0.168.192.in-addr.arpa`` to ``192.168.0.1``.
 ##

src/input/Manager.cc

@@ -393,7 +393,7 @@ bool Manager::CreateEventStream(RecordVal* fval)
 
 	bool allow_file_func = false;
 
-	if ( ! etype->IsEvent() )
+	if ( etype->Flavor() != FUNC_FLAVOR_EVENT )
 		{
 		reporter->Error("stream event is a function, not an event");
 		return false;
@@ -566,7 +566,7 @@ bool Manager::CreateTableStream(RecordVal* fval)
 		{
 		FuncType* etype = event->FType()->AsFuncType();
 
-		if ( ! etype->IsEvent() )
+		if ( etype->Flavor() != FUNC_FLAVOR_EVENT )
 			{
 			reporter->Error("stream event is a function, not an event");
 			return false;

src/logging/Manager.cc

@@ -327,7 +327,7 @@ bool Manager::CreateStream(EnumVal* id, RecordVal* sval)
 		// Make sure the event is prototyped as expected.
 		FuncType* etype = event->FType()->AsFuncType();
 
-		if ( ! etype->IsEvent() )
+		if ( etype->Flavor() != FUNC_FLAVOR_EVENT )
 			{
 			reporter->Error("stream event is a function, not an event");
 			return false;

src/logging/writers/Ascii.cc

@@ -19,6 +19,7 @@ Ascii::Ascii(WriterFrontend* frontend) : WriterBackend(frontend)
 	{
 	fd = 0;
 	ascii_done = false;
+	tsv = false;
 
 	output_to_stdout = BifConst::LogAscii::output_to_stdout;
 	include_meta = BifConst::LogAscii::include_meta;
@@ -80,7 +81,7 @@ void Ascii::CloseFile(double t)
 	if ( ! fd )
 		return;
 
-	if ( include_meta )
+	if ( include_meta && ! tsv )
 		WriteHeaderField("close", Timestamp(0));
 
 	safe_close(fd);
@@ -108,29 +109,29 @@ bool Ascii::DoInit(const WriterInfo& info, int num_fields, const Field* const *
 		return false;
 		}
 
+	for ( WriterInfo::config_map::const_iterator i = info.config.begin(); i != info.config.end(); i++ )
+		{
+		if ( strcmp(i->first, "tsv") == 0 )
+			{
+			if ( strcmp(i->second, "T") == 0 )
+				tsv = true;
+
+			else if ( strcmp(i->second, "F") == 0 )
+				tsv = false;
+
+			else
+				{
+				Error("invalid value for 'tsv', must be a string and either \"T\" or \"F\"");
+				return false;
+				}
+			}
+		}
+
 	if ( include_meta )
 		{
 		string names;
 		string types;
 
-		string str = string(meta_prefix, meta_prefix_len)
-			+ "separator " // Always use space as separator here.
-			+ get_escaped_string(string(separator, separator_len), false)
-			+ "\n";
-
-		if ( ! safe_write(fd, str.c_str(), str.length()) )
-			goto write_error;
-
-		if ( ! (WriteHeaderField("set_separator", get_escaped_string(
-		            string(set_separator, set_separator_len), false)) &&
-		        WriteHeaderField("empty_field", get_escaped_string(
-		            string(empty_field, empty_field_len), false)) &&
-		        WriteHeaderField("unset_field", get_escaped_string(
-		            string(unset_field, unset_field_len), false)) &&
-		        WriteHeaderField("path", get_escaped_string(path, false)) &&
-		        WriteHeaderField("open", Timestamp(0))) )
-			goto write_error;
-
 		for ( int i = 0; i < num_fields; ++i )
 			{
 			if ( i > 0 )
@@ -143,6 +144,34 @@ bool Ascii::DoInit(const WriterInfo& info, int num_fields, const Field* const *
 			types += fields[i]->TypeName().c_str();
 			}
 
+		if ( tsv )
+			{
+			// A single TSV-style line is all we need.
+			string str = names + "\n";
+			if ( ! safe_write(fd, str.c_str(), str.length()) )
+				goto write_error;
+
+			return true;
+			}
+
+		string str = string(meta_prefix, meta_prefix_len)
+			+ "separator " // Always use space as separator here.
+			+ get_escaped_string(string(separator, separator_len), false)
+			+ "\n";
+
+		if ( ! safe_write(fd, str.c_str(), str.length()) )
+			goto write_error;
+
+		if ( ! (WriteHeaderField("set_separator", get_escaped_string(
+			    string(set_separator, set_separator_len), false)) &&
+			WriteHeaderField("empty_field", get_escaped_string(
+			    string(empty_field, empty_field_len), false)) &&
+			WriteHeaderField("unset_field", get_escaped_string(
+			    string(unset_field, unset_field_len), false)) &&
+			WriteHeaderField("path", get_escaped_string(path, false)) &&
+			WriteHeaderField("open", Timestamp(0))) )
+			goto write_error;
+
 		if ( ! (WriteHeaderField("fields", names)
 			&& WriteHeaderField("types", types)) )
 			goto write_error;

src/logging/writers/Ascii.h

@@ -45,6 +45,7 @@ private:
 	// Options set from the script-level.
 	bool output_to_stdout;
 	bool include_meta;
+	bool tsv;
 
 	char* separator;
 	int separator_len;

src/parse.y

@@ -2,14 +2,14 @@
 // See the file "COPYING" in the main distribution directory for copyright.
 %}
 
-%expect 87
+%expect 88
 
 %token TOK_ADD TOK_ADD_TO TOK_ADDR TOK_ANY
 %token TOK_ATENDIF TOK_ATELSE TOK_ATIF TOK_ATIFDEF TOK_ATIFNDEF
 %token TOK_BOOL TOK_BREAK TOK_CASE TOK_CONST
 %token TOK_CONSTANT TOK_COPY TOK_COUNT TOK_COUNTER TOK_DEFAULT TOK_DELETE
 %token TOK_DOUBLE TOK_ELSE TOK_ENUM TOK_EVENT TOK_EXPORT TOK_FILE TOK_FOR
-%token TOK_FUNCTION TOK_GLOBAL TOK_ID TOK_IF TOK_INT
+%token TOK_FUNCTION TOK_GLOBAL TOK_HOOK TOK_ID TOK_IF TOK_INT
 %token TOK_INTERVAL TOK_LIST TOK_LOCAL TOK_MODULE
 %token TOK_NEXT TOK_OF TOK_PATTERN TOK_PATTERN_TEXT
 %token TOK_PORT TOK_PRINT TOK_RECORD TOK_REDEF
@@ -32,6 +32,7 @@
 
 %token TOK_NO_TEST
 
+%nonassoc TOK_HOOK
 %left ',' '|'
 %right '=' TOK_ADD_TO TOK_REMOVE_FROM
 %right '?' ':'
@@ -118,6 +119,7 @@ extern const char* g_curr_debug_error;
 
 #define YYLTYPE yyltype
 
+static int in_hook = 0;
 int in_init = 0;
 int in_record = 0;
 bool resolving_global_ID = false;
@@ -131,16 +133,18 @@ const char* cur_enum_elem_id = 0;
 type_decl_list* fake_type_decl_list = 0;
 TypeDecl* last_fake_type_decl = 0;
 
+static ID* cur_decl_type_id = 0;
+
 static void parser_new_enum (void)
 	{
 	/* Starting a new enum definition. */
 	assert(cur_enum_type == NULL);
-	cur_enum_type = new EnumType();
+	cur_enum_type = new EnumType(cur_decl_type_id->Name());
 
 	// For documentation purposes, a separate type object is created
 	// in order to avoid overlap that can be caused by redefs.
 	if ( generate_documentation )
-		cur_enum_type_doc = new CommentedEnumType();
+		cur_enum_type_doc = new CommentedEnumType(cur_decl_type_id->Name());
 	}
 
 static void parser_redef_enum (ID *id)
@@ -158,7 +162,7 @@ static void parser_redef_enum (ID *id)
 		}
 
 	if ( generate_documentation )
-		cur_enum_type_doc = new CommentedEnumType();
+		cur_enum_type_doc = new CommentedEnumType(id->Name());
 	}
 
 static void add_enum_comment (std::list<std::string>* comments)
@@ -209,7 +213,6 @@ static std::list<std::string>* concat_opt_docs (std::list<std::string>* pre,
 	Val* val;
 	RE_Matcher* re;
 	Expr* expr;
-	CallExpr* call_expr;
 	EventExpr* event_expr;
 	Stmt* stmt;
 	ListExpr* list;
@@ -456,17 +459,24 @@ expr:
 
 	|	'[' expr_list ']'
 			{
-			// A little crufty: we peek at the type of
-			// the first expression in the list.  If it's
-			// a record or a field assignment, then this
-			// is a record constructor.  If not, then this
-			// is a list used for an initializer.
-
 			set_location(@1, @3);
 
-			Expr* e0 = $2->Exprs()[0];
+			bool is_record_ctor = true;
-			if ( e0->Tag() == EXPR_FIELD_ASSIGN ||
+
-			     e0->Type()->Tag() == TYPE_RECORD )
+			// If every expression in the list is a field assignment,
+			// then treat it as a record constructor, else as a list
+			// used for an initializer.
+
+			for ( int i = 0; i < $2->Exprs().length(); ++i )
+				{
+				if ( $2->Exprs()[i]->Tag() != EXPR_FIELD_ASSIGN )
+					{
+					is_record_ctor = false;
+					break;
+					}
+				}
+
+			if ( is_record_ctor )
 				$$ = new RecordConstructorExpr($2);
 			else
 				$$ = $2;
@@ -507,7 +517,16 @@ expr:
 	|	expr '(' opt_expr_list ')'
 			{
 			set_location(@1, @4);
-			$$ = new CallExpr($1, $3);
+			$$ = new CallExpr($1, $3, in_hook > 0);
+			}
+
+	|	TOK_HOOK { ++in_hook; } expr
+			{
+			--in_hook;
+			set_location(@1, @3);
+			if ( $3->Tag() != EXPR_CALL )
+				$3->Error("not a valid hook call expression");
+			$$ = $3;
 			}
 
 	|	expr TOK_HAS_FIELD TOK_ID
@@ -859,7 +878,13 @@ type:
 	|	TOK_EVENT '(' formal_args ')'
 				{
 				set_location(@1, @3);
-				$$ = new FuncType($3, 0, 1);
+				$$ = new FuncType($3, 0, FUNC_FLAVOR_EVENT);
+				}
+
+	|	TOK_HOOK '(' formal_args ')'
+				{
+				set_location(@1, @3);
+				$$ = new FuncType($3, base_type(TYPE_BOOL), FUNC_FLAVOR_HOOK);
 				}
 
 	|	TOK_FILE TOK_OF type
@@ -991,12 +1016,27 @@ decl:
 				ID* id = $2;
 				if ( id->Type()->Tag() == TYPE_FUNC )
 					{
-					if ( id->Type()->AsFuncType()->IsEvent() )
+					switch ( id->Type()->AsFuncType()->Flavor() ) {
-						current_reST_doc->AddEvent(
+
-							new BroDocObj(id, reST_doc_comments));
+					case FUNC_FLAVOR_FUNCTION:
-					else
 						current_reST_doc->AddFunction(
 							new BroDocObj(id, reST_doc_comments));
+						break;
+
+					case FUNC_FLAVOR_EVENT:
+						current_reST_doc->AddEvent(
+							new BroDocObj(id, reST_doc_comments));
+						break;
+
+					case FUNC_FLAVOR_HOOK:
+						current_reST_doc->AddHook(
+							new BroDocObj(id, reST_doc_comments));
+						break;
+
+					default:
+						reporter->InternalError("invalid function flavor");
+						break;
+					}
 					}
 
 				else
@@ -1098,9 +1138,10 @@ decl:
 				}
 			}
 
-	|	TOK_TYPE global_id ':' type opt_attr ';'
+	|	TOK_TYPE global_id ':' { cur_decl_type_id = $2; } type opt_attr ';'
 			{
-			add_type($2, $4, $5, 0);
+			cur_decl_type_id = 0;
+			add_type($2, $5, $6, 0);
 
 			if ( generate_documentation )
 				{
@@ -1175,6 +1216,17 @@ func_hdr:
 				current_reST_doc->AddEventHandler(
 					new BroDocObj($2, reST_doc_comments));
 			}
+	|	TOK_HOOK def_global_id func_params
+			{
+			$3->ClearYieldType(FUNC_FLAVOR_HOOK);
+			$3->SetYieldType(base_type(TYPE_BOOL));
+			begin_func($2, current_module.c_str(),
+				   FUNC_FLAVOR_HOOK, 0, $3);
+			$$ = $3;
+			if ( generate_documentation )
+				current_reST_doc->AddHookHandler(
+					new BroDocObj($2, reST_doc_comments));
+			}
 	|	TOK_REDEF TOK_EVENT event_id func_params
 			{
 			begin_func($3, current_module.c_str(),
@@ -1209,9 +1261,9 @@ begin_func:
 
 func_params:
 		'(' formal_args ')' ':' type
-			{ $$ = new FuncType($2, $5, 0); }
+			{ $$ = new FuncType($2, $5, FUNC_FLAVOR_FUNCTION); }
 	|	'(' formal_args ')'
-			{ $$ = new FuncType($2, base_type(TYPE_VOID), 0); }
+			{ $$ = new FuncType($2, base_type(TYPE_VOID), FUNC_FLAVOR_FUNCTION); }
 	;
 
 opt_type:

src/patricia.c

@@ -66,6 +66,9 @@ static char copyright[] =
 
 #define Delete free
 
+// From Bro for reporting memory exhaustion.
+extern void out_of_memory(const char* where);
+
 /* { from prefix.c */
 
 /* prefix_tochar
@@ -251,6 +254,9 @@ New_Prefix2 (int family, void *dest, int bitlen, prefix_t *prefix)
         default_bitlen = 128;
 	if (prefix == NULL) {
             prefix = calloc(1, sizeof (prefix_t));
+            if (prefix == NULL)
+                out_of_memory("patrica/new_prefix2: unable to allocate memory");
+
 	    dynamic_allocated++;
 	}
 	memcpy (&prefix->add.sin6, dest, 16);
@@ -260,10 +266,14 @@ New_Prefix2 (int family, void *dest, int bitlen, prefix_t *prefix)
 		if (prefix == NULL) {
 #ifndef NT
             prefix = calloc(1, sizeof (prefix4_t));
+            if (prefix == NULL)
+                out_of_memory("patrica/new_prefix2: unable to allocate memory");
 #else
 			//for some reason, compiler is getting
 			//prefix4_t size incorrect on NT
 			prefix = calloc(1, sizeof (prefix_t));
+                        if (prefix == NULL)
+                            out_of_memory("patrica/new_prefix2: unable to allocate memory");
 #endif /* NT */
 
 			dynamic_allocated++;
@@ -396,6 +406,8 @@ patricia_tree_t *
 New_Patricia (int maxbits)
 {
     patricia_tree_t *patricia = calloc(1, sizeof *patricia);
+    if (patricia == NULL)
+        out_of_memory("patrica/new_patricia: unable to allocate memory");
 
     patricia->maxbits = maxbits;
     patricia->head = NULL;
@@ -665,6 +677,9 @@ patricia_lookup (patricia_tree_t *patricia, prefix_t *prefix)
 
     if (patricia->head == NULL) {
 	node = calloc(1, sizeof *node);
+        if (node == NULL)
+            out_of_memory("patrica/patrica_lookup: unable to allocate memory");
+
 	node->bit = prefix->bitlen;
 	node->prefix = Ref_Prefix (prefix);
 	node->parent = NULL;
@@ -776,6 +791,9 @@ patricia_lookup (patricia_tree_t *patricia, prefix_t *prefix)
     }
 
     new_node = calloc(1, sizeof *new_node);
+    if (new_node == NULL)
+        out_of_memory("patrica/patrica_lookup: unable to allocate memory");
+
     new_node->bit = prefix->bitlen;
     new_node->prefix = Ref_Prefix (prefix);
     new_node->parent = NULL;
@@ -828,6 +846,9 @@ patricia_lookup (patricia_tree_t *patricia, prefix_t *prefix)
     }
     else {
         glue = calloc(1, sizeof *glue);
+        if (glue == NULL)
+            out_of_memory("patrica/patrica_lookup: unable to allocate memory");
+
         glue->bit = differ_bit;
         glue->prefix = NULL;
         glue->parent = node->parent;

src/scan.l

@@ -287,6 +287,7 @@ for	return TOK_FOR;
 function	return TOK_FUNCTION;
 global	return TOK_GLOBAL;
 "?$"	return TOK_HAS_FIELD;
+hook	return TOK_HOOK;
 if	return TOK_IF;
 in	return TOK_IN;
 "!"{OWS}in/[^A-Za-z0-9]	return TOK_NOT_IN;	/* don't confuse w "! infoo"! */

src/util.cc

@@ -1416,7 +1416,7 @@ void safe_close(int fd)
 		}
 	}
 
-void out_of_memory(const char* where)
+extern "C" void out_of_memory(const char* where)
 	{
 	fprintf(stderr, "out of memory in %s.\n", where);
 

src/util.h

@@ -303,7 +303,7 @@ extern bool safe_write(int fd, const char* data, int len);
 // Wraps close(2) to emit error messages and abort on unrecoverable errors.
 extern void safe_close(int fd);
 
-extern void out_of_memory(const char* where);
+extern "C" void out_of_memory(const char* where);
 
 inline void* safe_realloc(void* ptr, size_t size)
 	{

testing/btest/Baseline/bifs.bytestring_to_double/out

@@ -0,0 +1,9 @@
+3.140000e+15
+-3.140000e+15
+4.000000e-308
+0.000000e+00
+-0.000000e+00
+inf
+-inf
+nan
+4.94e-324

testing/btest/Baseline/bifs.strptime/.stdout

@@ -1,2 +0,0 @@
-1350604800.0
-0.0

testing/btest/Baseline/bifs.strptime/out

@@ -0,0 +1,3 @@
+warning: strptime conversion failed: fmt:%m d:1980-10-24
+1350604800.0
+0.0

testing/btest/Baseline/bifs.strptime/reporter.log

@@ -1,10 +0,0 @@
-#separator \x09
-#set_separator	,
-#empty_field	(empty)
-#unset_field	-
-#path	reporter
-#open	2012-10-19-06-06-36
-#fields	ts	level	message	location
-#types	time	enum	string	string
-0.000000	Reporter::WARNING	strptime conversion failed: fmt:%m d:1980-10-24	(empty)
-#close	2012-10-19-06-06-36

testing/btest/Baseline/doc.autogen-reST-enums/autogen-reST-enums.rst

@@ -12,6 +12,12 @@ autogen-reST-enums.bro
 
 Summary
 ~~~~~~~
+Options
+#######
+==================================================================== ======================================================================
+:bro:id:`test_enum_option`: :bro:type:`TestEnum1` :bro:attr:`&redef` this should reference the TestEnum1 type and not a generic "enum" type
+==================================================================== ======================================================================
+
 Types
 #####
 ======================================= ======================================
@@ -30,6 +36,16 @@ Redefinitions
 
 Detailed Interface
 ~~~~~~~~~~~~~~~~~~
+Options
+#######
+.. bro:id:: test_enum_option
+
+   :Type: :bro:type:`TestEnum1`
+   :Attributes: :bro:attr:`&redef`
+   :Default: ``ONE``
+
+   this should reference the TestEnum1 type and not a generic "enum" type
+
 Types
 #####
 .. bro:type:: TestEnum1

testing/btest/Baseline/doc.autogen-reST-example/example.rst

@@ -73,9 +73,9 @@ Events
 
 Functions
 #########
-=============================================== =======================================
+=================================================== =======================================
-:bro:id:`Example::a_function`: :bro:type:`func` Summarize purpose of "a_function" here.
+:bro:id:`Example::a_function`: :bro:type:`function` Summarize purpose of "a_function" here.
-=============================================== =======================================
+=================================================== =======================================
 
 Redefinitions
 #############

testing/btest/Baseline/doc.autogen-reST-func-params/autogen-reST-func-params.rst

@@ -20,9 +20,9 @@ Types
 
 Functions
 #########
-===================================== ======================================
+========================================= ======================================
-:bro:id:`test_func`: :bro:type:`func` This is a global function declaration.
+:bro:id:`test_func`: :bro:type:`function` This is a global function declaration.
-===================================== ======================================
+========================================= ======================================
 
 Detailed Interface
 ~~~~~~~~~~~~~~~~~~

testing/btest/Baseline/language.hook/out

@@ -0,0 +1,18 @@
+myhook, &priority=10, [a=1156, b=hello world]
+myhook return F
+myhook return T
+myhook, &priority=5, [a=37, b=goobye world]
+F
+myhook3, 8
+T
+myhook4, 2
+myhook4, 1
+T
+myhook4, 2
+myhook4, 1
+myhook4 all handlers ran
+myhook, &priority=10, [a=2, b=it works]
+myhook return F
+myhook return T
+myhook, &priority=5, [a=37, b=goobye world]
+F

testing/btest/Baseline/language.hook_calls/invalid.out

@@ -0,0 +1,10 @@
+error in ./invalid.bro, line 9: hook cannot be called directly, use hook operator (myhook)
+warning in ./invalid.bro, line 9: expression value ignored (myhook(3))
+error in ./invalid.bro, line 10: hook cannot be called directly, use hook operator (myhook)
+error in ./invalid.bro, line 11: hook cannot be called directly, use hook operator (myhook)
+error in ./invalid.bro, line 12: not a valid hook call expression (2 + 2)
+warning in ./invalid.bro, line 12: expression value ignored (2 + 2)
+error in ./invalid.bro, line 13: not a valid hook call expression (2 + 2)
+error in ./invalid.bro, line 15: hook cannot be called directly, use hook operator (h)
+warning in ./invalid.bro, line 15: expression value ignored (h(3))
+error in ./invalid.bro, line 16: hook cannot be called directly, use hook operator (h)

testing/btest/Baseline/language.hook_calls/valid.out

@@ -0,0 +1,42 @@
+myhook(), 3
+other myhook(), 3
+myhook(), 3
+other myhook(), 3
+T
+myhook(), 0
+F
+-----------
+indirect()
+myhook(), 3
+other myhook(), 3
+indirect()
+myhook(), 3
+other myhook(), 3
+T
+-----------
+really_indirect()
+indirect()
+myhook(), 3
+other myhook(), 3
+really_indirect()
+indirect()
+myhook(), 3
+other myhook(), 3
+T
+-----------
+myhook(), 3
+other myhook(), 3
+myhook(), 3
+other myhook(), 3
+T
+myhook(), 3
+other myhook(), 3
+yes
+myhook(), 0
+double yes
+-----------
+myhook(), 3
+other myhook(), 3
+myhook(), 3
+other myhook(), 3
+T

testing/btest/Baseline/language.table-init-attrs/output

@@ -0,0 +1,91 @@
+my_set_ctor_init
+{
+test2,
+test3,
+test4,
+test1
+}
+
+my_table_ctor_init
+{
+[2] = test2,
+[1] = test1,
+[3] = test3
+}
+nope
+
+my_set_init
+{
+test2,
+test3,
+test4,
+test1
+}
+
+my_table_init
+{
+[2] = test2,
+[4] = test4,
+[1] = test1,
+[3] = test3
+}
+nope
+
+inception
+{
+[0] = {
+[13] = bar
+}
+}
+{
+[13] = bar
+}
+bar
+forty-two
+{
+
+}
+we need to go deeper
+{
+[0] = {
+[13] = bar
+}
+}
+{
+[13] = bar
+}
+bar
+forty-two
+{
+
+}
+we need to go deeper
+
+local table t1
+{
+[1] = foo
+}
+foo
+nope
+
+local table t2
+{
+[1] = foo
+}
+foo
+nope
+
+local table t3
+{
+
+}
+nope
+nope
+
+local table t4
+{
+
+}
+nope
+nope
+

testing/btest/Baseline/language.table-init-container-ctors/output

@@ -0,0 +1,50 @@
+table of set
+{
+[13] = {
+[bar, 2] ,
+[foo, 1] 
+},
+[5] = {
+[bah, 3] ,
+[baz, 4] 
+}
+}
+
+table of vector
+{
+[13] = [1, 2],
+[5] = [3, 4]
+}
+
+table of table
+{
+[13] = {
+[bar, 2] = 2,
+[foo, 1] = 1
+},
+[5] = {
+[bah, 3] = 3,
+[baz, 4] = 4
+}
+}
+
+table of record
+{
+[13] = [a=1, b=foo],
+[5] = [a=2, b=bar]
+}
+
+T
+T
+T
+T
+T
+T
+T
+T
+T
+T
+T
+T
+T
+T

testing/btest/Baseline/language.table/out

@@ -12,6 +12,7 @@ cardinality (PASS)
 cardinality (PASS)
 cardinality (PASS)
 cardinality (PASS)
+cardinality (PASS)
 iterate over table (PASS)
 iterate over table (PASS)
 iterate over table (PASS)
@@ -30,6 +31,11 @@ add element (PASS)
 in operator (PASS)
 add element (PASS)
 in operator (PASS)
+composite index add element (PASS)
+composite index in operator (PASS)
+composite index in operator (PASS)
+remove element (PASS)
+!in operator (PASS)
 remove element (PASS)
 !in operator (PASS)
 remove element (PASS)

testing/btest/Baseline/scripts.base.frameworks.logging.ascii-tsv/ssh-filtered.log

@@ -0,0 +1,6 @@
+t	id.orig_h	id.orig_p	id.resp_h	id.resp_p	status	country	b
+1353727995.082217	1.2.3.4	1234	2.3.4.5	80	success	unknown	-
+1353727995.082217	1.2.3.4	1234	2.3.4.5	80	-	US	-
+1353727995.082217	1.2.3.4	1234	2.3.4.5	80	failure	UK	-
+1353727995.082217	1.2.3.4	1234	2.3.4.5	80	-	BR	-
+1353727995.082217	1.2.3.4	1234	2.3.4.5	80	failure	(empty)	T

testing/btest/bifs/addr_count_conversion.bro

@@ -1,4 +1,4 @@
-# @TEST-EXEC: bro %INPUT >output
+# @TEST-EXEC: bro -b %INPUT >output
 # @TEST-EXEC: btest-diff output
 
 global v: index_vec;

testing/btest/bifs/addr_to_ptr_name.bro

@@ -1,4 +1,4 @@
-# @TEST-EXEC: bro %INPUT >output
+# @TEST-EXEC: bro -b %INPUT >output
 # @TEST-EXEC: btest-diff output
 
 print addr_to_ptr_name([2607:f8b0:4009:802::1012]);

testing/btest/bifs/addr_version.bro

@@ -1,4 +1,4 @@
-# @TEST-EXEC: bro %INPUT >out
+# @TEST-EXEC: bro -b %INPUT >out
 # @TEST-EXEC: btest-diff out
 
 print is_v4_addr(1.2.3.4);

testing/btest/bifs/all_set.bro

@@ -1,5 +1,5 @@
 #
-# @TEST-EXEC: bro %INPUT >out
+# @TEST-EXEC: bro -b %INPUT >out
 # @TEST-EXEC: btest-diff out
 
 event bro_init()

testing/btest/bifs/analyzer_name.bro

@@ -1,5 +1,5 @@
 #
-# @TEST-EXEC: bro %INPUT >out
+# @TEST-EXEC: bro -b %INPUT >out
 # @TEST-EXEC: btest-diff out
 
 event bro_init()

testing/btest/bifs/any_set.bro

@@ -1,5 +1,5 @@
 #
-# @TEST-EXEC: bro %INPUT >out
+# @TEST-EXEC: bro -b %INPUT >out
 # @TEST-EXEC: btest-diff out
 
 event bro_init()

testing/btest/bifs/bro_version.bro

@@ -1,5 +1,5 @@
 #
-# @TEST-EXEC: bro %INPUT
+# @TEST-EXEC: bro -b %INPUT
 
 event bro_init()
 	{

testing/btest/bifs/byte_len.bro

@@ -1,5 +1,5 @@
 #
-# @TEST-EXEC: bro %INPUT >out
+# @TEST-EXEC: bro -b %INPUT >out
 # @TEST-EXEC: btest-diff out
 
 event bro_init()

testing/btest/bifs/bytestring_to_double.bro

@@ -0,0 +1,26 @@
+#
+# @TEST-EXEC: bro -b %INPUT >out
+# @TEST-EXEC: btest-diff out
+
+event bro_init()
+	{
+	local s1 = "\x43\x26\x4f\xa0\x71\x30\x80\x00"; # 3.14e15
+	local s2 = "\xc3\x26\x4f\xa0\x71\x30\x80\x00"; #-3.14e15
+	local s3 = "\x00\x1c\xc3\x59\xe0\x67\xa3\x49"; # 4e-308
+	local s4 = "\x00\x00\x00\x00\x00\x00\x00\x00"; # 0.0
+	local s5 = "\x80\x00\x00\x00\x00\x00\x00\x00"; #-0.0
+	local s6 = "\x7f\xf0\x00\x00\x00\x00\x00\x00"; # Inf
+	local s7 = "\xff\xf0\x00\x00\x00\x00\x00\x00"; #-Inf
+	local s8 = "\x7f\xf8\x00\x00\x00\x00\x00\x00"; # NaN
+	local s9 = "\x00\x00\x00\x00\x00\x00\x00\x01"; # subnormal
+
+	print bytestring_to_double(s1);
+	print bytestring_to_double(s2);
+	print fmt("%e", bytestring_to_double(s3));
+	print fmt("%e", bytestring_to_double(s4));
+	print fmt("%e", bytestring_to_double(s5));
+	print bytestring_to_double(s6);
+	print bytestring_to_double(s7);
+	print bytestring_to_double(s8);
+	print fmt("%.2e", bytestring_to_double(s9));
+	}

testing/btest/bifs/bytestring_to_hexstr.bro

@@ -1,5 +1,5 @@
 #
-# @TEST-EXEC: bro %INPUT >out
+# @TEST-EXEC: bro -b %INPUT >out
 # @TEST-EXEC: btest-diff out
 
 event bro_init()

testing/btest/bifs/capture_state_updates.bro

@@ -1,5 +1,5 @@
 #
-# @TEST-EXEC: bro %INPUT >out
+# @TEST-EXEC: bro -b %INPUT >out
 # @TEST-EXEC: btest-diff out
 # @TEST-EXEC: test -f testfile
 

testing/btest/bifs/cat.bro

@@ -1,5 +1,5 @@
 #
-# @TEST-EXEC: bro %INPUT >out
+# @TEST-EXEC: bro -b %INPUT >out
 # @TEST-EXEC: btest-diff out
 
 event bro_init()

testing/btest/bifs/cat_string_array.bro

@@ -1,5 +1,5 @@
 #
-# @TEST-EXEC: bro %INPUT >out
+# @TEST-EXEC: bro -b %INPUT >out
 # @TEST-EXEC: btest-diff out
 
 event bro_init()

testing/btest/bifs/checkpoint_state.bro

@@ -1,5 +1,5 @@
 #
-# @TEST-EXEC: bro %INPUT
+# @TEST-EXEC: bro -b %INPUT
 # @TEST-EXEC: test -f .state/state.bst
 
 event bro_init()

testing/btest/bifs/clear_table.bro

@@ -1,5 +1,5 @@
 #
-# @TEST-EXEC: bro %INPUT > out
+# @TEST-EXEC: bro -b %INPUT > out
 # @TEST-EXEC: btest-diff out
 
 event bro_init()

testing/btest/bifs/convert_for_pattern.bro

@@ -1,5 +1,5 @@
 #
-# @TEST-EXEC: bro %INPUT >out
+# @TEST-EXEC: bro -b %INPUT >out
 # @TEST-EXEC: btest-diff out
 
 event bro_init()

testing/btest/bifs/count_to_addr.bro

@@ -1,5 +1,5 @@
 #
-# @TEST-EXEC: bro %INPUT >out
+# @TEST-EXEC: bro -b %INPUT >out
 # @TEST-EXEC: btest-diff out
 
 event bro_init()

testing/btest/bifs/create_file.bro

@@ -1,5 +1,5 @@
 #
-# @TEST-EXEC: bro %INPUT >out
+# @TEST-EXEC: bro -b %INPUT >out
 # @TEST-EXEC: btest-diff out
 # @TEST-EXEC: btest-diff testfile
 # @TEST-EXEC: btest-diff testfile2

testing/btest/bifs/current_analyzer.bro

@@ -1,5 +1,5 @@
 #
-# @TEST-EXEC: bro %INPUT
+# @TEST-EXEC: bro -b %INPUT
 
 event bro_init()
 	{

testing/btest/bifs/current_time.bro

@@ -1,5 +1,5 @@
 #
-# @TEST-EXEC: bro %INPUT
+# @TEST-EXEC: bro -b %INPUT
 
 event bro_init()
 	{

testing/btest/bifs/edit.bro

@@ -1,5 +1,5 @@
 #
-# @TEST-EXEC: bro %INPUT >out
+# @TEST-EXEC: bro -b %INPUT >out
 # @TEST-EXEC: btest-diff out
 
 event bro_init()

testing/btest/bifs/enable_raw_output.test

@@ -1,7 +1,7 @@
 # Files which enable raw output via the BiF shouldn't interpret NUL characters
 # in strings that are `print`ed to it.
 
-# @TEST-EXEC: bro %INPUT
+# @TEST-EXEC: bro -b %INPUT
 # @TEST-EXEC: tr '\000' 'X' <myfile >output
 # @TEST-EXEC: btest-diff output
 # @TEST-EXEC: cmp myfile hookfile

testing/btest/bifs/entropy_test.bro

@@ -1,5 +1,5 @@
 #
-# @TEST-EXEC: bro %INPUT >out
+# @TEST-EXEC: bro -b %INPUT >out
 # @TEST-EXEC: btest-diff out
 
 event bro_init()

testing/btest/bifs/escape_string.bro

@@ -1,5 +1,5 @@
 #
-# @TEST-EXEC: bro %INPUT >out
+# @TEST-EXEC: bro -b %INPUT >out
 # @TEST-EXEC: btest-diff out
 
 event bro_init()

testing/btest/bifs/exit.bro

@@ -1,5 +1,5 @@
 #
-# @TEST-EXEC: bro %INPUT >out || test $? -eq 7
+# @TEST-EXEC: bro -b %INPUT >out || test $? -eq 7
 # @TEST-EXEC: btest-diff out
 
 event bro_init()

testing/btest/bifs/file_mode.bro

@@ -1,5 +1,5 @@
 #
-# @TEST-EXEC: bro %INPUT >out
+# @TEST-EXEC: bro -b %INPUT >out
 # @TEST-EXEC: btest-diff out
 
 event bro_init()

testing/btest/bifs/find_all.bro

@@ -1,5 +1,5 @@
 #
-# @TEST-EXEC: bro %INPUT >out
+# @TEST-EXEC: bro -b %INPUT >out
 # @TEST-EXEC: btest-diff out
 
 event bro_init()

testing/btest/bifs/find_entropy.bro

@@ -1,5 +1,5 @@
 #
-# @TEST-EXEC: bro %INPUT >out
+# @TEST-EXEC: bro -b %INPUT >out
 # @TEST-EXEC: btest-diff out
 
 event bro_init()

testing/btest/bifs/find_last.bro

@@ -1,5 +1,5 @@
 #
-# @TEST-EXEC: bro %INPUT >out
+# @TEST-EXEC: bro -b %INPUT >out
 # @TEST-EXEC: btest-diff out
 
 event bro_init()

testing/btest/bifs/fmt.bro

@@ -1,5 +1,5 @@
 #
-# @TEST-EXEC: bro %INPUT >out
+# @TEST-EXEC: bro -b %INPUT >out
 # @TEST-EXEC: btest-diff out
 
 type color: enum { Red, Blue };

testing/btest/bifs/fmt_ftp_port.bro

@@ -1,5 +1,5 @@
 #
-# @TEST-EXEC: bro %INPUT >out
+# @TEST-EXEC: bro -b %INPUT >out
 # @TEST-EXEC: btest-diff out
 
 event bro_init()

testing/btest/bifs/get_matcher_stats.bro

@@ -1,5 +1,14 @@
 #
-# @TEST-EXEC: bro %INPUT
+# @TEST-EXEC: bro -b -s mysig %INPUT
+
+@TEST-START-FILE mysig.sig
+signature my_ftp_client {
+  ip-proto == tcp
+  payload /(|.*[\n\r]) *[uU][sS][eE][rR] /
+  tcp-state originator
+  event "matched my_ftp_client"
+}
+@TEST-END-FILE
 
 event bro_init()
 	{

testing/btest/bifs/get_port_transport_proto.bro

@@ -1,5 +1,5 @@
 #
-# @TEST-EXEC: bro %INPUT >out
+# @TEST-EXEC: bro -b %INPUT >out
 # @TEST-EXEC: btest-diff out
 
 event bro_init()

testing/btest/bifs/gethostname.bro

@@ -1,5 +1,5 @@
 #
-# @TEST-EXEC: bro %INPUT
+# @TEST-EXEC: bro -b %INPUT
 
 event bro_init()
 	{

testing/btest/bifs/getpid.bro

@@ -1,5 +1,5 @@
 #
-# @TEST-EXEC: bro %INPUT
+# @TEST-EXEC: bro -b %INPUT
 
 event bro_init()
 	{

testing/btest/bifs/getsetenv.bro

@@ -1,5 +1,5 @@
 #
-# @TEST-EXEC: TESTBRO=testvalue bro %INPUT >out
+# @TEST-EXEC: TESTBRO=testvalue bro -b %INPUT >out
 # @TEST-EXEC: btest-diff out
 
 event bro_init()

testing/btest/bifs/global_ids.bro

@@ -1,5 +1,5 @@
 #
-# @TEST-EXEC: bro %INPUT >out
+# @TEST-EXEC: bro -b %INPUT >out
 # @TEST-EXEC: btest-diff out
 
 event bro_init()

testing/btest/bifs/global_sizes.bro

@@ -1,5 +1,5 @@
 #
-# @TEST-EXEC: bro %INPUT >out
+# @TEST-EXEC: bro -b %INPUT >out
 # @TEST-EXEC: btest-diff out
 
 event bro_init()

testing/btest/bifs/hexdump.bro

@@ -1,5 +1,5 @@
 #
-# @TEST-EXEC: bro %INPUT >out
+# @TEST-EXEC: bro -b %INPUT >out
 # @TEST-EXEC: btest-diff out
 
 event bro_init()

testing/btest/bifs/identify_data.bro

@@ -1,5 +1,5 @@
 #
-# @TEST-EXEC: bro %INPUT | sed 's/PNG image data/PNG image/g' >out
+# @TEST-EXEC: bro -b %INPUT | sed 's/PNG image data/PNG image/g' >out
 # @TEST-EXEC: btest-diff out
 
 event bro_init()

testing/btest/bifs/is_ascii.bro

@@ -1,5 +1,5 @@
 #
-# @TEST-EXEC: bro %INPUT >out
+# @TEST-EXEC: bro -b %INPUT >out
 # @TEST-EXEC: btest-diff out
 
 event bro_init()

testing/btest/bifs/is_local_interface.bro

@@ -1,5 +1,5 @@
 #
-# @TEST-EXEC: bro %INPUT >out
+# @TEST-EXEC: bro -b %INPUT >out
 # @TEST-EXEC: btest-diff out
 
 event bro_init()

testing/btest/bifs/is_port.bro

@@ -1,5 +1,5 @@
 #
-# @TEST-EXEC: bro %INPUT >out
+# @TEST-EXEC: bro -b %INPUT >out
 # @TEST-EXEC: btest-diff out
 
 event bro_init()

testing/btest/bifs/join_string.bro

@@ -1,5 +1,5 @@
 #
-# @TEST-EXEC: bro %INPUT >out
+# @TEST-EXEC: bro -b %INPUT >out
 # @TEST-EXEC: btest-diff out
 
 event bro_init()

testing/btest/bifs/length.bro

@@ -1,5 +1,5 @@
 #
-# @TEST-EXEC: bro %INPUT > out
+# @TEST-EXEC: bro -b %INPUT > out
 # @TEST-EXEC: btest-diff out
 
 event bro_init()

testing/btest/bifs/lookup_ID.bro

@@ -1,5 +1,5 @@
 #
-# @TEST-EXEC: bro %INPUT >out
+# @TEST-EXEC: bro -b %INPUT >out
 # @TEST-EXEC: btest-diff out
 
 global a = "bro test";

testing/btest/bifs/lowerupper.bro

@@ -1,5 +1,5 @@
 #
-# @TEST-EXEC: bro %INPUT >out
+# @TEST-EXEC: bro -b %INPUT >out
 # @TEST-EXEC: btest-diff out
 
 event bro_init()

testing/btest/bifs/mask_addr.bro

@@ -1,4 +1,4 @@
-# @TEST-EXEC: bro %INPUT >output
+# @TEST-EXEC: bro -b %INPUT >output
 # @TEST-EXEC: btest-diff output
 
 const one_to_32: vector of count = {1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32};

testing/btest/bifs/math.bro

@@ -1,5 +1,5 @@
 #
-# @TEST-EXEC: bro %INPUT >out
+# @TEST-EXEC: bro -b %INPUT >out
 # @TEST-EXEC: btest-diff out
 
 event bro_init()

testing/btest/bifs/merge_pattern.bro

@@ -1,5 +1,5 @@
 #
-# @TEST-EXEC: bro %INPUT >out
+# @TEST-EXEC: bro -b %INPUT >out
 # @TEST-EXEC: btest-diff out
 
 event bro_init()