Merge remote-tracking branch 'origin/master' into topic/seth/metrics-merge

CHANGES

@@ -1,4 +1,69 @@
 
+2.1-195 | 2012-12-03 14:50:33 -0800
+
+  * Catching out-of-memory in patricia tree code. (Bill Parker)
+    
+2.1-194 | 2012-12-03 14:36:26 -0800
+
+  * Renaming ASCII writer filter option 'only_single_header_row' to
+    'tsv'. Also clarifying usage. Closes #912. (Robin Sommer)
+
+2.1-193 | 2012-12-03 14:11:14 -0800
+
+  * Fix a set of bugs with table/set attributes. (Jon Siwek)
+
+    - Identifiers that are initialized with set()/table() constructor
+      expressions now inherit attributes from the expression.  Before,
+      statements like
+
+         const i: set[string] = set() &redef;
+
+      associated the attribute with the set() constructor, but not the
+      "i" identifier, preventing redefinition.  Addresses #866.
+
+    - Allow &default attribute to apply to tables initialized as empty
+      (via either "{ }" or "table()") or if the expression supplied to it
+      can evaluate to a type that's promotable to the same yield type as
+      the table.
+
+2.1-191 | 2012-12-03 14:08:56 -0800
+
+  * Add test of record() constructor to table initializer unit test.
+    (Jon Siwek)
+
+  * Fix table(), set(), vector() constructors in table initializer
+    lists. Also adds type checking of yield values to table()
+    constructor and fixes the type checking of yield values in
+    vector() constructor. Addresses #5. (Jon Siwek)
+
+2.1-188 | 2012-12-03 14:04:29 -0800
+
+  * Hook functions now callable with "hook" expression (i.e., hook is
+    no longer a statement). The return value of the call is an
+    implicit boolean value of T if all hook handlers ran, or F if one
+    hook handler exited as a result of a break statement and
+    potentially prevented other handlers from running.
+
+    Scripts don't need to declare hooks with an explicit return type of bool
+    (internally, that's assumed), and any values given to (optional) return
+    statements in handler definitions are just ignored.
+
+    Addresses #918. (Jon Siwek)
+
+  * Clarification in hook documentation. (Jon Siwek)
+
+2.1-184 | 2012-12-03 13:59:50 -0800
+
+  * Slightly fix up file name extraction from Content-Disposition
+    headers. (Seth Hall)
+
+  * Adding -b flag to bro in unit tests so they run faster.
+
+  * Fixed a DNS attribute issue. Reported by Matt Thompson. (Seth
+    Hall)
+
+  * Adding NEWS placeholder for hooks and CSV mode. (Robin Sommer)
+    
 2.1-178 | 2012-11-23 19:35:32 -0800
 
   * The ASCII writer now supports a new filter config option

VERSION

@@ -1 +1 @@
-2.1-179
+2.1-195

doc/scripts/builtins.rst

@@ -505,15 +505,14 @@ The Bro scripting language supports the following built-in types.
     A hook is another flavor of function that shares characteristics of
     both a :bro:type:`function` and a :bro:type:`event`.  They are like
     events in that many handler bodies can be defined for the same hook
-    identifier, they have no return vale, and the order of execution
+    identifier and the order of execution can be enforced with
-    can be enforced with :bro:attr:`&priority`.  They are more like
+    :bro:attr:`&priority`.  They are more like functions in the way they
-    functions in the way they are invoked/called, because, unlike
+    are invoked/called, because, unlike events, their execution is
-    events, their execution is immediate and they do not get scheduled
+    immediate and they do not get scheduled through an event queue.
-    through an event queue.  Also, a unique feature of a hook is that
+    Also, a unique feature of a hook is that a given hook handler body
-    a given hook handler body can short-circuit the execution of
+    can short-circuit the execution of remaining hook handlers simply by
-    remaining hook handlers simply by exiting from the body as a result
+    exiting from the body as a result of a ``break`` statement (as
-    of a ``break`` statement (as opposed to a ``return`` or just
+    opposed to a ``return`` or just reaching the end of the body).
-    reaching the end of the body).
 
     A hook type is declared like::
 
@@ -549,18 +548,26 @@ The Bro scripting language supports the following built-in types.
             print "not going to happen", s;
             }
 
-    Note that, although the first (forward) declaration of ``myhook`` as
+    Note that the first (forward) declaration of ``myhook`` as a hook
-    a hook type isn't strictly required, when it is provided, the
+    type isn't strictly required.  Argument types must match for all
-    argument types must match.
+    hook handlers and any forward declaration of a given hook.
 
-    To invoke immediate execution of all hook handler bodies, a ``hook``
+    To invoke immediate execution of all hook handler bodies, they
-    statement must be used:
+    are called similarly to a function, except preceded by the ``hook``
+    keyword:
 
     .. code:: bro
 
         hook myhook("hi");
 
-    And the output would like like::
+    or
+
+    .. code:: bro
+
+        if ( hook myhook("hi") )
+            print "all handlers ran";
+
+    And the output would look like::
 
         priority 10 myhook handler, hi
         break out of myhook handling, bye
@@ -568,6 +575,12 @@ The Bro scripting language supports the following built-in types.
     Note how the modification to arguments can be seen by remaining
     hook handlers.
 
+    The return value of a hook call is an implicit :bro:type:`bool`
+    value with ``T`` meaning that all handlers for the hook were
+    executed and ``F`` meaning that only some of the handlers may have
+    executed due to one handler body exiting as a result of a ``break``
+    statement.
+
 Attributes
 ----------
 

scripts/base/frameworks/logging/writers/ascii.bro

@@ -2,11 +2,14 @@
 ##! to tweak the output format of ASCII logs.
 ##!
 ##! The ASCII writer supports currently one writer-specific filter option via
-##! ``config``: setting ``only_single_header_row`` to ``T`` turns the output into
+##! ``config``: setting ``tsv`` to the string ``T`` turns the output into into
-##! into CSV mode where only a single header row with the column names is printed
+##! "tab-separated-value" mode where only a single header row with the column names
-##! out as meta information. Example filter using this::
+##! is printed out as meta information, with no "# fields" prepended; no other meta
+##! data gets included in that mode.   
 ##! 
-##!    local my_filter: Log::Filter = [$name = "my-filter", $writer = Log::WRITER_ASCII, $config = table(["only_single_header_row"] = "T")]; 
+##! Example filter using this::
+##! 
+##!    local my_filter: Log::Filter = [$name = "my-filter", $writer = Log::WRITER_ASCII, $config = table(["tsv"] = "T")]; 
 ##!
 
 module LogAscii;

scripts/base/protocols/dns/main.bro

@@ -67,7 +67,7 @@ export {
 		ready:         bool            &default=F;
 		## The total number of resource records in a reply message's answer
 		## section.
-		total_answers: count           &default=0;
+		total_answers: count           &optional;
 		## The total number of resource records in a reply message's answer,
 		## authority, and additional sections.
 		total_replies: count           &optional;
@@ -231,6 +231,7 @@ event DNS::do_reply(c: connection, msg: dns_msg, ans: dns_answer, reply: string)
 		Log::write(DNS::LOG, c$dns);
 		# This record is logged and no longer pending.
 		delete c$dns_state$pending[c$dns$trans_id];
+		delete c$dns;
 		}
 	}
 

scripts/base/utils/files.bro

@@ -19,7 +19,7 @@ function generate_extraction_filename(prefix: string, c: connection, suffix: str
 ## the filename.
 function extract_filename_from_content_disposition(data: string): string
 	{
-	local filename = sub(data, /^.*[fF][iI][lL][eE][nN][aA][mM][eE]=/, "");
+	local filename = sub(data, /^.*[fF][iI][lL][eE][nN][aA][mM][eE][[:blank:]]*=[[:blank:]]*/, "");
 	# Remove quotes around the filename if they are there.
 	if ( /^\"/ in filename )
 		filename =  split_n(filename, /\"/, F, 2)[2];

src/Attr.cc

@@ -260,6 +260,11 @@ void Attributes::CheckAttr(Attr* a)
 				// Ok.
 				break;
 
+			if ( type->Tag() == TYPE_TABLE &&
+			     type->AsTableType()->IsUnspecifiedTable() )
+				// Ok.
+				break;
+
 			a->AttrExpr()->Error("&default value has inconsistent type", type);
 			}
 
@@ -290,6 +295,11 @@ void Attributes::CheckAttr(Attr* a)
 					// Ok.
 					break;
 
+				Expr* e = a->AttrExpr();
+				if ( check_and_promote_expr(e, ytype) )
+					// Ok.
+					break;
+
 				Error("&default value has inconsistent type 2");
 				}
 

src/Expr.cc

@@ -2663,7 +2663,7 @@ void AssignExpr::EvalIntoAggregate(const BroType* t, Val* aggr, Frame* f) const
 	TableVal* tv = aggr->AsTableVal();
 
 	Val* index = op1->Eval(f);
-	Val* v = op2->Eval(f);
+	Val* v = check_and_promote(op2->Eval(f), t->YieldType(), 1);
 	if ( ! index || ! v )
 		return;
 
@@ -3386,7 +3386,14 @@ Val* TableConstructorExpr::InitVal(const BroType* t, Val* aggr) const
 	if ( IsError() )
 		return 0;
 
-	return op->InitVal(t, aggr);
+	TableType* tt = Type()->AsTableType();
+	TableVal* tval = aggr ? aggr->AsTableVal() : new TableVal(tt, attrs);
+	const expr_list& exprs = op->AsListExpr()->Exprs();
+
+	loop_over_list(exprs, i)
+		exprs[i]->EvalIntoAggregate(t, tval, 0);
+
+	return tval;
 	}
 
 void TableConstructorExpr::ExprDescribe(ODesc* d) const
@@ -3438,7 +3445,7 @@ Val* SetConstructorExpr::Eval(Frame* f) const
 	if ( IsError() )
 		return 0;
 
-	TableVal* aggr = new TableVal(type->AsTableType(), 0);
+	TableVal* aggr = new TableVal(type->AsTableType(), attrs);
 	const expr_list& exprs = op->AsListExpr()->Exprs();
 
 	loop_over_list(exprs, i)
@@ -3456,7 +3463,26 @@ Val* SetConstructorExpr::InitVal(const BroType* t, Val* aggr) const
 	if ( IsError() )
 		return 0;
 
-	return op->InitVal(t, aggr);
+	const BroType* index_type = t->AsTableType()->Indices();
+	TableType* tt = Type()->AsTableType();
+	TableVal* tval = aggr ? aggr->AsTableVal() : new TableVal(tt, attrs);
+	const expr_list& exprs = op->AsListExpr()->Exprs();
+
+	loop_over_list(exprs, i)
+		{
+		Expr* e = exprs[i];
+		Val* element = check_and_promote(e->Eval(0), index_type, 1);
+
+		if ( ! element || ! tval->Assign(element, 0) )
+			{
+			Error(fmt("initialization type mismatch in set"), e);
+			return 0;
+			}
+
+		Unref(element);
+		}
+
+	return tval;
 	}
 
 void SetConstructorExpr::ExprDescribe(ODesc* d) const
@@ -3536,14 +3562,14 @@ Val* VectorConstructorExpr::InitVal(const BroType* t, Val* aggr) const
 	if ( IsError() )
 		return 0;
 
-	VectorVal* vec = aggr->AsVectorVal();
+	VectorType* vt = Type()->AsVectorType();
-	const BroType* vt = vec->Type()->AsVectorType()->YieldType();
+	VectorVal* vec = aggr ? aggr->AsVectorVal() : new VectorVal(vt);
 	const expr_list& exprs = op->AsListExpr()->Exprs();
 
 	loop_over_list(exprs, i)
 		{
 		Expr* e = exprs[i];
-		Val* v = check_and_promote(e->Eval(0), vt, 1);
+		Val* v = check_and_promote(e->Eval(0), t->YieldType(), 1);
 
 		if ( ! v || ! vec->Assign(i, v, e) )
 			{
@@ -4394,6 +4420,13 @@ CallExpr::CallExpr(Expr* arg_func, ListExpr* arg_args, bool in_hook)
 		return;
 		}
 
+	if ( func_type->AsFuncType()->Flavor() == FUNC_FLAVOR_HOOK && ! in_hook )
+		{
+		func->Error("hook cannot be called directly, use hook operator");
+		SetError();
+		return;
+		}
+
 	if ( ! func_type->MatchesIndex(args) )
 		SetError("argument type mismatch in function call");
 	else
@@ -4415,13 +4448,8 @@ CallExpr::CallExpr(Expr* arg_func, ListExpr* arg_args, bool in_hook)
 				break;
 
 			case FUNC_FLAVOR_HOOK:
-				// It's fine to not have a yield if it's known that the call
+				Error("hook has no yield type");
-				// is being done from a hook statement.
+				SetError();
-				if ( ! in_hook )
-					{
-					Error("hook called in expression, use hook statement instead");
-					SetError();
-					}
 				break;
 
 			default:

src/Expr.h

@@ -747,6 +747,8 @@ public:
 	TableConstructorExpr(ListExpr* constructor_list, attr_list* attrs);
 	~TableConstructorExpr()	{ Unref(attrs); }
 
+	Attributes* Attrs() { return attrs; }
+
 	Val* Eval(Frame* f) const;
 
 protected:
@@ -767,6 +769,8 @@ public:
 	SetConstructorExpr(ListExpr* constructor_list, attr_list* attrs);
 	~SetConstructorExpr()	{ Unref(attrs); }
 
+	Attributes* Attrs() { return attrs; }
+
 	Val* Eval(Frame* f) const;
 
 protected:

src/Func.cc

@@ -349,16 +349,31 @@ Val* BroFunc::Call(val_list* args, Frame* parent) const
 			break;
 			}
 
-		if ( flow == FLOW_BREAK && Flavor() == FUNC_FLAVOR_HOOK )
+		if ( Flavor() == FUNC_FLAVOR_HOOK )
 			{
-			// short-circuit execution of remaining hook handler bodies
+			// Ignore any return values of hook bodies, final return value
-			break;
+			// depends on whether a body returns as a result of break statement.
+			Unref(result);
+			result = 0;
+
+			if ( flow == FLOW_BREAK )
+				{
+				// Short-circuit execution of remaining hook handler bodies.
+				result = new Val(false, TYPE_BOOL);
+				break;
+				}
 			}
 		}
 
+	if ( Flavor() == FUNC_FLAVOR_HOOK )
+		{
+		if ( ! result )
+			result = new Val(true, TYPE_BOOL);
+		}
+
 	// Warn if the function returns something, but we returned from
 	// the function without an explicit return, or without a value.
-	if ( FType()->YieldType() && FType()->YieldType()->Tag() != TYPE_VOID &&
+	else if ( FType()->YieldType() && FType()->YieldType()->Tag() != TYPE_VOID &&
 	     (flow != FLOW_RETURN /* we fell off the end */ ||
 	      ! result /* explicit return with no result */) &&
 	     ! f->HasDelayed() )

src/SerialTypes.h

@@ -165,7 +165,6 @@ SERIAL_STMT(EVENT_BODY_LIST, 16)
 SERIAL_STMT(INIT_STMT, 17)
 SERIAL_STMT(NULL_STMT, 18)
 SERIAL_STMT(WHEN_STMT, 19)
-SERIAL_STMT(HOOK_STMT, 20)
 
 #define SERIAL_TYPE(name, val) SERIAL_CONST(name, val, BRO_TYPE)
 SERIAL_TYPE(BRO_TYPE, 1)

src/Stmt.cc

@@ -23,7 +23,7 @@ const char* stmt_name(BroStmtTag t)
 		"print", "event", "expr", "if", "when", "switch",
 		"for", "next", "break", "return", "add", "delete",
 		"list", "bodylist",
-		"<init>", "hook",
+		"<init>",
 		"null",
 	};
 
@@ -933,52 +933,6 @@ bool EventStmt::DoUnserialize(UnserialInfo* info)
 	return event_expr != 0;
 	}
 
-HookStmt::HookStmt(CallExpr* arg_e) : ExprStmt(STMT_HOOK, arg_e)
-	{
-	call_expr = arg_e;
-	}
-
-Val* HookStmt::Exec(Frame* f, stmt_flow_type& flow) const
-	{
-	RegisterAccess();
-
-	Val* ret = call_expr->Eval(f);
-	Unref(ret);
-
-	flow = FLOW_NEXT;
-
-	return 0;
-	}
-
-TraversalCode HookStmt::Traverse(TraversalCallback* cb) const
-	{
-	TraversalCode tc = cb->PreStmt(this);
-	HANDLE_TC_STMT_PRE(tc);
-
-	// call expr is stored in base class's "e" field.
-	tc = e->Traverse(cb);
-	HANDLE_TC_STMT_PRE(tc);
-
-	tc = cb->PostStmt(this);
-	HANDLE_TC_STMT_POST(tc);
-	}
-
-IMPLEMENT_SERIAL(HookStmt, SER_HOOK_STMT);
-
-bool HookStmt::DoSerialize(SerialInfo* info) const
-	{
-	DO_SERIALIZE(SER_HOOK_STMT, ExprStmt);
-	return call_expr->Serialize(info);
-	}
-
-bool HookStmt::DoUnserialize(UnserialInfo* info)
-	{
-	DO_UNSERIALIZE(ExprStmt);
-
-	call_expr = (CallExpr*) Expr::Unserialize(info, EXPR_CALL);
-	return call_expr != 0;
-	}
-
 ForStmt::ForStmt(id_list* arg_loop_vars, Expr* loop_expr)
 : ExprStmt(STMT_FOR, loop_expr)
 	{
@@ -1378,7 +1332,10 @@ ReturnStmt::ReturnStmt(Expr* arg_e) : ExprStmt(STMT_RETURN, arg_e)
 		}
 
 	else if ( ! e )
-		Error("return statement needs expression");
+		{
+		if ( ft->Flavor() != FUNC_FLAVOR_HOOK )
+			Error("return statement needs expression");
+		}
 
 	else
 		(void) check_and_promote_expr(e, yt);
@@ -1990,7 +1947,6 @@ int same_stmt(const Stmt* s1, const Stmt* s2)
 	case STMT_RETURN:
 	case STMT_EXPR:
 	case STMT_EVENT:
-	case STMT_HOOK:
 		{
 		const ExprStmt* e1 = (const ExprStmt*) s1;
 		const ExprStmt* e2 = (const ExprStmt*) s2;

src/Stmt.h

@@ -286,24 +286,6 @@ protected:
 	EventExpr* event_expr;
 };
 
-class HookStmt : public ExprStmt {
-public:
-	HookStmt(CallExpr* e);
-
-	Val* Exec(Frame* f, stmt_flow_type& flow) const;
-
-	TraversalCode Traverse(TraversalCallback* cb) const;
-
-protected:
-	friend class Stmt;
-
-	HookStmt() { call_expr = 0; }
-
-	DECLARE_SERIAL(HookStmt);
-
-	CallExpr* call_expr;
-};
-
 class ForStmt : public ExprStmt {
 public:
 	ForStmt(id_list* loop_vars, Expr* loop_expr);

src/StmtEnums.h

@@ -15,7 +15,7 @@ typedef enum {
 	STMT_RETURN,
 	STMT_ADD, STMT_DELETE,
 	STMT_LIST, STMT_EVENT_BODY_LIST,
-	STMT_INIT, STMT_HOOK,
+	STMT_INIT,
 	STMT_NULL
 #define NUM_STMTS (int(STMT_NULL) + 1)
 } BroStmtTag;

src/Val.cc

@@ -3117,6 +3117,9 @@ void VectorVal::ValDescribe(ODesc* d) const
 
 Val* check_and_promote(Val* v, const BroType* t, int is_init)
 	{
+	if ( ! v )
+		return 0;
+
 	BroType* vt = v->Type();
 
 	vt = flatten_type(vt);

src/Var.cc

@@ -109,6 +109,36 @@ static void make_var(ID* id, BroType* t, init_class c, Expr* init,
 	if ( attr )
 		id->AddAttrs(new Attributes(attr, t, false));
 
+	if ( init )
+		{
+		switch ( init->Tag() ) {
+		case EXPR_TABLE_CONSTRUCTOR:
+			{
+			TableConstructorExpr* ctor = (TableConstructorExpr*) init;
+			if ( ctor->Attrs() )
+				{
+				::Ref(ctor->Attrs());
+				id->AddAttrs(ctor->Attrs());
+				}
+			}
+			break;
+
+		case EXPR_SET_CONSTRUCTOR:
+			{
+			SetConstructorExpr* ctor = (SetConstructorExpr*) init;
+			if ( ctor->Attrs() )
+				{
+				::Ref(ctor->Attrs());
+				id->AddAttrs(ctor->Attrs());
+				}
+			}
+			break;
+
+		default:
+			break;
+		}
+		}
+
 	if ( id->FindAttr(ATTR_PERSISTENT) || id->FindAttr(ATTR_SYNCHRONIZED) )
 		{
 		if ( dt == VAR_CONST )
@@ -294,12 +324,12 @@ void add_type(ID* id, BroType* t, attr_list* attr, int /* is_event */)
 void begin_func(ID* id, const char* module_name, function_flavor flavor,
 		int is_redef, FuncType* t)
 	{
-	if ( flavor == FUNC_FLAVOR_EVENT || flavor == FUNC_FLAVOR_HOOK )
+	if ( flavor == FUNC_FLAVOR_EVENT )
 		{
 		const BroType* yt = t->YieldType();
 
 		if ( yt && yt->Tag() != TYPE_VOID )
-			id->Error("event/hook cannot yield a value", t);
+			id->Error("event cannot yield a value", t);
 
 		t->ClearYieldType(flavor);
 		}

src/logging/writers/Ascii.cc

@@ -19,7 +19,7 @@ Ascii::Ascii(WriterFrontend* frontend) : WriterBackend(frontend)
 	{
 	fd = 0;
 	ascii_done = false;
-	only_single_header_row = false;
+	tsv = false;
 
 	output_to_stdout = BifConst::LogAscii::output_to_stdout;
 	include_meta = BifConst::LogAscii::include_meta;
@@ -81,7 +81,7 @@ void Ascii::CloseFile(double t)
 	if ( ! fd )
 		return;
 
-	if ( include_meta && ! only_single_header_row )
+	if ( include_meta && ! tsv )
 		WriteHeaderField("close", Timestamp(0));
 
 	safe_close(fd);
@@ -111,17 +111,17 @@ bool Ascii::DoInit(const WriterInfo& info, int num_fields, const Field* const *
 
 	for ( WriterInfo::config_map::const_iterator i = info.config.begin(); i != info.config.end(); i++ )
 		{
-		if ( strcmp(i->first, "only_single_header_row") == 0 )
+		if ( strcmp(i->first, "tsv") == 0 )
 			{
 			if ( strcmp(i->second, "T") == 0 )
-				only_single_header_row = true;
+				tsv = true;
 
 			else if ( strcmp(i->second, "F") == 0 )
-				only_single_header_row = false;
+				tsv = false;
 
 			else
 				{
-				Error("invalid value for 'only_single_header_row', must be boolean (T/F)");
+				Error("invalid value for 'tsv', must be a string and either \"T\" or \"F\"");
 				return false;
 				}
 			}
@@ -144,9 +144,9 @@ bool Ascii::DoInit(const WriterInfo& info, int num_fields, const Field* const *
 			types += fields[i]->TypeName().c_str();
 			}
 
-		if ( only_single_header_row )
+		if ( tsv )
 			{
-			// A single CSV-style line is all we need.
+			// A single TSV-style line is all we need.
 			string str = names + "\n";
 			if ( ! safe_write(fd, str.c_str(), str.length()) )
 				goto write_error;

src/logging/writers/Ascii.h

@@ -45,7 +45,7 @@ private:
 	// Options set from the script-level.
 	bool output_to_stdout;
 	bool include_meta;
-	bool only_single_header_row;
+	bool tsv;
 
 	char* separator;
 	int separator_len;

src/parse.y

@@ -32,6 +32,7 @@
 
 %token TOK_NO_TEST
 
+%nonassoc TOK_HOOK
 %left ',' '|'
 %right '=' TOK_ADD_TO TOK_REMOVE_FROM
 %right '?' ':'
@@ -56,7 +57,6 @@
 %type <re> pattern
 %type <expr> expr init anonymous_function
 %type <event_expr> event
-%type <call_expr> hook
 %type <stmt> stmt stmt_list func_body for_head
 %type <type> type opt_type enum_body
 %type <func_type> func_hdr func_params
@@ -119,6 +119,7 @@ extern const char* g_curr_debug_error;
 
 #define YYLTYPE yyltype
 
+static int in_hook = 0;
 int in_init = 0;
 int in_record = 0;
 bool resolving_global_ID = false;
@@ -212,7 +213,6 @@ static std::list<std::string>* concat_opt_docs (std::list<std::string>* pre,
 	Val* val;
 	RE_Matcher* re;
 	Expr* expr;
-	CallExpr* call_expr;
 	EventExpr* event_expr;
 	Stmt* stmt;
 	ListExpr* list;
@@ -517,7 +517,16 @@ expr:
 	|	expr '(' opt_expr_list ')'
 			{
 			set_location(@1, @4);
-			$$ = new CallExpr($1, $3);
+			$$ = new CallExpr($1, $3, in_hook > 0);
+			}
+
+	|	TOK_HOOK { ++in_hook; } expr
+			{
+			--in_hook;
+			set_location(@1, @3);
+			if ( $3->Tag() != EXPR_CALL )
+				$3->Error("not a valid hook call expression");
+			$$ = $3;
 			}
 
 	|	expr TOK_HAS_FIELD TOK_ID
@@ -875,7 +884,7 @@ type:
 	|	TOK_HOOK '(' formal_args ')'
 				{
 				set_location(@1, @3);
-				$$ = new FuncType($3, 0, FUNC_FLAVOR_HOOK);
+				$$ = new FuncType($3, base_type(TYPE_BOOL), FUNC_FLAVOR_HOOK);
 				}
 
 	|	TOK_FILE TOK_OF type
@@ -1209,6 +1218,8 @@ func_hdr:
 			}
 	|	TOK_HOOK def_global_id func_params
 			{
+			$3->ClearYieldType(FUNC_FLAVOR_HOOK);
+			$3->SetYieldType(base_type(TYPE_BOOL));
 			begin_func($2, current_module.c_str(),
 				   FUNC_FLAVOR_HOOK, 0, $3);
 			$$ = $3;
@@ -1372,14 +1383,6 @@ stmt:
 			    brofiler.AddStmt($$);
 			}
 
-	|	TOK_HOOK hook ';' opt_no_test
-			{
-			set_location(@1, @4);
-			$$ = new HookStmt($2);
-			if ( ! $4 )
-			    brofiler.AddStmt($$);
-			}
-
 	|	TOK_IF '(' expr ')' stmt
 			{
 			set_location(@1, @4);
@@ -1533,14 +1536,6 @@ event:
 			}
 	;
 
-hook:
-		expr '(' opt_expr_list ')'
-			{
-			set_location(@1, @4);
-			$$ = new CallExpr($1, $3, true);
-			}
-	;
-
 case_list:
 		case_list case
 			{ $1->append($2); }

src/patricia.c

@@ -2,7 +2,7 @@
  * Dave Plonka <plonka@doit.wisc.edu>
  *
  * This product includes software developed by the University of Michigan,
- * Merit Network, Inc., and their contributors. 
+ * Merit Network, Inc., and their contributors.
  *
  * This file had been called "radix.c" in the MRT sources.
  *
@@ -12,28 +12,28 @@
  */
 
 /* From copyright.txt:
- * 
+ *
  * Copyright (c) 1997, 1998, 1999
- * 
+ *
- * 
+ *
  * The Regents of the University of Michigan ("The Regents") and Merit Network,
  * Inc.  All rights reserved.
  * Redistribution and use in source and binary forms, with or without
  * modification, are permitted provided that the following conditions are met:
- * 1.  Redistributions of source code must retain the above 
+ * 1.  Redistributions of source code must retain the above
- *     copyright notice, this list of conditions and the 
+ *     copyright notice, this list of conditions and the
  *     following disclaimer.
- * 2.  Redistributions in binary form must reproduce the above 
+ * 2.  Redistributions in binary form must reproduce the above
- *     copyright notice, this list of conditions and the 
+ *     copyright notice, this list of conditions and the
- *     following disclaimer in the documentation and/or other 
+ *     following disclaimer in the documentation and/or other
  *     materials provided with the distribution.
- * 3.  All advertising materials mentioning features or use of 
+ * 3.  All advertising materials mentioning features or use of
- *     this software must display the following acknowledgement:  
+ *     this software must display the following acknowledgement:
  * This product includes software developed by the University of Michigan, Merit
- * Network, Inc., and their contributors. 
+ * Network, Inc., and their contributors.
  * 4.  Neither the name of the University, Merit Network, nor the
- *     names of their contributors may be used to endorse or 
+ *     names of their contributors may be used to endorse or
- *     promote products derived from this software without 
+ *     promote products derived from this software without
  *     specific prior written permission.
  * THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS "AS IS" AND ANY
  * EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
@@ -44,7 +44,7 @@
  * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON
  * ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
  * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
- * SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.  
+ * SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
  */
 
 static char copyright[] =
@@ -66,6 +66,9 @@ static char copyright[] =
 
 #define Delete free
 
+// From Bro for reporting memory exhaustion.
+extern void out_of_memory(const char* where);
+
 /* { from prefix.c */
 
 /* prefix_tochar
@@ -80,7 +83,7 @@ prefix_tochar (prefix_t * prefix)
     return ((u_char *) & prefix->add.sin);
 }
 
-int 
+int
 comp_with_mask (void *addr, void *dest, u_int mask)
 {
 
@@ -95,15 +98,15 @@ comp_with_mask (void *addr, void *dest, u_int mask)
 }
 
 /* inet_pton substitute implementation
- * Uses inet_addr to convert an IP address in dotted decimal notation into 
+ * Uses inet_addr to convert an IP address in dotted decimal notation into
  * unsigned long and copies the result to dst.
- * Only supports AF_INET.  Follows standard error return conventions of 
+ * Only supports AF_INET.  Follows standard error return conventions of
  * inet_pton.
  */
 int
 local_inet_pton (int af, const char *src, void *dst)
 {
-    u_long result;  
+    u_long result;
 
     if (af == AF_INET) {
 	result = inet_addr(src);
@@ -166,7 +169,7 @@ my_inet_pton (int af, const char *src, void *dst)
     }
 }
 
-/* 
+/*
  * convert prefix information to ascii string with length
  * thread safe and (almost) re-entrant implementation
  */
@@ -251,6 +254,9 @@ New_Prefix2 (int family, void *dest, int bitlen, prefix_t *prefix)
         default_bitlen = 128;
 	if (prefix == NULL) {
             prefix = calloc(1, sizeof (prefix_t));
+            if (prefix == NULL)
+                out_of_memory("patrica/new_prefix2: unable to allocate memory");
+
 	    dynamic_allocated++;
 	}
 	memcpy (&prefix->add.sin6, dest, 16);
@@ -260,12 +266,16 @@ New_Prefix2 (int family, void *dest, int bitlen, prefix_t *prefix)
 		if (prefix == NULL) {
 #ifndef NT
             prefix = calloc(1, sizeof (prefix4_t));
+            if (prefix == NULL)
+                out_of_memory("patrica/new_prefix2: unable to allocate memory");
 #else
 			//for some reason, compiler is getting
 			//prefix4_t size incorrect on NT
-			prefix = calloc(1, sizeof (prefix_t)); 
+			prefix = calloc(1, sizeof (prefix_t));
+                        if (prefix == NULL)
+                            out_of_memory("patrica/new_prefix2: unable to allocate memory");
 #endif /* NT */
-		
+
 			dynamic_allocated++;
 		}
 		memcpy (&prefix->add.sin, dest, 4);
@@ -368,7 +378,7 @@ Ref_Prefix (prefix_t * prefix)
     return (prefix);
 }
 
-void 
+void
 Deref_Prefix (prefix_t * prefix)
 {
     if (prefix == NULL)
@@ -396,6 +406,8 @@ patricia_tree_t *
 New_Patricia (int maxbits)
 {
     patricia_tree_t *patricia = calloc(1, sizeof *patricia);
+    if (patricia == NULL)
+        out_of_memory("patrica/new_patricia: unable to allocate memory");
 
     patricia->maxbits = maxbits;
     patricia->head = NULL;
@@ -503,10 +515,10 @@ patricia_search_exact (patricia_tree_t *patricia, prefix_t *prefix)
 	if (BIT_TEST (addr[node->bit >> 3], 0x80 >> (node->bit & 0x07))) {
 #ifdef PATRICIA_DEBUG
 	    if (node->prefix)
-    	        fprintf (stderr, "patricia_search_exact: take right %s/%d\n", 
+    	        fprintf (stderr, "patricia_search_exact: take right %s/%d\n",
 	                 prefix_toa (node->prefix), node->prefix->bitlen);
 	    else
-    	        fprintf (stderr, "patricia_search_exact: take right at %d\n", 
+    	        fprintf (stderr, "patricia_search_exact: take right at %d\n",
 			 node->bit);
 #endif /* PATRICIA_DEBUG */
 	    node = node->r;
@@ -514,10 +526,10 @@ patricia_search_exact (patricia_tree_t *patricia, prefix_t *prefix)
 	else {
 #ifdef PATRICIA_DEBUG
 	    if (node->prefix)
-    	        fprintf (stderr, "patricia_search_exact: take left %s/%d\n", 
+    	        fprintf (stderr, "patricia_search_exact: take left %s/%d\n",
 	                 prefix_toa (node->prefix), node->prefix->bitlen);
 	    else
-    	        fprintf (stderr, "patricia_search_exact: take left at %d\n", 
+    	        fprintf (stderr, "patricia_search_exact: take left at %d\n",
 			 node->bit);
 #endif /* PATRICIA_DEBUG */
 	    node = node->l;
@@ -529,7 +541,7 @@ patricia_search_exact (patricia_tree_t *patricia, prefix_t *prefix)
 
 #ifdef PATRICIA_DEBUG
     if (node->prefix)
-        fprintf (stderr, "patricia_search_exact: stop at %s/%d\n", 
+        fprintf (stderr, "patricia_search_exact: stop at %s/%d\n",
 	         prefix_toa (node->prefix), node->prefix->bitlen);
     else
         fprintf (stderr, "patricia_search_exact: stop at %d\n", node->bit);
@@ -541,7 +553,7 @@ patricia_search_exact (patricia_tree_t *patricia, prefix_t *prefix)
     if (comp_with_mask (prefix_tochar (node->prefix), prefix_tochar (prefix),
 			bitlen)) {
 #ifdef PATRICIA_DEBUG
-        fprintf (stderr, "patricia_search_exact: found %s/%d\n", 
+        fprintf (stderr, "patricia_search_exact: found %s/%d\n",
 	         prefix_toa (node->prefix), node->prefix->bitlen);
 #endif /* PATRICIA_DEBUG */
 	return (node);
@@ -575,7 +587,7 @@ patricia_search_best2 (patricia_tree_t *patricia, prefix_t *prefix, int inclusiv
 
 	if (node->prefix) {
 #ifdef PATRICIA_DEBUG
-            fprintf (stderr, "patricia_search_best: push %s/%d\n", 
+            fprintf (stderr, "patricia_search_best: push %s/%d\n",
 	             prefix_toa (node->prefix), node->prefix->bitlen);
 #endif /* PATRICIA_DEBUG */
 	    stack[cnt++] = node;
@@ -584,10 +596,10 @@ patricia_search_best2 (patricia_tree_t *patricia, prefix_t *prefix, int inclusiv
 	if (BIT_TEST (addr[node->bit >> 3], 0x80 >> (node->bit & 0x07))) {
 #ifdef PATRICIA_DEBUG
 	    if (node->prefix)
-    	        fprintf (stderr, "patricia_search_best: take right %s/%d\n", 
+    	        fprintf (stderr, "patricia_search_best: take right %s/%d\n",
 	                 prefix_toa (node->prefix), node->prefix->bitlen);
 	    else
-    	        fprintf (stderr, "patricia_search_best: take right at %d\n", 
+    	        fprintf (stderr, "patricia_search_best: take right at %d\n",
 			 node->bit);
 #endif /* PATRICIA_DEBUG */
 	    node = node->r;
@@ -595,10 +607,10 @@ patricia_search_best2 (patricia_tree_t *patricia, prefix_t *prefix, int inclusiv
 	else {
 #ifdef PATRICIA_DEBUG
 	    if (node->prefix)
-    	        fprintf (stderr, "patricia_search_best: take left %s/%d\n", 
+    	        fprintf (stderr, "patricia_search_best: take left %s/%d\n",
 	                 prefix_toa (node->prefix), node->prefix->bitlen);
 	    else
-    	        fprintf (stderr, "patricia_search_best: take left at %d\n", 
+    	        fprintf (stderr, "patricia_search_best: take left at %d\n",
 			 node->bit);
 #endif /* PATRICIA_DEBUG */
 	    node = node->l;
@@ -615,7 +627,7 @@ patricia_search_best2 (patricia_tree_t *patricia, prefix_t *prefix, int inclusiv
     if (node == NULL)
         fprintf (stderr, "patricia_search_best: stop at null\n");
     else if (node->prefix)
-        fprintf (stderr, "patricia_search_best: stop at %s/%d\n", 
+        fprintf (stderr, "patricia_search_best: stop at %s/%d\n",
 	         prefix_toa (node->prefix), node->prefix->bitlen);
     else
         fprintf (stderr, "patricia_search_best: stop at %d\n", node->bit);
@@ -627,14 +639,14 @@ patricia_search_best2 (patricia_tree_t *patricia, prefix_t *prefix, int inclusiv
     while (--cnt >= 0) {
 	node = stack[cnt];
 #ifdef PATRICIA_DEBUG
-        fprintf (stderr, "patricia_search_best: pop %s/%d\n", 
+        fprintf (stderr, "patricia_search_best: pop %s/%d\n",
 	         prefix_toa (node->prefix), node->prefix->bitlen);
 #endif /* PATRICIA_DEBUG */
-	if (comp_with_mask (prefix_tochar (node->prefix), 
+	if (comp_with_mask (prefix_tochar (node->prefix),
 			    prefix_tochar (prefix),
 			    node->prefix->bitlen)) {
 #ifdef PATRICIA_DEBUG
-            fprintf (stderr, "patricia_search_best: found %s/%d\n", 
+            fprintf (stderr, "patricia_search_best: found %s/%d\n",
 	             prefix_toa (node->prefix), node->prefix->bitlen);
 #endif /* PATRICIA_DEBUG */
 	    return (node);
@@ -665,6 +677,9 @@ patricia_lookup (patricia_tree_t *patricia, prefix_t *prefix)
 
     if (patricia->head == NULL) {
 	node = calloc(1, sizeof *node);
+        if (node == NULL)
+            out_of_memory("patrica/patrica_lookup: unable to allocate memory");
+
 	node->bit = prefix->bitlen;
 	node->prefix = Ref_Prefix (prefix);
 	node->parent = NULL;
@@ -672,7 +687,7 @@ patricia_lookup (patricia_tree_t *patricia, prefix_t *prefix)
 	node->data = NULL;
 	patricia->head = node;
 #ifdef PATRICIA_DEBUG
-	fprintf (stderr, "patricia_lookup: new_node #0 %s/%d (head)\n", 
+	fprintf (stderr, "patricia_lookup: new_node #0 %s/%d (head)\n",
 		 prefix_toa (prefix), prefix->bitlen);
 #endif /* PATRICIA_DEBUG */
 	patricia->num_active_node++;
@@ -691,7 +706,7 @@ patricia_lookup (patricia_tree_t *patricia, prefix_t *prefix)
 		break;
 #ifdef PATRICIA_DEBUG
 	    if (node->prefix)
-    	        fprintf (stderr, "patricia_lookup: take right %s/%d\n", 
+    	        fprintf (stderr, "patricia_lookup: take right %s/%d\n",
 	                 prefix_toa (node->prefix), node->prefix->bitlen);
 	    else
     	        fprintf (stderr, "patricia_lookup: take right at %d\n", node->bit);
@@ -703,7 +718,7 @@ patricia_lookup (patricia_tree_t *patricia, prefix_t *prefix)
 		break;
 #ifdef PATRICIA_DEBUG
 	    if (node->prefix)
-    	        fprintf (stderr, "patricia_lookup: take left %s/%d\n", 
+    	        fprintf (stderr, "patricia_lookup: take left %s/%d\n",
 	             prefix_toa (node->prefix), node->prefix->bitlen);
 	    else
     	        fprintf (stderr, "patricia_lookup: take left at %d\n", node->bit);
@@ -716,7 +731,7 @@ patricia_lookup (patricia_tree_t *patricia, prefix_t *prefix)
 
     assert (node->prefix);
 #ifdef PATRICIA_DEBUG
-    fprintf (stderr, "patricia_lookup: stop at %s/%d\n", 
+    fprintf (stderr, "patricia_lookup: stop at %s/%d\n",
 	     prefix_toa (node->prefix), node->prefix->bitlen);
 #endif /* PATRICIA_DEBUG */
 
@@ -751,7 +766,7 @@ patricia_lookup (patricia_tree_t *patricia, prefix_t *prefix)
 	parent = node->parent;
 #ifdef PATRICIA_DEBUG
 	if (node->prefix)
-            fprintf (stderr, "patricia_lookup: up to %s/%d\n", 
+            fprintf (stderr, "patricia_lookup: up to %s/%d\n",
 	             prefix_toa (node->prefix), node->prefix->bitlen);
 	else
             fprintf (stderr, "patricia_lookup: up to %d\n", node->bit);
@@ -760,8 +775,8 @@ patricia_lookup (patricia_tree_t *patricia, prefix_t *prefix)
 
     if (differ_bit == bitlen && node->bit == bitlen) {
 	if (node->prefix) {
-#ifdef PATRICIA_DEBUG 
+#ifdef PATRICIA_DEBUG
-    	    fprintf (stderr, "patricia_lookup: found %s/%d\n", 
+    	    fprintf (stderr, "patricia_lookup: found %s/%d\n",
 		     prefix_toa (node->prefix), node->prefix->bitlen);
 #endif /* PATRICIA_DEBUG */
 	    return (node);
@@ -776,6 +791,9 @@ patricia_lookup (patricia_tree_t *patricia, prefix_t *prefix)
     }
 
     new_node = calloc(1, sizeof *new_node);
+    if (new_node == NULL)
+        out_of_memory("patrica/patrica_lookup: unable to allocate memory");
+
     new_node->bit = prefix->bitlen;
     new_node->prefix = Ref_Prefix (prefix);
     new_node->parent = NULL;
@@ -795,7 +813,7 @@ patricia_lookup (patricia_tree_t *patricia, prefix_t *prefix)
 	    node->l = new_node;
 	}
 #ifdef PATRICIA_DEBUG
-	fprintf (stderr, "patricia_lookup: new_node #2 %s/%d (child)\n", 
+	fprintf (stderr, "patricia_lookup: new_node #2 %s/%d (child)\n",
 		 prefix_toa (prefix), prefix->bitlen);
 #endif /* PATRICIA_DEBUG */
 	return (new_node);
@@ -822,12 +840,15 @@ patricia_lookup (patricia_tree_t *patricia, prefix_t *prefix)
 	}
 	node->parent = new_node;
 #ifdef PATRICIA_DEBUG
-	fprintf (stderr, "patricia_lookup: new_node #3 %s/%d (parent)\n", 
+	fprintf (stderr, "patricia_lookup: new_node #3 %s/%d (parent)\n",
 		 prefix_toa (prefix), prefix->bitlen);
 #endif /* PATRICIA_DEBUG */
     }
     else {
         glue = calloc(1, sizeof *glue);
+        if (glue == NULL)
+            out_of_memory("patrica/patrica_lookup: unable to allocate memory");
+
         glue->bit = differ_bit;
         glue->prefix = NULL;
         glue->parent = node->parent;
@@ -856,7 +877,7 @@ patricia_lookup (patricia_tree_t *patricia, prefix_t *prefix)
 	}
 	node->parent = glue;
 #ifdef PATRICIA_DEBUG
-	fprintf (stderr, "patricia_lookup: new_node #4 %s/%d (glue+node)\n", 
+	fprintf (stderr, "patricia_lookup: new_node #4 %s/%d (glue+node)\n",
 		 prefix_toa (prefix), prefix->bitlen);
 #endif /* PATRICIA_DEBUG */
     }
@@ -874,13 +895,13 @@ patricia_remove (patricia_tree_t *patricia, patricia_node_t *node)
 
     if (node->r && node->l) {
 #ifdef PATRICIA_DEBUG
-	fprintf (stderr, "patricia_remove: #0 %s/%d (r & l)\n", 
+	fprintf (stderr, "patricia_remove: #0 %s/%d (r & l)\n",
 		 prefix_toa (node->prefix), node->prefix->bitlen);
 #endif /* PATRICIA_DEBUG */
-	
+
 	/* this might be a placeholder node -- have to check and make sure
 	 * there is a prefix aossciated with it ! */
-	if (node->prefix != NULL) 
+	if (node->prefix != NULL)
 	  Deref_Prefix (node->prefix);
 	node->prefix = NULL;
 	/* Also I needed to clear data pointer -- masaki */
@@ -890,7 +911,7 @@ patricia_remove (patricia_tree_t *patricia, patricia_node_t *node)
 
     if (node->r == NULL && node->l == NULL) {
 #ifdef PATRICIA_DEBUG
-	fprintf (stderr, "patricia_remove: #1 %s/%d (!r & !l)\n", 
+	fprintf (stderr, "patricia_remove: #1 %s/%d (!r & !l)\n",
 		 prefix_toa (node->prefix), node->prefix->bitlen);
 #endif /* PATRICIA_DEBUG */
 	parent = node->parent;
@@ -937,7 +958,7 @@ patricia_remove (patricia_tree_t *patricia, patricia_node_t *node)
     }
 
 #ifdef PATRICIA_DEBUG
-    fprintf (stderr, "patricia_remove: #2 %s/%d (r ^ l)\n", 
+    fprintf (stderr, "patricia_remove: #2 %s/%d (r ^ l)\n",
 	     prefix_toa (node->prefix), node->prefix->bitlen);
 #endif /* PATRICIA_DEBUG */
     if (node->r) {
@@ -996,7 +1017,7 @@ try_search_exact (patricia_tree_t *tree, char *string)
         printf ("try_search_exact: not found\n");
     }
     else {
-        printf ("try_search_exact: %s/%d found\n", 
+        printf ("try_search_exact: %s/%d found\n",
 	        prefix_toa (node->prefix), node->prefix->bitlen);
     }
     Deref_Prefix (prefix);
@@ -1023,7 +1044,7 @@ try_search_best (patricia_tree_t *tree, char *string)
     if ((node = patricia_search_best (tree, prefix)) == NULL)
         printf ("try_search_best: not found\n");
     else
-        printf ("try_search_best: %s/%d found\n", 
+        printf ("try_search_best: %s/%d found\n",
 	        prefix_toa (node->prefix), node->prefix->bitlen);
     Deref_Prefix (prefix);
     return 0; // [RS] What is supposed to be returned here?

src/util.cc

@@ -1416,7 +1416,7 @@ void safe_close(int fd)
 		}
 	}
 
-void out_of_memory(const char* where)
+extern "C" void out_of_memory(const char* where)
 	{
 	fprintf(stderr, "out of memory in %s.\n", where);
 

src/util.h

@@ -303,7 +303,7 @@ extern bool safe_write(int fd, const char* data, int len);
 // Wraps close(2) to emit error messages and abort on unrecoverable errors.
 extern void safe_close(int fd);
 
-extern void out_of_memory(const char* where);
+extern "C" void out_of_memory(const char* where);
 
 inline void* safe_realloc(void* ptr, size_t size)
 	{

testing/btest/Baseline/bifs.strptime/.stdout

@@ -1,2 +0,0 @@
-1350604800.0
-0.0

testing/btest/Baseline/bifs.strptime/out

@@ -0,0 +1,3 @@
+warning: strptime conversion failed: fmt:%m d:1980-10-24
+1350604800.0
+0.0

testing/btest/Baseline/bifs.strptime/reporter.log

@@ -1,10 +0,0 @@
-#separator \x09
-#set_separator	,
-#empty_field	(empty)
-#unset_field	-
-#path	reporter
-#open	2012-10-19-06-06-36
-#fields	ts	level	message	location
-#types	time	enum	string	string
-0.000000	Reporter::WARNING	strptime conversion failed: fmt:%m d:1980-10-24	(empty)
-#close	2012-10-19-06-06-36

testing/btest/Baseline/language.hook/out

@@ -1,7 +1,18 @@
 myhook, &priority=10, [a=1156, b=hello world]
+myhook return F
+myhook return T
 myhook, &priority=5, [a=37, b=goobye world]
+F
 myhook3, 8
+T
 myhook4, 2
 myhook4, 1
+T
+myhook4, 2
+myhook4, 1
+myhook4 all handlers ran
 myhook, &priority=10, [a=2, b=it works]
+myhook return F
+myhook return T
 myhook, &priority=5, [a=37, b=goobye world]
+F

testing/btest/Baseline/language.hook_calls/invalid.out

@@ -0,0 +1,10 @@
+error in ./invalid.bro, line 9: hook cannot be called directly, use hook operator (myhook)
+warning in ./invalid.bro, line 9: expression value ignored (myhook(3))
+error in ./invalid.bro, line 10: hook cannot be called directly, use hook operator (myhook)
+error in ./invalid.bro, line 11: hook cannot be called directly, use hook operator (myhook)
+error in ./invalid.bro, line 12: not a valid hook call expression (2 + 2)
+warning in ./invalid.bro, line 12: expression value ignored (2 + 2)
+error in ./invalid.bro, line 13: not a valid hook call expression (2 + 2)
+error in ./invalid.bro, line 15: hook cannot be called directly, use hook operator (h)
+warning in ./invalid.bro, line 15: expression value ignored (h(3))
+error in ./invalid.bro, line 16: hook cannot be called directly, use hook operator (h)

testing/btest/Baseline/language.hook_calls/valid.out

@@ -0,0 +1,42 @@
+myhook(), 3
+other myhook(), 3
+myhook(), 3
+other myhook(), 3
+T
+myhook(), 0
+F
+-----------
+indirect()
+myhook(), 3
+other myhook(), 3
+indirect()
+myhook(), 3
+other myhook(), 3
+T
+-----------
+really_indirect()
+indirect()
+myhook(), 3
+other myhook(), 3
+really_indirect()
+indirect()
+myhook(), 3
+other myhook(), 3
+T
+-----------
+myhook(), 3
+other myhook(), 3
+myhook(), 3
+other myhook(), 3
+T
+myhook(), 3
+other myhook(), 3
+yes
+myhook(), 0
+double yes
+-----------
+myhook(), 3
+other myhook(), 3
+myhook(), 3
+other myhook(), 3
+T

testing/btest/Baseline/language.invalid_hook/out

@@ -1 +0,0 @@
-error in /Users/jsiwek/Projects/bro/bro/testing/btest/.tmp/language.invalid_hook/invalid_hook.bro, line 15: hook called in expression, use hook statement instead (myhook(nope))

testing/btest/Baseline/language.table-init-attrs/output

@@ -0,0 +1,91 @@
+my_set_ctor_init
+{
+test2,
+test3,
+test4,
+test1
+}
+
+my_table_ctor_init
+{
+[2] = test2,
+[1] = test1,
+[3] = test3
+}
+nope
+
+my_set_init
+{
+test2,
+test3,
+test4,
+test1
+}
+
+my_table_init
+{
+[2] = test2,
+[4] = test4,
+[1] = test1,
+[3] = test3
+}
+nope
+
+inception
+{
+[0] = {
+[13] = bar
+}
+}
+{
+[13] = bar
+}
+bar
+forty-two
+{
+
+}
+we need to go deeper
+{
+[0] = {
+[13] = bar
+}
+}
+{
+[13] = bar
+}
+bar
+forty-two
+{
+
+}
+we need to go deeper
+
+local table t1
+{
+[1] = foo
+}
+foo
+nope
+
+local table t2
+{
+[1] = foo
+}
+foo
+nope
+
+local table t3
+{
+
+}
+nope
+nope
+
+local table t4
+{
+
+}
+nope
+nope
+

testing/btest/Baseline/language.table-init-container-ctors/output

@@ -0,0 +1,50 @@
+table of set
+{
+[13] = {
+[bar, 2] ,
+[foo, 1] 
+},
+[5] = {
+[bah, 3] ,
+[baz, 4] 
+}
+}
+
+table of vector
+{
+[13] = [1, 2],
+[5] = [3, 4]
+}
+
+table of table
+{
+[13] = {
+[bar, 2] = 2,
+[foo, 1] = 1
+},
+[5] = {
+[bah, 3] = 3,
+[baz, 4] = 4
+}
+}
+
+table of record
+{
+[13] = [a=1, b=foo],
+[5] = [a=2, b=bar]
+}
+
+T
+T
+T
+T
+T
+T
+T
+T
+T
+T
+T
+T
+T
+T

testing/btest/bifs/addr_count_conversion.bro

@@ -1,4 +1,4 @@
-# @TEST-EXEC: bro %INPUT >output
+# @TEST-EXEC: bro -b %INPUT >output
 # @TEST-EXEC: btest-diff output
 
 global v: index_vec;

testing/btest/bifs/addr_to_ptr_name.bro

@@ -1,4 +1,4 @@
-# @TEST-EXEC: bro %INPUT >output
+# @TEST-EXEC: bro -b %INPUT >output
 # @TEST-EXEC: btest-diff output
 
 print addr_to_ptr_name([2607:f8b0:4009:802::1012]);

testing/btest/bifs/addr_version.bro

@@ -1,4 +1,4 @@
-# @TEST-EXEC: bro %INPUT >out
+# @TEST-EXEC: bro -b %INPUT >out
 # @TEST-EXEC: btest-diff out
 
 print is_v4_addr(1.2.3.4);

testing/btest/bifs/all_set.bro

@@ -1,5 +1,5 @@
 #
-# @TEST-EXEC: bro %INPUT >out
+# @TEST-EXEC: bro -b %INPUT >out
 # @TEST-EXEC: btest-diff out
 
 event bro_init()

testing/btest/bifs/analyzer_name.bro

@@ -1,5 +1,5 @@
 #
-# @TEST-EXEC: bro %INPUT >out
+# @TEST-EXEC: bro -b %INPUT >out
 # @TEST-EXEC: btest-diff out
 
 event bro_init()

testing/btest/bifs/any_set.bro

@@ -1,5 +1,5 @@
 #
-# @TEST-EXEC: bro %INPUT >out
+# @TEST-EXEC: bro -b %INPUT >out
 # @TEST-EXEC: btest-diff out
 
 event bro_init()

testing/btest/bifs/bro_version.bro

@@ -1,5 +1,5 @@
 #
-# @TEST-EXEC: bro %INPUT
+# @TEST-EXEC: bro -b %INPUT
 
 event bro_init()
 	{

testing/btest/bifs/byte_len.bro

@@ -1,5 +1,5 @@
 #
-# @TEST-EXEC: bro %INPUT >out
+# @TEST-EXEC: bro -b %INPUT >out
 # @TEST-EXEC: btest-diff out
 
 event bro_init()

testing/btest/bifs/bytestring_to_double.bro

@@ -1,5 +1,5 @@
 #
-# @TEST-EXEC: bro %INPUT >out
+# @TEST-EXEC: bro -b %INPUT >out
 # @TEST-EXEC: btest-diff out
 
 event bro_init()

testing/btest/bifs/bytestring_to_hexstr.bro

@@ -1,5 +1,5 @@
 #
-# @TEST-EXEC: bro %INPUT >out
+# @TEST-EXEC: bro -b %INPUT >out
 # @TEST-EXEC: btest-diff out
 
 event bro_init()

testing/btest/bifs/capture_state_updates.bro

@@ -1,5 +1,5 @@
 #
-# @TEST-EXEC: bro %INPUT >out
+# @TEST-EXEC: bro -b %INPUT >out
 # @TEST-EXEC: btest-diff out
 # @TEST-EXEC: test -f testfile
 

testing/btest/bifs/cat.bro

@@ -1,5 +1,5 @@
 #
-# @TEST-EXEC: bro %INPUT >out
+# @TEST-EXEC: bro -b %INPUT >out
 # @TEST-EXEC: btest-diff out
 
 event bro_init()

testing/btest/bifs/cat_string_array.bro

@@ -1,5 +1,5 @@
 #
-# @TEST-EXEC: bro %INPUT >out
+# @TEST-EXEC: bro -b %INPUT >out
 # @TEST-EXEC: btest-diff out
 
 event bro_init()

testing/btest/bifs/checkpoint_state.bro

@@ -1,5 +1,5 @@
 #
-# @TEST-EXEC: bro %INPUT
+# @TEST-EXEC: bro -b %INPUT
 # @TEST-EXEC: test -f .state/state.bst
 
 event bro_init()

testing/btest/bifs/clear_table.bro

@@ -1,5 +1,5 @@
 #
-# @TEST-EXEC: bro %INPUT > out
+# @TEST-EXEC: bro -b %INPUT > out
 # @TEST-EXEC: btest-diff out
 
 event bro_init()

testing/btest/bifs/convert_for_pattern.bro

@@ -1,5 +1,5 @@
 #
-# @TEST-EXEC: bro %INPUT >out
+# @TEST-EXEC: bro -b %INPUT >out
 # @TEST-EXEC: btest-diff out
 
 event bro_init()

testing/btest/bifs/count_to_addr.bro

@@ -1,5 +1,5 @@
 #
-# @TEST-EXEC: bro %INPUT >out
+# @TEST-EXEC: bro -b %INPUT >out
 # @TEST-EXEC: btest-diff out
 
 event bro_init()

testing/btest/bifs/create_file.bro

@@ -1,5 +1,5 @@
 #
-# @TEST-EXEC: bro %INPUT >out
+# @TEST-EXEC: bro -b %INPUT >out
 # @TEST-EXEC: btest-diff out
 # @TEST-EXEC: btest-diff testfile
 # @TEST-EXEC: btest-diff testfile2

testing/btest/bifs/current_analyzer.bro

@@ -1,5 +1,5 @@
 #
-# @TEST-EXEC: bro %INPUT
+# @TEST-EXEC: bro -b %INPUT
 
 event bro_init()
 	{

testing/btest/bifs/current_time.bro

@@ -1,5 +1,5 @@
 #
-# @TEST-EXEC: bro %INPUT
+# @TEST-EXEC: bro -b %INPUT
 
 event bro_init()
 	{

testing/btest/bifs/edit.bro

@@ -1,5 +1,5 @@
 #
-# @TEST-EXEC: bro %INPUT >out
+# @TEST-EXEC: bro -b %INPUT >out
 # @TEST-EXEC: btest-diff out
 
 event bro_init()

testing/btest/bifs/enable_raw_output.test

@@ -1,7 +1,7 @@
 # Files which enable raw output via the BiF shouldn't interpret NUL characters
 # in strings that are `print`ed to it.
 
-# @TEST-EXEC: bro %INPUT
+# @TEST-EXEC: bro -b %INPUT
 # @TEST-EXEC: tr '\000' 'X' <myfile >output
 # @TEST-EXEC: btest-diff output
 # @TEST-EXEC: cmp myfile hookfile

testing/btest/bifs/entropy_test.bro

@@ -1,5 +1,5 @@
 #
-# @TEST-EXEC: bro %INPUT >out
+# @TEST-EXEC: bro -b %INPUT >out
 # @TEST-EXEC: btest-diff out
 
 event bro_init()

testing/btest/bifs/escape_string.bro

@@ -1,5 +1,5 @@
 #
-# @TEST-EXEC: bro %INPUT >out
+# @TEST-EXEC: bro -b %INPUT >out
 # @TEST-EXEC: btest-diff out
 
 event bro_init()

testing/btest/bifs/exit.bro

@@ -1,5 +1,5 @@
 #
-# @TEST-EXEC: bro %INPUT >out || test $? -eq 7
+# @TEST-EXEC: bro -b %INPUT >out || test $? -eq 7
 # @TEST-EXEC: btest-diff out
 
 event bro_init()

testing/btest/bifs/file_mode.bro

@@ -1,5 +1,5 @@
 #
-# @TEST-EXEC: bro %INPUT >out
+# @TEST-EXEC: bro -b %INPUT >out
 # @TEST-EXEC: btest-diff out
 
 event bro_init()

testing/btest/bifs/find_all.bro

@@ -1,5 +1,5 @@
 #
-# @TEST-EXEC: bro %INPUT >out
+# @TEST-EXEC: bro -b %INPUT >out
 # @TEST-EXEC: btest-diff out
 
 event bro_init()

testing/btest/bifs/find_entropy.bro

@@ -1,5 +1,5 @@
 #
-# @TEST-EXEC: bro %INPUT >out
+# @TEST-EXEC: bro -b %INPUT >out
 # @TEST-EXEC: btest-diff out
 
 event bro_init()

testing/btest/bifs/find_last.bro

@@ -1,5 +1,5 @@
 #
-# @TEST-EXEC: bro %INPUT >out
+# @TEST-EXEC: bro -b %INPUT >out
 # @TEST-EXEC: btest-diff out
 
 event bro_init()

testing/btest/bifs/fmt.bro

@@ -1,5 +1,5 @@
 #
-# @TEST-EXEC: bro %INPUT >out
+# @TEST-EXEC: bro -b %INPUT >out
 # @TEST-EXEC: btest-diff out
 
 type color: enum { Red, Blue };

testing/btest/bifs/fmt_ftp_port.bro

@@ -1,5 +1,5 @@
 #
-# @TEST-EXEC: bro %INPUT >out
+# @TEST-EXEC: bro -b %INPUT >out
 # @TEST-EXEC: btest-diff out
 
 event bro_init()

testing/btest/bifs/get_matcher_stats.bro

@@ -1,5 +1,14 @@
 #
-# @TEST-EXEC: bro %INPUT
+# @TEST-EXEC: bro -b -s mysig %INPUT
+
+@TEST-START-FILE mysig.sig
+signature my_ftp_client {
+  ip-proto == tcp
+  payload /(|.*[\n\r]) *[uU][sS][eE][rR] /
+  tcp-state originator
+  event "matched my_ftp_client"
+}
+@TEST-END-FILE
 
 event bro_init()
 	{

testing/btest/bifs/get_port_transport_proto.bro

@@ -1,5 +1,5 @@
 #
-# @TEST-EXEC: bro %INPUT >out
+# @TEST-EXEC: bro -b %INPUT >out
 # @TEST-EXEC: btest-diff out
 
 event bro_init()

testing/btest/bifs/gethostname.bro

@@ -1,5 +1,5 @@
 #
-# @TEST-EXEC: bro %INPUT
+# @TEST-EXEC: bro -b %INPUT
 
 event bro_init()
 	{

testing/btest/bifs/getpid.bro

@@ -1,5 +1,5 @@
 #
-# @TEST-EXEC: bro %INPUT
+# @TEST-EXEC: bro -b %INPUT
 
 event bro_init()
 	{

testing/btest/bifs/getsetenv.bro

@@ -1,5 +1,5 @@
 #
-# @TEST-EXEC: TESTBRO=testvalue bro %INPUT >out
+# @TEST-EXEC: TESTBRO=testvalue bro -b %INPUT >out
 # @TEST-EXEC: btest-diff out
 
 event bro_init()

testing/btest/bifs/global_ids.bro

@@ -1,5 +1,5 @@
 #
-# @TEST-EXEC: bro %INPUT >out
+# @TEST-EXEC: bro -b %INPUT >out
 # @TEST-EXEC: btest-diff out
 
 event bro_init()

testing/btest/bifs/global_sizes.bro

@@ -1,5 +1,5 @@
 #
-# @TEST-EXEC: bro %INPUT >out
+# @TEST-EXEC: bro -b %INPUT >out
 # @TEST-EXEC: btest-diff out
 
 event bro_init()

testing/btest/bifs/hexdump.bro

@@ -1,5 +1,5 @@
 #
-# @TEST-EXEC: bro %INPUT >out
+# @TEST-EXEC: bro -b %INPUT >out
 # @TEST-EXEC: btest-diff out
 
 event bro_init()

testing/btest/bifs/identify_data.bro

@@ -1,5 +1,5 @@
 #
-# @TEST-EXEC: bro %INPUT | sed 's/PNG image data/PNG image/g' >out
+# @TEST-EXEC: bro -b %INPUT | sed 's/PNG image data/PNG image/g' >out
 # @TEST-EXEC: btest-diff out
 
 event bro_init()

testing/btest/bifs/is_ascii.bro

@@ -1,5 +1,5 @@
 #
-# @TEST-EXEC: bro %INPUT >out
+# @TEST-EXEC: bro -b %INPUT >out
 # @TEST-EXEC: btest-diff out
 
 event bro_init()

testing/btest/bifs/is_local_interface.bro

@@ -1,5 +1,5 @@
 #
-# @TEST-EXEC: bro %INPUT >out
+# @TEST-EXEC: bro -b %INPUT >out
 # @TEST-EXEC: btest-diff out
 
 event bro_init()

testing/btest/bifs/is_port.bro

@@ -1,5 +1,5 @@
 #
-# @TEST-EXEC: bro %INPUT >out
+# @TEST-EXEC: bro -b %INPUT >out
 # @TEST-EXEC: btest-diff out
 
 event bro_init()

testing/btest/bifs/join_string.bro

@@ -1,5 +1,5 @@
 #
-# @TEST-EXEC: bro %INPUT >out
+# @TEST-EXEC: bro -b %INPUT >out
 # @TEST-EXEC: btest-diff out
 
 event bro_init()

testing/btest/bifs/length.bro

@@ -1,5 +1,5 @@
 #
-# @TEST-EXEC: bro %INPUT > out
+# @TEST-EXEC: bro -b %INPUT > out
 # @TEST-EXEC: btest-diff out
 
 event bro_init()

testing/btest/bifs/lookup_ID.bro

@@ -1,5 +1,5 @@
 #
-# @TEST-EXEC: bro %INPUT >out
+# @TEST-EXEC: bro -b %INPUT >out
 # @TEST-EXEC: btest-diff out
 
 global a = "bro test";

testing/btest/bifs/lowerupper.bro

@@ -1,5 +1,5 @@
 #
-# @TEST-EXEC: bro %INPUT >out
+# @TEST-EXEC: bro -b %INPUT >out
 # @TEST-EXEC: btest-diff out
 
 event bro_init()

testing/btest/bifs/mask_addr.bro

@@ -1,4 +1,4 @@
-# @TEST-EXEC: bro %INPUT >output
+# @TEST-EXEC: bro -b %INPUT >output
 # @TEST-EXEC: btest-diff output
 
 const one_to_32: vector of count = {1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32};

testing/btest/bifs/math.bro

@@ -1,5 +1,5 @@
 #
-# @TEST-EXEC: bro %INPUT >out
+# @TEST-EXEC: bro -b %INPUT >out
 # @TEST-EXEC: btest-diff out
 
 event bro_init()

testing/btest/bifs/merge_pattern.bro

@@ -1,5 +1,5 @@
 #
-# @TEST-EXEC: bro %INPUT >out
+# @TEST-EXEC: bro -b %INPUT >out
 # @TEST-EXEC: btest-diff out
 
 event bro_init()

testing/btest/bifs/netbios-functions.bro

@@ -1,5 +1,5 @@
 #
-# @TEST-EXEC: bro %INPUT >out
+# @TEST-EXEC: bro -b %INPUT >out
 # @TEST-EXEC: btest-diff out
 
 event bro_init()

testing/btest/bifs/order.bro

@@ -1,5 +1,5 @@
 #
-# @TEST-EXEC: bro %INPUT >out
+# @TEST-EXEC: bro -b %INPUT >out
 # @TEST-EXEC: btest-diff out
 
 function myfunc1(a: addr, b: addr): int

testing/btest/bifs/parse_ftp.bro

@@ -1,5 +1,5 @@
 #
-# @TEST-EXEC: bro %INPUT >out
+# @TEST-EXEC: bro -b %INPUT >out
 # @TEST-EXEC: btest-diff out
 
 event bro_init()

testing/btest/bifs/piped_exec.bro

@@ -1,4 +1,4 @@
-# @TEST-EXEC: bro %INPUT >output
+# @TEST-EXEC: bro -b %INPUT >output
 # @TEST-EXEC: btest-diff output
 # @TEST-EXEC: btest-diff test.txt
 

testing/btest/bifs/ptr_name_to_addr.bro

@@ -1,4 +1,4 @@
-# @TEST-EXEC: bro %INPUT >output
+# @TEST-EXEC: bro -b %INPUT >output
 # @TEST-EXEC: btest-diff output
 
 global v6 = ptr_name_to_addr("2.1.0.1.0.0.0.0.0.0.0.0.0.0.0.0.2.0.8.0.9.0.0.4.0.b.8.f.7.0.6.2.ip6.arpa");

testing/btest/bifs/raw_bytes_to_v4_addr.bro

@@ -1,5 +1,5 @@
 #
-# @TEST-EXEC: bro %INPUT >out
+# @TEST-EXEC: bro -b %INPUT >out
 # @TEST-EXEC: btest-diff out
 
 event bro_init()

testing/btest/bifs/reading_traces.bro

@@ -1,5 +1,5 @@
 
-# @TEST-EXEC: bro %INPUT >out1
+# @TEST-EXEC: bro -b %INPUT >out1
 # @TEST-EXEC: btest-diff out1
 # @TEST-EXEC: bro -r $TRACES/web.trace %INPUT >out2
 # @TEST-EXEC: btest-diff out2

testing/btest/bifs/record_type_to_vector.bro

@@ -1,5 +1,5 @@
 #
-# @TEST-EXEC: bro %INPUT >out
+# @TEST-EXEC: bro -b %INPUT >out
 # @TEST-EXEC: btest-diff out
 
 type myrecord: record {

testing/btest/bifs/records_fields.bro

@@ -1,5 +1,5 @@
 #
-# @TEST-EXEC: bro %INPUT >out
+# @TEST-EXEC: bro -b %INPUT >out
 # @TEST-EXEC: btest-diff out
 
 type r: record {

testing/btest/bifs/remask_addr.bro

@@ -1,4 +1,4 @@
-# @TEST-EXEC: bro %INPUT >output
+# @TEST-EXEC: bro -b %INPUT >output
 # @TEST-EXEC: btest-diff output
 
 const one_to_32: vector of count = {1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32};

testing/btest/bifs/resize.bro

@@ -1,5 +1,5 @@
 #
-# @TEST-EXEC: bro %INPUT >out
+# @TEST-EXEC: bro -b %INPUT >out
 # @TEST-EXEC: btest-diff out
 
 event bro_init()

testing/btest/bifs/resource_usage.bro

@@ -1,5 +1,5 @@
 #
-# @TEST-EXEC: bro %INPUT
+# @TEST-EXEC: bro -b %INPUT
 
 event bro_init()
 	{

testing/btest/bifs/rotate_file.bro

@@ -1,5 +1,5 @@
 #
-# @TEST-EXEC: bro %INPUT >out
+# @TEST-EXEC: bro -b %INPUT >out
 # @TEST-EXEC: btest-diff out
 
 event bro_init()

testing/btest/bifs/rotate_file_by_name.bro

@@ -1,5 +1,5 @@
 #
-# @TEST-EXEC: bro %INPUT >out
+# @TEST-EXEC: bro -b %INPUT >out
 # @TEST-EXEC: btest-diff out
 
 event bro_init()

testing/btest/bifs/same_object.bro

@@ -1,5 +1,5 @@
 #
-# @TEST-EXEC: bro %INPUT >out
+# @TEST-EXEC: bro -b %INPUT >out
 # @TEST-EXEC: btest-diff out
 
 event bro_init()

testing/btest/bifs/sort.bro

@@ -1,5 +1,5 @@
 #
-# @TEST-EXEC: bro %INPUT >out
+# @TEST-EXEC: bro -b %INPUT >out
 # @TEST-EXEC: btest-diff out
 
 function myfunc1(a: addr, b: addr): int

testing/btest/bifs/sort_string_array.bro

@@ -1,5 +1,5 @@
 #
-# @TEST-EXEC: bro %INPUT >out
+# @TEST-EXEC: bro -b %INPUT >out
 # @TEST-EXEC: btest-diff out
 
 event bro_init()

testing/btest/bifs/split.bro

@@ -1,5 +1,5 @@
 #
-# @TEST-EXEC: bro %INPUT >out
+# @TEST-EXEC: bro -b %INPUT >out
 # @TEST-EXEC: btest-diff out
 
 event bro_init()