* origin/topic/dnthayer/bif-tests:
Improve "fmt" BIF documentation comment
Improve tests of the type_name BIF
Improve test cases for "order" BIF
Fix documentation of sort BIF and add more tests
Fix documentation for system_env BIF
Deprecate the parse_dotted_addr BIF (use to_addr instead)
Improve tests for to_port and type_name BIFs
Improve tests for sort, order, and system_env BIFs
Fix the join_string_vec BIF and add more tests
Add more tests for previously-untested BIFs
Add more tests for previously-untested BIFs
Add more tests for previously-untested BIFs
Add more tests for previously-untested BIFs
Add tests for previously-untested strings BIFs
The Logger class is now in charge of reporting all errors, warnings,
informational messages, weirds, and syslogs. All other components
route their messages through the global bro_logger singleton.
The Logger class comes with these reporting methods:
void Message(const char* fmt, ...);
void Warning(const char* fmt, ...);
void Error(const char* fmt, ...);
void FatalError(const char* fmt, ...); // Terminate Bro.
void Weird(const char* name);
[ .. some more Weird() variants ... ]
void Syslog(const char* fmt, ...);
void InternalWarning(const char* fmt, ...);
void InternalError(const char* fmt, ...); // Terminates Bro.
See Logger.h for more information on these.
Generally, the reporting now works as follows:
- All non-fatal message are reported in one of two ways:
(1) At startup (i.e., before we start processing packets),
they are logged to stderr.
(2) During processing, they turn into events:
event log_message%(msg: string, location: string%);
event log_warning%(msg: string, location: string%);
event log_error%(msg: string, location: string%);
The script level can then handle them as desired.
If we don't have an event handler, we fall back to
reporting on stderr.
- All fatal errors are logged to stderr and Bro terminates
immediately.
- Syslog(msg) directly syslogs, but doesn't do anything else.
The three main types of messages can also be generated on the
scripting layer via new Log::* bifs:
Log::error(msg: string);
Log::warning(msg: string);
Log::message(msg: string);
These pass through the bro_logger as well and thus are handled in the
same way. Their output includes location information.
More changes:
- Removed the alarm statement and the alarm_hook event.
- Adapted lots of locations to use the bro_logger, including some
of the messages that were previously either just written to
stdout, or even funneled through the alarm mechanism.
- No distinction anymore between Error() and RunTime(). There's
now only one class of errors; the line was quite blurred already
anyway.
- util.h: all the error()/warn()/message()/run_time()/pinpoint()
functions are gone. Use the bro_logger instead now.
- Script errors are formatted a bit differently due to the
changes. What I've seen so far looks ok to me, but let me know
if there's something odd.
Notes:
- The default handlers for the new log_* events are just dummy
implementations for now since we need to integrate all this into
the new scripts anyway.
- I'm not too happy with the names of the Logger class and its
instance bro_logger. We now have a LogMgr as well, which makes
this all a bit confusing. But I didn't have a good idea for
better names so I stuck with them for now.
Perhaps we should merge Logger and LogMgr?
with the field.
This works now:
type X: record {
a: table[string] of bool &default=table( ["foo"] = T );
b: table[string] of bool &default=table();
c: set[string] &default=set("A", "B", "C");
d: set[string] &default=set();
};
I think previously the intend was to associate &default with the
table/set (i.e., define the default value for non-existing indices).
However, that was already not working: the error checking was
reporting type mismatches. So, this shouldn't break anything and make
things more consistent.
The documentation framework now sees "##Text" and "## Text" as
equivalent documentation comments. This prevents unintentional
indentation in the generated reST as a result of the later style, but
still allows embedded reST markup that relies on indentation of more
than two spaces to work as expected.
Comments associated with record fields and enums values are able
to span multiple "##"-stylized comments, allowing for more robust
reST markup to be embedded.
The documentation framework now tracks record fields through
a new CommentedTypeDecl subclass of TypeDecl that the parser constructs
in parallel with the real TypeDecl.
This adds a new subclass of EnumType, CommentedEnumType, and removes
any previous changes to EnumType that were done to support the
autodoc framework.
Dummy CommentedEnumType and ID's are constructed in parallel with the
real EnumType ID's during parsing and passed on to the autodoc framework.
This allows the generated documentation to track enum redefs, with
a special case being the "Notice" enum type.
"##" style comments before identifiers and "##<" style after identifiers
in the body of an enum type declaration will now show up in the
auto-generated reST documentation.
- Fixing a crash with an invalid pointer.
- Fixing a namespacing problem with is_ftp_data_conn() and check_relay_3().
- Fixing the do-we-have-an-event-handler-defined check.
Standard test-suite passes.
Seth, I think you can give it a try now ...
Changed BroType to track a char* instead of an ID* that represents
the declared type's identifier. It was also necessary to serialize
this information or else it can be lost (e.g. FieldDecl's in RecordType
always seem to get serialized at some point).
DescribeReST() functions added to many classes to get the output
closer to being reST compatible; still needs tweaking for Sphinx
(reST->HTML) compatibility.
- Moving all functions into the Log::* namespace, using the recent
bifcl updates. Moved logging-specific stuff to logging.bif.
- Log::create_stream() now takes a record Log::Stream as its second
argument, which specifies columns and (optionally) the event.
- All the internal BiFs are now called "Log::__<something>", with
script-level wrappers "Log::<something>". That first allows to add
additional code at the script-level, and second makes things better
comprehendible as now all relevant functionality is collected (and
later documetned) in policy/logging.bro.
- New function Log::flush(id), which does the obvious assuming the
writer supports it.
- add_default_filter() is now called implicitly with every
create_stream(). Seems that we usually want that functionality, and
when not, remove_default_filter() gets rid of it.
- The namespace of a stream's ID is now used as the default "path"
(e.g., if the namespace is SSH, the default log file is "ssh.log").
- Updated policy/test-logging.bro as well as the btest tests according
to these changes.
* origin/topic/gregor/bif-tuning:
Refactor: BifTypePtr --> BifType
Bif const: make sure const is indeed a constant.
Support any type in bif const declaration.
Tweak for bifcl
Fix to bifcl wrt namespaces.
Enable declaration of set, vector, and table types in bifs.
Moving type declarations into its own bif file
Support namespaces / modules in bif. Checkpoint.
Support namespaces / modules in bif. Checkpoint.
Remove leftovers from removing "declare enum" from bifcl
Use namespaces for NetVar type pointers.
Remove unused and unnecessary "declare enum" from bifcl
Bif: add record type declaration.
Minor tweaks for bif language.
enum type: don't allow mixing of explicit value and auto-increment.
Add support for enum with explicit enumerator values.
Closes#403.
logging framework.
- To enable passing a type into a bif, there's now a new
BroType-derived class TypeType and a corresponding TYPE_TYPE tag.
With that, a Val can now have a type as its value.
This is experimental for now.
- RecordVal's get a new method CoerceTo() to coerce their value into a
another record type with the usual semantics. Most of the code in
there was previously in RecordContructorExpr::InitVal(), which is
now calling the new CoerceTo() method.
Updated enum type. New description:
Enum's are supported in .bif and .bro scripts.
An enum in a bif will become available in the event engine and
the policy layer.
It is possible to assign an explicit value to an enum enumerator
element, or the enum type can automatically assign values. However,
the styles cannot be mixed. If automatic assignement is used, the first
element will have a value of 0, the next will have a value of 1, etc.
Enum type variables and identifiers can be formated using the "%s"
format specifier, in which case the symbolic name will be printed.
If the "%d" format specifier is used, the numerical value is
printed.
Example automatic assignment:
type foo: enum {
BAR_A, # value will be 0
BAR_B, # value will be 1
BAR_C, # value will be 2
};
Example with explicit assignment:
type foobar: enum {
BAR_X = 10, # value will be 10
BAR_Y = 23, # value will be 23
BAR_Z = 42, # value will be 42
};
Enumerator values can only by positive integer literals.
The literals can be specified in (0x....), but not in octal (bro policy
layer limitation). So, do not use 0123 as value in bifs!
Each enumerator value can only be used once per enum (C allows
to use the same value multiple times).
All these restrictions are enforced by the policy script layer and not
the bif compiler!
Enums can be redef'ed, i.e., extended. If the enum is automatic
increment assignment, then the value will continue to increment.
If the enum uses explicit assignment, then the redef need to use
explicit assignments as well.
Example 1::
redef enum foo += {
BAR_D, # value will be 3
BAR_E, # value will be 4
BAR_F, # value will be 5
};
Example 2::
redef enum foobar += {
BAR_W = 100,
};
* Adding support for enums with explicit enumerator values (see doc
below) to bifcl and policy layer.
* Bifcl: remove (partially written) output files on error and
do a nice exit(1) instead of harsh abort() on parse errors.
* CMakeText: if bifcl fails, remove output files (failsafe,
in case bifcl fails to clean up after itself).
Enum description
----------------
Enum's are supported in .bif and .bro scripts.
An enum in a bif will become available in the event engine and
the policy layer.
Enums are "C-style". The first element in an enum will have a
value of 0, the next value will be 1, etc.
It is possible to assign an enumerator value to an element. If
next element does not have an explicit value, its values will be
the value of the last element + 1
Example::
type foo: enum {
BAR_A, # value will be 0
BAR_B, # value will be 1
BAR_C = 10, # value will be 10
BAR_D, # value will be 11
};
Enumerator values can only by positive integer literals.
The literals can be specified in (0x....), but not in octal (bro policy
layer limitation). So, do not use 0123 as value in bifs!
Each enumerator value can only be used once per enum (C allows
to use the same value multiple times). This makes reverse mapping from
value to name (e.g., in %s format strings) unambigious. This is enforced
in by the policy script.
Enums can be redef'ed, i.e., extended. Enumerator values will continue
to increment. If there are multiple redefs in different policy scripts,
then name <-> value mappings will obviously depend on the order in
which scripts are loaded (which might not be obvious).
Example::
redef enum foo += {
BAR_E, # value will be 12
BAR_F = 5, # value will be 5
BAR_G, # value will be 6
};
This is per #375.
Record types can now get additional fields later via '+='. The added
fields must however either be &optional or have a &default value.
Example:
type Foo: record {
a: count;
b: count &optional;
};
redef record Foo += {
c: count &default=42;
d: count &optional;
};
global f: Foo = [$a=21];
print f;
Output:
[a=21, b=<uninitialized>, c=42, d=<uninitialized>]
table/set indices.
This addresses #367. In principle, the fix is quite straightford.
However, it turns out that sometimes record fields lost their
attributes on assignment, and then the hashing can't decide anymore
whether a field is optional or not. So that needed to be fixed as
well.
