* New bro runtime options: -Z or --doc-scripts enables documentation mode
* New BroDoc, BroBifDoc, and BroDocObj interfaces to support script
documentation
* Modifications to the bro scanner (scan.l) to get it to keep track of
which script is being scanned/parsed and which document is being generated
* Modifications to scan.l and the bro parser (parse.y) to produce/consume
script comments denoted with "##"
* Documentation is currently generated for the following
** Script author
** Script summary
** @load's
** capture_filters
** modules (namespaces)
Most of the remaining framework/infrastructure work should be in extracting
the interesting BroObj objects as the parser sees them and better formatting
the reST documents.
* origin/topic/gregor/fix-val-64bit:
Fixing endianess error in XDR when data is not 4-byte aligned.
Fix for Val constructor with new int64 typedefs.
New fix for OS X 10.5 compile error wrt llabs()
Revert "Fix for OS X 10.5 compile error wrt llabs()"
- Duplicates of the distribution's configuration files are now always
installed with a .example suffix
- Added --binary-package configure option to toggle configure logic
specific to the creation of binary packages.
- When not in binary packaging mode, `make install` never overwrites
existing configure files in case they've been modified. The previous
behavior (CMake's default) would only avoid overwriting modified files
if one consistently uses the same build directory and doesn't reconfigure.
- Fixed an issue with Mac package's pre-install script not preserving ACLs
- Minor cleanup/refactor of the make-mac/rpm-packages scripts
The event has moved from the filters to the streams, and must now be
specificed when creating the stream. (Not clear yet whether that is a
indeed the right interface).
When an event was globally decleared, previously it did not get
assigned a value initially until the first implementation body was
added. That then triggered an "not used" error when passing such an
event as argument into a bif. Now we always assign a function value
immediately, just without any body inititally.
When globally declaring an event, i
This pretty much follows the proposal on the projects page.
It includes:
- A new LogMgr, maintaining the set of writers.
- The abstract LogWriter API.
- An initial implementation in the form of LogWriterAscii
producing tab-separated columns.
Note that things are only partially working right now, things are
subject to change, and it's all not much tested at all. That's why I'm
creating separate branch for now.
Example:
bro -B logging test-logging && cat debug.log
1298063168.409852/1298063168.410368 [logging] Created new logging stream 'SSH::LOG_SSH'
1298063168.409852/1298063168.410547 [logging] Created new filter 'default' for stream 'SSH::LOG_SSH'
1298063168.409852/1298063168.410564 [logging] writer : Ascii
1298063168.409852/1298063168.410574 [logging] path : ssh_log_ssh
1298063168.409852/1298063168.410584 [logging] path_func : not set
1298063168.409852/1298063168.410594 [logging] event : not set
1298063168.409852/1298063168.410604 [logging] pred : not set
1298063168.409852/1298063168.410614 [logging] field t: time
1298063168.409852/1298063168.410625 [logging] field id.orig_h: addr
1298063168.409852/1298063168.410635 [logging] field id.orig_p: port
1298063168.409852/1298063168.410645 [logging] field id.resp_h: addr
1298063168.409852/1298063168.410655 [logging] field id.resp_p: port
1298063168.409852/1298063168.410665 [logging] field status: string
1298063168.409852/1298063168.410675 [logging] field country: string
1298063168.409852/1298063168.410817 [logging] Wrote record to filter 'default' on stream 'SSH::LOG_SSH'
1298063168.409852/1298063168.410865 [logging] Wrote record to filter 'default' on stream 'SSH::LOG_SSH'
1298063168.409852/1298063168.410906 [logging] Wrote record to filter 'default' on stream 'SSH::LOG_SSH'
1298063168.409852/1298063168.410945 [logging] Wrote record to filter 'default' on stream 'SSH::LOG_SSH'
1298063168.409852/1298063168.411044 [logging] Wrote record to filter 'default' on stream 'SSH::LOG_SSH
> cat ssh_log_ssh.log
1298063168.40985 1.2.3.4 66770 2.3.4.5 65616 success unknown
1298063168.40985 1.2.3.4 66770 2.3.4.5 65616 failure US
1298063168.40985 1.2.3.4 66770 2.3.4.5 65616 failure UK
1298063168.40985 1.2.3.4 66770 2.3.4.5 65616 success BR
1298063168.40985 1.2.3.4 66770 2.3.4.5 65616 failure MX
logging framework.
- To enable passing a type into a bif, there's now a new
BroType-derived class TypeType and a corresponding TYPE_TYPE tag.
With that, a Val can now have a type as its value.
This is experimental for now.
- RecordVal's get a new method CoerceTo() to coerce their value into a
another record type with the usual semantics. Most of the code in
there was previously in RecordContructorExpr::InitVal(), which is
now calling the new CoerceTo() method.
Revamp of const delcaration in bifs:
* Can only declare are const in the bif, but we cannot assign a value
or attribute to it. One has to do this in a policy file (bro.init)
* Type specification in bif is now mandatory
* Support any type in bifs (previously only bools were supported).
This will also help with automatic documentation generation, since all
const are now defined in the policy layer and thus can be documented
from there. The bif just gives the C++ layer easy access.
(now actually commiting all the files)
This change is actually two-fold:
a) bif's now accept module XYZ; statements and module::ID for
function, const, event, enum, etc. declartation
b) Added C++-namespaces to variables, functions, etc. that are declared
in bif but accessed from C++
This required some (lightweight) re-factoring of the C++ codes.
Note, event's don't have their own C++ namespace yet, since this
would require a rather huge re-factoring.
Compiles and passes test suite.
New namespace feature not tested yet.
Documentation to follow.
This change is actually two-fold:
a) bif's now accept module XYZ; statements and module::ID for
function, const, event, enum, etc. declartation
b) Added C++-namespaces to variables, functions, etc. that are declared
in bif but accessed from C++
This required some (lightweight) re-factoring of the C++ codes.
Note, event's don't have their own C++ namespace yet, since this
would require a rather huge re-factoring.
Compiles and passes test suite.
New namespace feature not tested yet.
Documentation to follow.
Enums defined in bifs and records declared in bifs are now available
in the C++ layer in namespaces (before they were in the global namespace
with enum_* and rectype_* prefixes).
Namespaces are now BroTypePtr::Enum::<name-of-enum> and
BroTypePtr::Record::<name-of-record>
One can now declare (but not define) a record type in bif:
type <my_record_type_name> : record;
This adds the netvar glue so that the event engine knows about the type. One
still has to define the type in bro.init. Would be nice, if we could
just define the record type here and then copy to the .bif.bro file, but
type delcarations in bro can be quite powerful. Don't know whether it's
worth it extend the bif-language to be able to handle that all.... Or
we just support a simple form of record type definitions
The type has be called <my_record_type_name> in bro.init and it will
be availabe as a RecordType * rectype_<my_record_type_name> in the event
engine.
TODO: add other types (tables, sets)
Updated enum type. New description:
Enum's are supported in .bif and .bro scripts.
An enum in a bif will become available in the event engine and
the policy layer.
It is possible to assign an explicit value to an enum enumerator
element, or the enum type can automatically assign values. However,
the styles cannot be mixed. If automatic assignement is used, the first
element will have a value of 0, the next will have a value of 1, etc.
Enum type variables and identifiers can be formated using the "%s"
format specifier, in which case the symbolic name will be printed.
If the "%d" format specifier is used, the numerical value is
printed.
Example automatic assignment:
type foo: enum {
BAR_A, # value will be 0
BAR_B, # value will be 1
BAR_C, # value will be 2
};
Example with explicit assignment:
type foobar: enum {
BAR_X = 10, # value will be 10
BAR_Y = 23, # value will be 23
BAR_Z = 42, # value will be 42
};
Enumerator values can only by positive integer literals.
The literals can be specified in (0x....), but not in octal (bro policy
layer limitation). So, do not use 0123 as value in bifs!
Each enumerator value can only be used once per enum (C allows
to use the same value multiple times).
All these restrictions are enforced by the policy script layer and not
the bif compiler!
Enums can be redef'ed, i.e., extended. If the enum is automatic
increment assignment, then the value will continue to increment.
If the enum uses explicit assignment, then the redef need to use
explicit assignments as well.
Example 1::
redef enum foo += {
BAR_D, # value will be 3
BAR_E, # value will be 4
BAR_F, # value will be 5
};
Example 2::
redef enum foobar += {
BAR_W = 100,
};
* Adding support for enums with explicit enumerator values (see doc
below) to bifcl and policy layer.
* Bifcl: remove (partially written) output files on error and
do a nice exit(1) instead of harsh abort() on parse errors.
* CMakeText: if bifcl fails, remove output files (failsafe,
in case bifcl fails to clean up after itself).
Enum description
----------------
Enum's are supported in .bif and .bro scripts.
An enum in a bif will become available in the event engine and
the policy layer.
Enums are "C-style". The first element in an enum will have a
value of 0, the next value will be 1, etc.
It is possible to assign an enumerator value to an element. If
next element does not have an explicit value, its values will be
the value of the last element + 1
Example::
type foo: enum {
BAR_A, # value will be 0
BAR_B, # value will be 1
BAR_C = 10, # value will be 10
BAR_D, # value will be 11
};
Enumerator values can only by positive integer literals.
The literals can be specified in (0x....), but not in octal (bro policy
layer limitation). So, do not use 0123 as value in bifs!
Each enumerator value can only be used once per enum (C allows
to use the same value multiple times). This makes reverse mapping from
value to name (e.g., in %s format strings) unambigious. This is enforced
in by the policy script.
Enums can be redef'ed, i.e., extended. Enumerator values will continue
to increment. If there are multiple redefs in different policy scripts,
then name <-> value mappings will obviously depend on the order in
which scripts are loaded (which might not be obvious).
Example::
redef enum foo += {
BAR_E, # value will be 12
BAR_F = 5, # value will be 5
BAR_G, # value will be 6
};
Val::Val had prototypes for int, long, int64, etc. But depending on the
architecture some of those might be the same (int64 and long) thus
yielding a compile error.
Fix: only use int32, int64, etc. for prototype. ints and longs can still
be passed, since they will match one of these fixed-width types
regardless of platform.
Also fix some more compiler warnings with format strings.
* origin/topic/seth/fix-compiler-warnings:
Fixed problem with PRI macros.
PRI macros are currently not working for some reason.
Two more small compile time error fixes.
Cleaned up the output from running binpac.
Added line to expect shift/reduce errors in parse.in
Cleaned up g++ warnings.
Addition: I fixed a few more warnings I was getting, and tweaked some
of the existing changes slightly.
* remotes/origin/topic/robin/work:
*Now* this passes the test suite.
Fixes to SSL/TLS analyzer
Added new TLS ciphers
Removing some apparently unnecessary lines.
A few smaller tweaks.
Prepared the old analyzer for extracting SSL extensions.
Fixed bug in do_split implementation.
Removed an accidental debugging printf.
Readded the other changes to remove CheckString calls from strings.bif.
Fixed the problem with do_split function which caused it to bail 1 separator early.
Modification from rmkml to support SSL extensions.
Updated SSL analyzer and Bro script with lots of new ciphers.
* remotes/origin/topic/seth/strings-without-checkstring:
*Now* this passes the test suite.
Removing some apparently unnecessary lines.
A few smaller tweaks.
Fixed bug in do_split implementation.
Removed an accidental debugging printf.
Readded the other changes to remove CheckString calls from strings.bif.
Fixed the problem with do_split function which caused it to bail 1 separator early.
