It works with a simple example, but that's as much testing as it has
seen so far.
Remote::Destination has a new attribute "request_logs: bool"
indicating whether we are interested in the peer's log. Default is
false. If true, Bro will send an explicit "I want your logs" message
over to the other side, which will then start sending log records
back.
When such log records are received, they will be recorded exactly in
the same way as on the remote side, i.e., same fields/writer/path. All
filtering is already performed on the remote side.
Log::Filter has two new attributes, "log_local: bool" and
"log_remote: bool" (both true by default). If log_local is false, this
filter will not record anything locally but still process everything
normally otherwise and potentially forward to remote. If log_remote is
false, this filter will never send anything to remote even if a peer
has requested logs. (Note that with the defaults, requesting logs will
mean getting everything.)
Note that with log forwarding, *both* sides must create the
Filter::Stream. If the remote sends log records for a specific stream,
but the local side hasn't created it, the data will be discarded.
Filtes on the other hand shouldn't created locally; and if they are,
they are ignored for records received from remote).
- Moving all functions into the Log::* namespace, using the recent
bifcl updates. Moved logging-specific stuff to logging.bif.
- Log::create_stream() now takes a record Log::Stream as its second
argument, which specifies columns and (optionally) the event.
- All the internal BiFs are now called "Log::__<something>", with
script-level wrappers "Log::<something>". That first allows to add
additional code at the script-level, and second makes things better
comprehendible as now all relevant functionality is collected (and
later documetned) in policy/logging.bro.
- New function Log::flush(id), which does the obvious assuming the
writer supports it.
- add_default_filter() is now called implicitly with every
create_stream(). Seems that we usually want that functionality, and
when not, remove_default_filter() gets rid of it.
- The namespace of a stream's ID is now used as the default "path"
(e.g., if the namespace is SSH, the default log file is "ssh.log").
- Updated policy/test-logging.bro as well as the btest tests according
to these changes.
* origin/topic/gregor/bif-tuning:
Refactor: BifTypePtr --> BifType
Bif const: make sure const is indeed a constant.
Support any type in bif const declaration.
Tweak for bifcl
Fix to bifcl wrt namespaces.
Enable declaration of set, vector, and table types in bifs.
Moving type declarations into its own bif file
Support namespaces / modules in bif. Checkpoint.
Support namespaces / modules in bif. Checkpoint.
Remove leftovers from removing "declare enum" from bifcl
Use namespaces for NetVar type pointers.
Remove unused and unnecessary "declare enum" from bifcl
Bif: add record type declaration.
Minor tweaks for bif language.
enum type: don't allow mixing of explicit value and auto-increment.
Add support for enum with explicit enumerator values.
Closes#403.
* New bro runtime options: -Z or --doc-scripts enables documentation mode
* New BroDoc, BroBifDoc, and BroDocObj interfaces to support script
documentation
* Modifications to the bro scanner (scan.l) to get it to keep track of
which script is being scanned/parsed and which document is being generated
* Modifications to scan.l and the bro parser (parse.y) to produce/consume
script comments denoted with "##"
* Documentation is currently generated for the following
** Script author
** Script summary
** @load's
** capture_filters
** modules (namespaces)
Most of the remaining framework/infrastructure work should be in extracting
the interesting BroObj objects as the parser sees them and better formatting
the reST documents.
* origin/topic/gregor/fix-val-64bit:
Fixing endianess error in XDR when data is not 4-byte aligned.
Fix for Val constructor with new int64 typedefs.
New fix for OS X 10.5 compile error wrt llabs()
Revert "Fix for OS X 10.5 compile error wrt llabs()"
The event has moved from the filters to the streams, and must now be
specificed when creating the stream. (Not clear yet whether that is a
indeed the right interface).
When an event was globally decleared, previously it did not get
assigned a value initially until the first implementation body was
added. That then triggered an "not used" error when passing such an
event as argument into a bif. Now we always assign a function value
immediately, just without any body inititally.
When globally declaring an event, i
This pretty much follows the proposal on the projects page.
It includes:
- A new LogMgr, maintaining the set of writers.
- The abstract LogWriter API.
- An initial implementation in the form of LogWriterAscii
producing tab-separated columns.
Note that things are only partially working right now, things are
subject to change, and it's all not much tested at all. That's why I'm
creating separate branch for now.
Example:
bro -B logging test-logging && cat debug.log
1298063168.409852/1298063168.410368 [logging] Created new logging stream 'SSH::LOG_SSH'
1298063168.409852/1298063168.410547 [logging] Created new filter 'default' for stream 'SSH::LOG_SSH'
1298063168.409852/1298063168.410564 [logging] writer : Ascii
1298063168.409852/1298063168.410574 [logging] path : ssh_log_ssh
1298063168.409852/1298063168.410584 [logging] path_func : not set
1298063168.409852/1298063168.410594 [logging] event : not set
1298063168.409852/1298063168.410604 [logging] pred : not set
1298063168.409852/1298063168.410614 [logging] field t: time
1298063168.409852/1298063168.410625 [logging] field id.orig_h: addr
1298063168.409852/1298063168.410635 [logging] field id.orig_p: port
1298063168.409852/1298063168.410645 [logging] field id.resp_h: addr
1298063168.409852/1298063168.410655 [logging] field id.resp_p: port
1298063168.409852/1298063168.410665 [logging] field status: string
1298063168.409852/1298063168.410675 [logging] field country: string
1298063168.409852/1298063168.410817 [logging] Wrote record to filter 'default' on stream 'SSH::LOG_SSH'
1298063168.409852/1298063168.410865 [logging] Wrote record to filter 'default' on stream 'SSH::LOG_SSH'
1298063168.409852/1298063168.410906 [logging] Wrote record to filter 'default' on stream 'SSH::LOG_SSH'
1298063168.409852/1298063168.410945 [logging] Wrote record to filter 'default' on stream 'SSH::LOG_SSH'
1298063168.409852/1298063168.411044 [logging] Wrote record to filter 'default' on stream 'SSH::LOG_SSH
> cat ssh_log_ssh.log
1298063168.40985 1.2.3.4 66770 2.3.4.5 65616 success unknown
1298063168.40985 1.2.3.4 66770 2.3.4.5 65616 failure US
1298063168.40985 1.2.3.4 66770 2.3.4.5 65616 failure UK
1298063168.40985 1.2.3.4 66770 2.3.4.5 65616 success BR
1298063168.40985 1.2.3.4 66770 2.3.4.5 65616 failure MX
logging framework.
- To enable passing a type into a bif, there's now a new
BroType-derived class TypeType and a corresponding TYPE_TYPE tag.
With that, a Val can now have a type as its value.
This is experimental for now.
- RecordVal's get a new method CoerceTo() to coerce their value into a
another record type with the usual semantics. Most of the code in
there was previously in RecordContructorExpr::InitVal(), which is
now calling the new CoerceTo() method.
Revamp of const delcaration in bifs:
* Can only declare are const in the bif, but we cannot assign a value
or attribute to it. One has to do this in a policy file (bro.init)
* Type specification in bif is now mandatory
* Support any type in bifs (previously only bools were supported).
This will also help with automatic documentation generation, since all
const are now defined in the policy layer and thus can be documented
from there. The bif just gives the C++ layer easy access.
(now actually commiting all the files)
This change is actually two-fold:
a) bif's now accept module XYZ; statements and module::ID for
function, const, event, enum, etc. declartation
b) Added C++-namespaces to variables, functions, etc. that are declared
in bif but accessed from C++
This required some (lightweight) re-factoring of the C++ codes.
Note, event's don't have their own C++ namespace yet, since this
would require a rather huge re-factoring.
Compiles and passes test suite.
New namespace feature not tested yet.
Documentation to follow.
This change is actually two-fold:
a) bif's now accept module XYZ; statements and module::ID for
function, const, event, enum, etc. declartation
b) Added C++-namespaces to variables, functions, etc. that are declared
in bif but accessed from C++
This required some (lightweight) re-factoring of the C++ codes.
Note, event's don't have their own C++ namespace yet, since this
would require a rather huge re-factoring.
Compiles and passes test suite.
New namespace feature not tested yet.
Documentation to follow.
Enums defined in bifs and records declared in bifs are now available
in the C++ layer in namespaces (before they were in the global namespace
with enum_* and rectype_* prefixes).
Namespaces are now BroTypePtr::Enum::<name-of-enum> and
BroTypePtr::Record::<name-of-record>
One can now declare (but not define) a record type in bif:
type <my_record_type_name> : record;
This adds the netvar glue so that the event engine knows about the type. One
still has to define the type in bro.init. Would be nice, if we could
just define the record type here and then copy to the .bif.bro file, but
type delcarations in bro can be quite powerful. Don't know whether it's
worth it extend the bif-language to be able to handle that all.... Or
we just support a simple form of record type definitions
The type has be called <my_record_type_name> in bro.init and it will
be availabe as a RecordType * rectype_<my_record_type_name> in the event
engine.
TODO: add other types (tables, sets)
Updated enum type. New description:
Enum's are supported in .bif and .bro scripts.
An enum in a bif will become available in the event engine and
the policy layer.
It is possible to assign an explicit value to an enum enumerator
element, or the enum type can automatically assign values. However,
the styles cannot be mixed. If automatic assignement is used, the first
element will have a value of 0, the next will have a value of 1, etc.
Enum type variables and identifiers can be formated using the "%s"
format specifier, in which case the symbolic name will be printed.
If the "%d" format specifier is used, the numerical value is
printed.
Example automatic assignment:
type foo: enum {
BAR_A, # value will be 0
BAR_B, # value will be 1
BAR_C, # value will be 2
};
Example with explicit assignment:
type foobar: enum {
BAR_X = 10, # value will be 10
BAR_Y = 23, # value will be 23
BAR_Z = 42, # value will be 42
};
Enumerator values can only by positive integer literals.
The literals can be specified in (0x....), but not in octal (bro policy
layer limitation). So, do not use 0123 as value in bifs!
Each enumerator value can only be used once per enum (C allows
to use the same value multiple times).
All these restrictions are enforced by the policy script layer and not
the bif compiler!
Enums can be redef'ed, i.e., extended. If the enum is automatic
increment assignment, then the value will continue to increment.
If the enum uses explicit assignment, then the redef need to use
explicit assignments as well.
Example 1::
redef enum foo += {
BAR_D, # value will be 3
BAR_E, # value will be 4
BAR_F, # value will be 5
};
Example 2::
redef enum foobar += {
BAR_W = 100,
};
* Adding support for enums with explicit enumerator values (see doc
below) to bifcl and policy layer.
* Bifcl: remove (partially written) output files on error and
do a nice exit(1) instead of harsh abort() on parse errors.
* CMakeText: if bifcl fails, remove output files (failsafe,
in case bifcl fails to clean up after itself).
Enum description
----------------
Enum's are supported in .bif and .bro scripts.
An enum in a bif will become available in the event engine and
the policy layer.
Enums are "C-style". The first element in an enum will have a
value of 0, the next value will be 1, etc.
It is possible to assign an enumerator value to an element. If
next element does not have an explicit value, its values will be
the value of the last element + 1
Example::
type foo: enum {
BAR_A, # value will be 0
BAR_B, # value will be 1
BAR_C = 10, # value will be 10
BAR_D, # value will be 11
};
Enumerator values can only by positive integer literals.
The literals can be specified in (0x....), but not in octal (bro policy
layer limitation). So, do not use 0123 as value in bifs!
Each enumerator value can only be used once per enum (C allows
to use the same value multiple times). This makes reverse mapping from
value to name (e.g., in %s format strings) unambigious. This is enforced
in by the policy script.
Enums can be redef'ed, i.e., extended. Enumerator values will continue
to increment. If there are multiple redefs in different policy scripts,
then name <-> value mappings will obviously depend on the order in
which scripts are loaded (which might not be obvious).
Example::
redef enum foo += {
BAR_E, # value will be 12
BAR_F = 5, # value will be 5
BAR_G, # value will be 6
};
Val::Val had prototypes for int, long, int64, etc. But depending on the
architecture some of those might be the same (int64 and long) thus
yielding a compile error.
Fix: only use int32, int64, etc. for prototype. ints and longs can still
be passed, since they will match one of these fixed-width types
regardless of platform.
Also fix some more compiler warnings with format strings.
