- bro.init was renamed to base/init-bare.bro and base/all.bro
was renamed to init-default.bro.
- To run in "bare mode" with only the init-bare.bro and no other
scripts from base/, use either -b or --bare-mode.
- The environment variable to run in "bare mode" has been removed.
- policy/ renamed to scripts/
- By default BROPATH now contains:
- scripts/
- scripts/policy
- scripts/site
- *Nearly* all tests pass.
- All of scripts/base/ is loaded by main.cc
- Can be disabled by setting $BRO_NO_BASE_SCRIPTS
- Scripts in scripts/base/ don't use relative path loading to ease use of BRO_NO_BASE_SCRIPTS (to copy and paste that script).
- The scripts in scripts/base/protocols/ only (or soon will only) do logging and state building.
- The scripts in scripts/base/frameworks/ add functionality without causing any additional overhead.
- All "detection" activity happens through scripts in scripts/policy/.
- Communications framework modified temporarily to need an environment variable to actually enable (ENABLE_COMMUNICATION=1)
- This is so the communications framework can be loaded as part
of the base without causing trouble when it's not needed.
- This will be removed once a resolution to ticket #540 is reached.
Also removing the -l command-line option as that can now be done at
the script-level.
A couple tests fail now that use -l. Leaving that until we have
script-level replacement.
* origin/topic/robin/reporting:
Syslog BiF now goes through the reporter as well.
Avoiding infinite loops when an error message handlers triggers errors itself.
Renaming the Logger to Reporter.
Overhauling the internal reporting of messages to the user.
Updating a bunch of tests/baselines as well.
Conflicts:
aux/broccoli
policy.old/alarm.bro
policy/all.bro
policy/bro.init
policy/frameworks/notice/weird.bro
policy/notice.bro
src/SSL-binpac.cc
src/bro.bif
src/main.cc
* origin/fastpath:
Raise internal error when failing to read contents of state file with -x option
I've changed this to use error() instead of internal_error(). The
latter should only be used for logic errors that indicate a bug in
Bro. In this case, the message flags a problem that's more likely to
be external.
The Logger class is now in charge of reporting all errors, warnings,
informational messages, weirds, and syslogs. All other components
route their messages through the global bro_logger singleton.
The Logger class comes with these reporting methods:
void Message(const char* fmt, ...);
void Warning(const char* fmt, ...);
void Error(const char* fmt, ...);
void FatalError(const char* fmt, ...); // Terminate Bro.
void Weird(const char* name);
[ .. some more Weird() variants ... ]
void Syslog(const char* fmt, ...);
void InternalWarning(const char* fmt, ...);
void InternalError(const char* fmt, ...); // Terminates Bro.
See Logger.h for more information on these.
Generally, the reporting now works as follows:
- All non-fatal message are reported in one of two ways:
(1) At startup (i.e., before we start processing packets),
they are logged to stderr.
(2) During processing, they turn into events:
event log_message%(msg: string, location: string%);
event log_warning%(msg: string, location: string%);
event log_error%(msg: string, location: string%);
The script level can then handle them as desired.
If we don't have an event handler, we fall back to
reporting on stderr.
- All fatal errors are logged to stderr and Bro terminates
immediately.
- Syslog(msg) directly syslogs, but doesn't do anything else.
The three main types of messages can also be generated on the
scripting layer via new Log::* bifs:
Log::error(msg: string);
Log::warning(msg: string);
Log::message(msg: string);
These pass through the bro_logger as well and thus are handled in the
same way. Their output includes location information.
More changes:
- Removed the alarm statement and the alarm_hook event.
- Adapted lots of locations to use the bro_logger, including some
of the messages that were previously either just written to
stdout, or even funneled through the alarm mechanism.
- No distinction anymore between Error() and RunTime(). There's
now only one class of errors; the line was quite blurred already
anyway.
- util.h: all the error()/warn()/message()/run_time()/pinpoint()
functions are gone. Use the bro_logger instead now.
- Script errors are formatted a bit differently due to the
changes. What I've seen so far looks ok to me, but let me know
if there's something odd.
Notes:
- The default handlers for the new log_* events are just dummy
implementations for now since we need to integrate all this into
the new scripts anyway.
- I'm not too happy with the names of the Logger class and its
instance bro_logger. We now have a LogMgr as well, which makes
this all a bit confusing. But I didn't have a good idea for
better names so I stuck with them for now.
Perhaps we should merge Logger and LogMgr?
Originally docs were written right after parsing, but it changed to after
the bro_init event happens when I was experimenting with auto-documenting
logging streams by querying the LogMgr after bro_init. That experiment
dead-ended, and that location is bad for other reasons: the doc framework
may try to access BroObj's that have already been freed.
* remotes/origin/topic/jsiwek/doc-framework:
Adding example documentation for a script's use of logging features.
Adding &log attribute to static attr_names array.
Small typo fix.
Bro doc mode now tracks record redefs that extend its field list.
BroBifDoc was unneeded; now dead code, so removed.
Bro doc mode now only does a "shallow" copy of declared record types
Bro's doc mode now terminates after processing bro_init but before net_run
Fixes related to `make doc` handling of script summary text (##! comments)
Overhaul of "doc" build target for generating policy script documentation.
Add parser error hint when in doc mode about checking ## comment syntax.
Move stuff related to policy script documentation from doc/ to doc/scripts/
Fixing example.bro's auto-reST generation baseline test.
Generated script reST documentation is also written out at this time
instead of at the end of lexical scanning.
The persistence serializer will no longer write out Bro's state to the
.state directory when in doc mode.
filter. (Seth Hall and Robin Sommer)
- Merging in the patch from #264, which provides support for mixed
VLAN and MPLS traffic.
- Changing Bro's default filter from being built dynamically to being
a static "ip or not ip". To get the old behaviour back (i.e., the
dynamically built filter), redef "all_packets" to false.
- print-filter.bro now always prints the filter that Bro is actually
using, even if overriden from the command line.
the random number generator.
This works like the corresponding command line option but is more
convinient when writing tests as it can be set in btest.cfg.
Removing everything related to trace rewriting.
(I wasn't too careful in ensuring that I catch everything in the
scripts; Seth is working on those anyway.)
(Merging by cherry-picking the corresponding commit, as the branch was
accidentally made off of the logging stuff).
the random number generator.
This works like the corresponding command line option but is more
convinient when writing tests as it can be set in btest.cfg.
* New bro runtime options: -Z or --doc-scripts enables documentation mode
* New BroDoc, BroBifDoc, and BroDocObj interfaces to support script
documentation
* Modifications to the bro scanner (scan.l) to get it to keep track of
which script is being scanned/parsed and which document is being generated
* Modifications to scan.l and the bro parser (parse.y) to produce/consume
script comments denoted with "##"
* Documentation is currently generated for the following
** Script author
** Script summary
** @load's
** capture_filters
** modules (namespaces)
Most of the remaining framework/infrastructure work should be in extracting
the interesting BroObj objects as the parser sees them and better formatting
the reST documents.
This pretty much follows the proposal on the projects page.
It includes:
- A new LogMgr, maintaining the set of writers.
- The abstract LogWriter API.
- An initial implementation in the form of LogWriterAscii
producing tab-separated columns.
Note that things are only partially working right now, things are
subject to change, and it's all not much tested at all. That's why I'm
creating separate branch for now.
Example:
bro -B logging test-logging && cat debug.log
1298063168.409852/1298063168.410368 [logging] Created new logging stream 'SSH::LOG_SSH'
1298063168.409852/1298063168.410547 [logging] Created new filter 'default' for stream 'SSH::LOG_SSH'
1298063168.409852/1298063168.410564 [logging] writer : Ascii
1298063168.409852/1298063168.410574 [logging] path : ssh_log_ssh
1298063168.409852/1298063168.410584 [logging] path_func : not set
1298063168.409852/1298063168.410594 [logging] event : not set
1298063168.409852/1298063168.410604 [logging] pred : not set
1298063168.409852/1298063168.410614 [logging] field t: time
1298063168.409852/1298063168.410625 [logging] field id.orig_h: addr
1298063168.409852/1298063168.410635 [logging] field id.orig_p: port
1298063168.409852/1298063168.410645 [logging] field id.resp_h: addr
1298063168.409852/1298063168.410655 [logging] field id.resp_p: port
1298063168.409852/1298063168.410665 [logging] field status: string
1298063168.409852/1298063168.410675 [logging] field country: string
1298063168.409852/1298063168.410817 [logging] Wrote record to filter 'default' on stream 'SSH::LOG_SSH'
1298063168.409852/1298063168.410865 [logging] Wrote record to filter 'default' on stream 'SSH::LOG_SSH'
1298063168.409852/1298063168.410906 [logging] Wrote record to filter 'default' on stream 'SSH::LOG_SSH'
1298063168.409852/1298063168.410945 [logging] Wrote record to filter 'default' on stream 'SSH::LOG_SSH'
1298063168.409852/1298063168.411044 [logging] Wrote record to filter 'default' on stream 'SSH::LOG_SSH
> cat ssh_log_ssh.log
1298063168.40985 1.2.3.4 66770 2.3.4.5 65616 success unknown
1298063168.40985 1.2.3.4 66770 2.3.4.5 65616 failure US
1298063168.40985 1.2.3.4 66770 2.3.4.5 65616 failure UK
1298063168.40985 1.2.3.4 66770 2.3.4.5 65616 success BR
1298063168.40985 1.2.3.4 66770 2.3.4.5 65616 failure MX
* remotes/origin/topic/bro_dns_fake_message:
Reworded BRO_DNS_FAKE message.
Added a BRO_DNS_FAKE message to the help output.
Changed to show on/off rather than the variable's content.
- Asynchronous DNS resolver support assumed by default
- HAVE_ASYNC_DNS, HAVE_NB_DNS, and USE_NB_DNS preprocessor definitions
removed
- In the process, I added FindBIND and CheckNameserCompat CMake modules
