This is to avoid ambiguity between compressed hex notation and
module namespacing, both which use "::". E.g.: "aaaa::bbbb" could
be an identifier or an IPv6 address, but "[aaaa::bbbb]" is now
clearly the address.
Also added IPv6 mixed notation to allow an IPv4 dotted-decimal
address to be specified in the lower 32-bits.
most stuff is inplace, logging framework needs a few changes merged before continuing here...
Conflicts:
src/CMakeLists.txt
src/LogMgr.h
src/logging/Manager.cc
src/main.cc
- Reorganize top-level 'doc' Makefile target so submodules can easily
add their own doc-generating routines to it. e.g. the Bro project
makes a placeholder 'doc' target, then adds 'restdoc', 'sphinxdoc';
later Broccoli can add it's own target as a dependency for generating
API docs.
- Fixed generated docs for BIFs not being organized under a base/
subdirectory like the original source files.
- Fixed documentation style for function parameters not applying to
functions declared as record fields.
- Misc. script documentation tweaks to address warnings given by Sphinx.
Any added prefixes are now used *after* all input files have been
parsed to look for a prefixed, flattened version of the input file
somewhere in BROPATH and, if found, load it.
For example, if "lcl" is in @prefixes, and site.bro is loaded, then
a file named "lcl.site.bro" that's in BROPATH would end up being
automatically loaded as well. Packages work similarly, e.g. loading
"protocols/http" means a file named "lcl.protocols.http.bro" in BROPATH
gets loaded automatically.
For example a script can do "@load ./foo" to load a script named
foo.bro that lives in the same directory or "@load ../bar" to load
a script named bar.bro in the parent directory, even if those
directories are not contained in BROPATH.
Also removing the -l command-line option as that can now be done at
the script-level.
A couple tests fail now that use -l. Leaving that until we have
script-level replacement.
* origin/topic/robin/reporting:
Syslog BiF now goes through the reporter as well.
Avoiding infinite loops when an error message handlers triggers errors itself.
Renaming the Logger to Reporter.
Overhauling the internal reporting of messages to the user.
Updating a bunch of tests/baselines as well.
Conflicts:
aux/broccoli
policy.old/alarm.bro
policy/all.bro
policy/bro.init
policy/frameworks/notice/weird.bro
policy/notice.bro
src/SSL-binpac.cc
src/bro.bif
src/main.cc
Added an arg to the search_for_files() util function that can return
the subpath of BROPATH's policy/ dir in which the loaded file is found.
This subpath is then used in both the the reST file's document title
(so that script's named e.g. "base.bro" actually have some context) and
in figuring out how to interlink with other generated docs of other
scripts that are found in @load directives.
I still need to overhaul things so the loading of "packages" is
documented in a meaningful way and that the CMake targets are able
to generate indexes for packages.
Caused by changes made while working on the way loaded scripts are tracked
(commit 9de6e9170c to master branch) that
didn't take into account an unmerged commit on the policy-scripts-new branch
(56a946568a).
(re-added warning comments that were removed in the former commit since they
still seem relevant now)
The Logger class is now in charge of reporting all errors, warnings,
informational messages, weirds, and syslogs. All other components
route their messages through the global bro_logger singleton.
The Logger class comes with these reporting methods:
void Message(const char* fmt, ...);
void Warning(const char* fmt, ...);
void Error(const char* fmt, ...);
void FatalError(const char* fmt, ...); // Terminate Bro.
void Weird(const char* name);
[ .. some more Weird() variants ... ]
void Syslog(const char* fmt, ...);
void InternalWarning(const char* fmt, ...);
void InternalError(const char* fmt, ...); // Terminates Bro.
See Logger.h for more information on these.
Generally, the reporting now works as follows:
- All non-fatal message are reported in one of two ways:
(1) At startup (i.e., before we start processing packets),
they are logged to stderr.
(2) During processing, they turn into events:
event log_message%(msg: string, location: string%);
event log_warning%(msg: string, location: string%);
event log_error%(msg: string, location: string%);
The script level can then handle them as desired.
If we don't have an event handler, we fall back to
reporting on stderr.
- All fatal errors are logged to stderr and Bro terminates
immediately.
- Syslog(msg) directly syslogs, but doesn't do anything else.
The three main types of messages can also be generated on the
scripting layer via new Log::* bifs:
Log::error(msg: string);
Log::warning(msg: string);
Log::message(msg: string);
These pass through the bro_logger as well and thus are handled in the
same way. Their output includes location information.
More changes:
- Removed the alarm statement and the alarm_hook event.
- Adapted lots of locations to use the bro_logger, including some
of the messages that were previously either just written to
stdout, or even funneled through the alarm mechanism.
- No distinction anymore between Error() and RunTime(). There's
now only one class of errors; the line was quite blurred already
anyway.
- util.h: all the error()/warn()/message()/run_time()/pinpoint()
functions are gone. Use the bro_logger instead now.
- Script errors are formatted a bit differently due to the
changes. What I've seen so far looks ok to me, but let me know
if there's something odd.
Notes:
- The default handlers for the new log_* events are just dummy
implementations for now since we need to integrate all this into
the new scripts anyway.
- I'm not too happy with the names of the Logger class and its
instance bro_logger. We now have a LogMgr as well, which makes
this all a bit confusing. But I didn't have a good idea for
better names so I stuck with them for now.
Perhaps we should merge Logger and LogMgr?
This change primarily improves the way Bro detects and prevents
the same script from being loaded twice. It now compares inode
numbers instead of path names.
With a directory "foo" somewhere in BROPATH, "@load foo" now checks if
there's a file "foo/__load__.bro". If so, it reads that file in. (If
not, Bro reports the same error as before, complaining that it can't
read a directory).
The output reST filename now reflects the subdir information (by flattening
'/' path separators into the '^' character). This is to prevent file name
conflicts when generated reST docs, but during the CMake 'doc' target to
build HTML docs, everything gets unflattened.
Generated script reST documentation is also written out at this time
instead of at the end of lexical scanning.
The persistence serializer will no longer write out Bro's state to the
.state directory when in doc mode.
For a bro script that is documented like:
## ...
## bar: ...
## Returns: ...
global foo(bar: string): string;
the generated reST documentation will automatically substitute meaningful
markup (reST field lists) for the parameter and return type comments
The &log keyword now operates as discussed:
- When associated with individual record fields, it defines them
as being logged.
- When associated with a complete record type, it defines all fields
to be logged.
- When associated with a record extension, it defines all added
fields to be logged.
Note that for nested record types, the inner fields must likewise
be declared with &log. Consequently, conn_id is now declared with
&log in bro.init.
Vectors are now allowed to be logged and will be recorded as an
ordered set of items.
This functionality is better done manually by the script writer
embedding reST into the script summary section (##! comments).
This allows flexibility in choosing between different methods
to convey the same information (e.g. ":Author: <author>" or
the ".. codeauthor:: <author>" directive that Sphinx
configurations can recognize).
The documentation framework now sees "##Text" and "## Text" as
equivalent documentation comments. This prevents unintentional
indentation in the generated reST as a result of the later style, but
still allows embedded reST markup that relies on indentation of more
than two spaces to work as expected.
Comments associated with record fields and enums values are able
to span multiple "##"-stylized comments, allowing for more robust
reST markup to be embedded.
The documentation framework now tracks record fields through
a new CommentedTypeDecl subclass of TypeDecl that the parser constructs
in parallel with the real TypeDecl.
And (to be consistent with current conventions for reST documentation)
update places in the auto-documentation-generation framework
where tabs were used in the generated reST.
The scanner can now be told to start/stop producing new token types that
assist in documenting record field types (and eventually enums also).
TOK_DOC:
Produced on "##" style comments; documents the field that follows.
TOK_POST_DOC:
Produced on "##<" style comments; documents the previous field.
