Commit graph

5909 commits

Author SHA1 Message Date
Jon Siwek
c2f0332b5f Removing lexical scanner recognition of "## Author:" comments.
This functionality is better done manually by the script writer
embedding reST into the script summary section (##! comments).
This allows flexibility in choosing between different methods
to convey the same information (e.g. ":Author: <author>" or
the ".. codeauthor:: <author>" directive that Sphinx
configurations can recognize).
2011-03-24 09:51:47 -05:00
Jon Siwek
8b79971e21 Fix auto-generated reST :doc: references to strip .bro file suffixes.
It's allowed for a script to "@load example.bro", but Sphinx doesn't want
that file extension for the purposes of generating cross-referencing links
to other documentation.
2011-03-23 16:59:41 -05:00
Jon Siwek
037a6906af Canonicalize "##" and "## " stylized comments, revise record/enum documentation tracking
The documentation framework now sees "##Text" and "## Text" as
equivalent documentation comments.  This prevents unintentional
indentation in the generated reST as a result of the later style, but
still allows embedded reST markup that relies on indentation of more
than two spaces to work as expected.

Comments associated with record fields and enums values are able
to span multiple "##"-stylized comments, allowing for more robust
reST markup to be embedded.

The documentation framework now tracks record fields through
a new CommentedTypeDecl subclass of TypeDecl that the parser constructs
in parallel with the real TypeDecl.
2011-03-23 16:08:18 -05:00
Jon Siwek
bbe7c98ab3 Major reformatting of auto-generated reST documentation.
Introduces reST directives and roles in a "bro" domain that Sphinx
will be taught to recognize.
2011-03-22 16:05:59 -05:00
Robin Sommer
05f2104fec Merge branch 'topic/robin/record-coercion' into topic/robin/logging-internals
* topic/robin/record-coercion:
  Fixing a bug with nested record ctors.
  Enabling automatic coercion from record type A to be B as long as A has all the types that B has.

Conflicts:
	src/Expr.cc
2011-03-21 17:24:30 -07:00
Jon Siwek
384fa03c26 Revising notice, port analysis, packet filter, auto-generated documentation.
They're now all optional sections -- if a given bro script doesn't use the
functionality, then the documentation doesn't mention anything about it.
2011-03-21 15:08:39 -05:00
Jon Siwek
e0a77cb794 Auto-doc framework now handles multiple script authors more conveniently. 2011-03-21 14:03:11 -05:00
Jon Siwek
5fbcde7344 Teach Desc class how to (optionally) use spaces for indentation.
And (to be consistent with current conventions for reST documentation)
update places in the auto-documentation-generation framework
where tabs were used in the generated reST.
2011-03-21 13:49:53 -05:00
Jon Siwek
e491caeeb7 Add note to auto-generated reST about not editing it directly 2011-03-21 12:49:54 -05:00
Robin Sommer
7526058071 Fixing a bug with nested record ctors.
If a record field was initialized with another record ctor, there was
no record type coercion in place in case of a type mismatch.
2011-03-17 17:19:46 -07:00
Robin Sommer
d84d4b8a57 Moving uid from conn_id to connection, and making output determistic
if a hash seed is given.
2011-03-16 14:25:49 -07:00
Robin Sommer
881071cc99 Extending conn_id with a globally unique identifiers. 2011-03-15 20:42:56 -07:00
Jon Siwek
9e13d15f29 Tabifying BroDoc* sources to make consistent with general style. 2011-03-15 16:48:51 -05:00
Jon Siwek
dbf2b9996e Revising BroDoc*.h internal/api documentation. 2011-03-15 15:38:43 -05:00
Jon Siwek
f67c0892e5 Revise enum documentation autogeneration.
This adds a new subclass of EnumType, CommentedEnumType, and removes
any previous changes to EnumType that were done to support the
autodoc framework.

Dummy CommentedEnumType and ID's are constructed in parallel with the
real EnumType ID's during parsing and passed on to the autodoc framework.

This allows the generated documentation to track enum redefs, with
a special case being the "Notice" enum type.
2011-03-15 14:51:50 -05:00
Robin Sommer
8eb241fde7 Removing the EXPIRE_DFA_STATES code. 2011-03-14 18:55:40 -07:00
Robin Sommer
57fe369c4e Removing active mapping. 2011-03-14 18:31:17 -07:00
Jon Siwek
b1dc5d3a1c Update to auto-generated documentation format.
Comments are now clearly marked.

Enums are now documented in order of their integral value, not
their identifier string.
2011-03-14 14:49:58 -05:00
Jon Siwek
f47c8e4d93 Changes to parser to allow enum type documentation comments.
"##" style comments before identifiers and "##<" style after identifiers
in the body of an enum type declaration will now show up in the
auto-generated reST documentation.
2011-03-14 14:13:52 -05:00
Jon Siwek
c4ca6f098c Changes to scanner and parser to allow record field comments.
The scanner can now be told to start/stop producing new token types that
assist in documenting record field types (and eventually enums also).

TOK_DOC:
    Produced on "##" style comments; documents the field that follows.

TOK_POST_DOC:
    Produced on "##<" style comments; documents the previous field.
2011-03-14 13:10:49 -05:00
Jon Siwek
4b0eb8127d Changes to add comments per enum or record type field. 2011-03-14 11:50:46 -05:00
Robin Sommer
45ebfbb2b8 Bug fixes.
- Fixing a crash with an invalid pointer.

- Fixing a namespacing problem with is_ftp_data_conn() and check_relay_3().

- Fixing the do-we-have-an-event-handler-defined check.

Standard test-suite passes.

Seth, I think you can give it a try now ...
2011-03-10 18:18:01 -08:00
Jon Siwek
15fd5297a3 Fixes for declared type ID tracking & adding DescribeReST()'s
Changed BroType to track a char* instead of an ID* that represents
the declared type's identifier.  It was also necessary to serialize
this information or else it can be lost (e.g. FieldDecl's in RecordType
always seem to get serialized at some point).

DescribeReST() functions added to many classes to get the output
closer to being reST compatible; still needs tweaking for Sphinx
(reST->HTML) compatibility.
2011-03-10 16:22:45 -06:00
Robin Sommer
d54c705898 Adding new environment variable BRO_SEED_FILE to set the seed file for
the random number generator.

This works like the corresponding command line option but is more
convinient when writing tests as it can be set in btest.cfg.
2011-03-09 19:42:24 -08:00
Robin Sommer
de227b8d88 When creating a new stream, we check now that all log field have
supported types.

Also not reporting a run-time error anymore when logging to a stream
that hasn't been created; just fail silently as this may happen due to
other earlier errors.
2011-03-09 19:42:24 -08:00
Robin Sommer
89e8ea7353 Enums are now logged with their ID names, not anymore with their
numerical values.
2011-03-09 18:06:50 -08:00
Robin Sommer
c6e3174bc8 The logging systems now supports fields of type set[<atomic_type>]. 2011-03-09 18:01:41 -08:00
Robin Sommer
52c54859b6 Bugfix: the header line was ending with a separator.
Test baselines needed adaption as well.
2011-03-09 16:55:29 -08:00
Robin Sommer
b69ecff3ee More options for the ASCII writer.
# The prefix for the header line if included.
	const header_prefix = "# " &redef;

	# The string to use for empty string fields.
	const empty_field = "" &redef;

	# The string to use for an unset optional field.
	const unset_field = "-" &redef;
2011-03-09 16:52:46 -08:00
Robin Sommer
cb9e0a5d5a If a field value contains the separator, that is now escape with hex
characters.
2011-03-09 16:26:11 -08:00
Jon Siwek
287a3a3cb8 Changes to parser to pass along identifiers to doc framework. 2011-03-09 15:47:58 -06:00
Jon Siwek
1310f2387c Changing BroDocObj wrapper class to wrap ID instead of BroObj. 2011-03-09 15:10:05 -06:00
Jon Siwek
1c962cbb89 Changes to make declared types track their identifier.
Only occurs when in documentation mode.
2011-03-09 14:53:22 -06:00
Robin Sommer
c6d20dbfdf Adding a few options to the ASCII writer.
module LogAscii;

export {
	# Output everything to stdout rather than into files. This is primarily
	# for testing purposes.
	const output_to_stdout = F &redef;

	# The separator between fields.
	const separator = "\t" &redef;

	# True to include a header line with column names.
	const include_header = T &redef;
}
2011-03-08 21:44:46 -08:00
Robin Sommer
26eab74ecc The ASCII writer can now deal with /dev/* paths.
It will not longer try to add a ".log" extension.
2011-03-08 17:59:05 -08:00
Robin Sommer
4b7c5905f1 New functions Log::disable_stream() and Log::enable_stream().
When disabled, all outout to a stream will be ignored (but no error
raised).
2011-03-08 17:54:11 -08:00
Robin Sommer
5d8b9f2e74 New function Log::get_filter() that allows to retrieve previously
installed filter by name.

This filter can then be modified and reinstalled via add_filter(),
which will replace the old one with the same name.
2011-03-08 17:31:46 -08:00
Robin Sommer
df54cc6e78 New table Log::rotation_control that enables to control rotation
for individual files, overriding defaults.

The interface isn't the greatest but the best I can come up with right
now.
2011-03-08 16:30:53 -08:00
Robin Sommer
eb736a34b3 '[]' is now a valid record ctor. 2011-03-08 16:28:59 -08:00
Robin Sommer
83bd6584ee Enabling record coercion for a table's &default attribute. 2011-03-08 16:27:07 -08:00
Robin Sommer
d6cef16f77 Rotation support.
This follows rather closely how rotation currently works in
rotate-logs.bro. logging.bro now defines:

        # Default rotation interval; zero disables rotation.
        const default_rotation_interval = 0secs &redef;

        # Default naming suffix format.
        const default_rotation_date_format = "%y-%m-%d_%H.%M.%S" &redef;

        # Default postprocessor for writers outputting into files.
        const default_rotation_postprocessor = "" &redef;

        # Default function to construct the name of the rotated file.
        # The default implementation includes
        # default_rotation_date_format into the file name.
        global default_rotation_path_func: function(info: RotationInfo) : string &redef;

Writer support for rotation is optional, usually it will only make
sense for file-based writers.

TODO: Currently, there's no way to customize rotation on a per file
basis, there are only the global defaults as described above.
Individual customization is coming next.
2011-03-06 19:32:44 -08:00
Jon Siwek
1bad6e3a95 Changes to auto-document dpd_config (port analysis) script settings. 2011-03-05 16:11:26 -06:00
Jon Siwek
0d7ffe998f Debugging output from doc-framework now only in debug mode.
Also a minor format fix for documenting empty string lists.
2011-03-05 14:22:06 -06:00
Robin Sommer
3f413a2539 Remote logging for the new logging framework.
It works with a simple example, but that's as much testing as it has
seen so far.

Remote::Destination has a new attribute "request_logs: bool"
indicating whether we are interested in the peer's log. Default is
false. If true, Bro will send an explicit "I want your logs" message
over to the other side, which will then start sending log records
back.

When such log records are received, they will be recorded exactly in
the same way as on the remote side, i.e., same fields/writer/path. All
filtering is already performed on the remote side.

Log::Filter has two new attributes, "log_local: bool" and
"log_remote: bool" (both true by default). If log_local is false, this
filter will not record anything locally but still process everything
normally otherwise and potentially forward to remote. If log_remote is
false, this filter will never send anything to remote even if a peer
has requested logs. (Note that with the defaults, requesting logs will
mean getting everything.)

Note that with log forwarding, *both* sides must create the
Filter::Stream. If the remote sends log records for a specific stream,
but the local side hasn't created it, the data will be discarded.
Filtes on the other hand shouldn't created locally; and if they are,
they are ignored for records received from remote).
2011-03-03 16:45:29 -08:00
Jon Siwek
f7d031ed60 Merge remote-tracking branch 'origin/master' into topic/jsiwek/doc-framework 2011-03-02 14:54:06 -06:00
Robin Sommer
c355f5d1fa Adding options to enable/disable local and remote logging.
Only the local option has an effect right now.

Also moving Log::default_writer out of the bif into logging.bro.
2011-03-01 17:13:50 -08:00
Robin Sommer
d673c8c64c Adding a new RecordVal method LookupWithDefault().
The method honors a potential &default attribute, and refs the Val it
returns.
2011-02-28 22:20:29 -08:00
Robin Sommer
ba2c6f6139 Internal refactoring to provide injection points for remotely received
log records.

Also added some additional type-safety check to later make sure that
what a peer sends is actually of the expected format.
2011-02-28 21:52:56 -08:00
Robin Sommer
e6edc52d54 Moving the map of writers from the filters to the streams.
This is in preparation for remote logging.
2011-02-28 18:24:19 -08:00
Robin Sommer
ab15437339 Working on the logging API exposed to scripts.
- Moving all functions into the Log::* namespace, using the recent
  bifcl updates. Moved logging-specific stuff to logging.bif.

- Log::create_stream() now takes a record Log::Stream as its second
  argument, which specifies columns and (optionally) the event.

- All the internal BiFs are now called "Log::__<something>", with
  script-level wrappers "Log::<something>". That first allows to add
  additional code at the script-level, and second makes things better
  comprehendible as now all relevant functionality is collected (and
  later documetned) in policy/logging.bro.

- New function Log::flush(id), which does the obvious assuming the
  writer supports it.

- add_default_filter() is now called implicitly with every
  create_stream(). Seems that we usually want that functionality, and
  when not, remove_default_filter() gets rid of it.

- The namespace of a stream's ID is now used as the default "path"
  (e.g., if the namespace is SSH, the default log file is "ssh.log").

- Updated policy/test-logging.bro as well as the btest tests according
  to these changes.
2011-02-27 15:09:37 -08:00