Log rotation is now controlled directly through Filter records.
Also addressed a TODO in the default_path_func regarding the
LogMgr::AddFilter function generating internal filter path
suggestions/fallbacks. Now, if the user doesn't explicitly set a filter
path, the filter's path will be the result of the first call to
default_path_func (happens during the first write to the log). And in
that case the path suggestion argument to the path_func is an empty
string.
# The prefix for the header line if included.
const header_prefix = "# " &redef;
# The string to use for empty string fields.
const empty_field = "" &redef;
# The string to use for an unset optional field.
const unset_field = "-" &redef;
module LogAscii;
export {
# Output everything to stdout rather than into files. This is primarily
# for testing purposes.
const output_to_stdout = F &redef;
# The separator between fields.
const separator = "\t" &redef;
# True to include a header line with column names.
const include_header = T &redef;
}
This follows rather closely how rotation currently works in
rotate-logs.bro. logging.bro now defines:
# Default rotation interval; zero disables rotation.
const default_rotation_interval = 0secs &redef;
# Default naming suffix format.
const default_rotation_date_format = "%y-%m-%d_%H.%M.%S" &redef;
# Default postprocessor for writers outputting into files.
const default_rotation_postprocessor = "" &redef;
# Default function to construct the name of the rotated file.
# The default implementation includes
# default_rotation_date_format into the file name.
global default_rotation_path_func: function(info: RotationInfo) : string &redef;
Writer support for rotation is optional, usually it will only make
sense for file-based writers.
TODO: Currently, there's no way to customize rotation on a per file
basis, there are only the global defaults as described above.
Individual customization is coming next.
- Moving all functions into the Log::* namespace, using the recent
bifcl updates. Moved logging-specific stuff to logging.bif.
- Log::create_stream() now takes a record Log::Stream as its second
argument, which specifies columns and (optionally) the event.
- All the internal BiFs are now called "Log::__<something>", with
script-level wrappers "Log::<something>". That first allows to add
additional code at the script-level, and second makes things better
comprehendible as now all relevant functionality is collected (and
later documetned) in policy/logging.bro.
- New function Log::flush(id), which does the obvious assuming the
writer supports it.
- add_default_filter() is now called implicitly with every
create_stream(). Seems that we usually want that functionality, and
when not, remove_default_filter() gets rid of it.
- The namespace of a stream's ID is now used as the default "path"
(e.g., if the namespace is SSH, the default log file is "ssh.log").
- Updated policy/test-logging.bro as well as the btest tests according
to these changes.
This pretty much follows the proposal on the projects page.
It includes:
- A new LogMgr, maintaining the set of writers.
- The abstract LogWriter API.
- An initial implementation in the form of LogWriterAscii
producing tab-separated columns.
Note that things are only partially working right now, things are
subject to change, and it's all not much tested at all. That's why I'm
creating separate branch for now.
Example:
bro -B logging test-logging && cat debug.log
1298063168.409852/1298063168.410368 [logging] Created new logging stream 'SSH::LOG_SSH'
1298063168.409852/1298063168.410547 [logging] Created new filter 'default' for stream 'SSH::LOG_SSH'
1298063168.409852/1298063168.410564 [logging] writer : Ascii
1298063168.409852/1298063168.410574 [logging] path : ssh_log_ssh
1298063168.409852/1298063168.410584 [logging] path_func : not set
1298063168.409852/1298063168.410594 [logging] event : not set
1298063168.409852/1298063168.410604 [logging] pred : not set
1298063168.409852/1298063168.410614 [logging] field t: time
1298063168.409852/1298063168.410625 [logging] field id.orig_h: addr
1298063168.409852/1298063168.410635 [logging] field id.orig_p: port
1298063168.409852/1298063168.410645 [logging] field id.resp_h: addr
1298063168.409852/1298063168.410655 [logging] field id.resp_p: port
1298063168.409852/1298063168.410665 [logging] field status: string
1298063168.409852/1298063168.410675 [logging] field country: string
1298063168.409852/1298063168.410817 [logging] Wrote record to filter 'default' on stream 'SSH::LOG_SSH'
1298063168.409852/1298063168.410865 [logging] Wrote record to filter 'default' on stream 'SSH::LOG_SSH'
1298063168.409852/1298063168.410906 [logging] Wrote record to filter 'default' on stream 'SSH::LOG_SSH'
1298063168.409852/1298063168.410945 [logging] Wrote record to filter 'default' on stream 'SSH::LOG_SSH'
1298063168.409852/1298063168.411044 [logging] Wrote record to filter 'default' on stream 'SSH::LOG_SSH
> cat ssh_log_ssh.log
1298063168.40985 1.2.3.4 66770 2.3.4.5 65616 success unknown
1298063168.40985 1.2.3.4 66770 2.3.4.5 65616 failure US
1298063168.40985 1.2.3.4 66770 2.3.4.5 65616 failure UK
1298063168.40985 1.2.3.4 66770 2.3.4.5 65616 success BR
1298063168.40985 1.2.3.4 66770 2.3.4.5 65616 failure MX
