Commit graph

311 commits

Author SHA1 Message Date
Robin Sommer
18bcac7256 Removing everything related to trace rewriting.
(I wasn't too careful in ensuring that I catch everything in the
scripts; Seth is working on those anyway.)
2011-03-16 17:51:23 -07:00
Robin Sommer
d84d4b8a57 Moving uid from conn_id to connection, and making output determistic
if a hash seed is given.
2011-03-16 14:25:49 -07:00
Robin Sommer
881071cc99 Extending conn_id with a globally unique identifiers. 2011-03-15 20:42:56 -07:00
Robin Sommer
0f854315e9 New test. 2011-03-10 18:18:58 -08:00
Robin Sommer
45ebfbb2b8 Bug fixes.
- Fixing a crash with an invalid pointer.

- Fixing a namespacing problem with is_ftp_data_conn() and check_relay_3().

- Fixing the do-we-have-an-event-handler-defined check.

Standard test-suite passes.

Seth, I think you can give it a try now ...
2011-03-10 18:18:01 -08:00
Robin Sommer
871561939b Updating baselines. 2011-03-09 19:42:24 -08:00
Robin Sommer
170a8bd403 Setting random number generator seed in btest.cfg. 2011-03-09 19:42:24 -08:00
Robin Sommer
d54c705898 Adding new environment variable BRO_SEED_FILE to set the seed file for
the random number generator.

This works like the corresponding command line option but is more
convinient when writing tests as it can be set in btest.cfg.
2011-03-09 19:42:24 -08:00
Robin Sommer
de227b8d88 When creating a new stream, we check now that all log field have
supported types.

Also not reporting a run-time error anymore when logging to a stream
that hasn't been created; just fail silently as this may happen due to
other earlier errors.
2011-03-09 19:42:24 -08:00
Robin Sommer
5beee9e45e Updating TODO.
Just one to go. :)
2011-03-09 18:08:17 -08:00
Robin Sommer
89e8ea7353 Enums are now logged with their ID names, not anymore with their
numerical values.
2011-03-09 18:06:50 -08:00
Robin Sommer
c6e3174bc8 The logging systems now supports fields of type set[<atomic_type>]. 2011-03-09 18:01:41 -08:00
Robin Sommer
52c54859b6 Bugfix: the header line was ending with a separator.
Test baselines needed adaption as well.
2011-03-09 16:55:29 -08:00
Robin Sommer
b69ecff3ee More options for the ASCII writer.
# The prefix for the header line if included.
	const header_prefix = "# " &redef;

	# The string to use for empty string fields.
	const empty_field = "" &redef;

	# The string to use for an unset optional field.
	const unset_field = "-" &redef;
2011-03-09 16:52:46 -08:00
Robin Sommer
cb9e0a5d5a If a field value contains the separator, that is now escape with hex
characters.
2011-03-09 16:26:11 -08:00
Robin Sommer
c6d20dbfdf Adding a few options to the ASCII writer.
module LogAscii;

export {
	# Output everything to stdout rather than into files. This is primarily
	# for testing purposes.
	const output_to_stdout = F &redef;

	# The separator between fields.
	const separator = "\t" &redef;

	# True to include a header line with column names.
	const include_header = T &redef;
}
2011-03-08 21:44:46 -08:00
Robin Sommer
26eab74ecc The ASCII writer can now deal with /dev/* paths.
It will not longer try to add a ".log" extension.
2011-03-08 17:59:05 -08:00
Robin Sommer
4b7c5905f1 New functions Log::disable_stream() and Log::enable_stream().
When disabled, all outout to a stream will be ignored (but no error
raised).
2011-03-08 17:54:11 -08:00
Robin Sommer
88d114053c Removing unnessary namespace qualifiers in logging.bro. 2011-03-08 17:32:21 -08:00
Robin Sommer
5d8b9f2e74 New function Log::get_filter() that allows to retrieve previously
installed filter by name.

This filter can then be modified and reinstalled via add_filter(),
which will replace the old one with the same name.
2011-03-08 17:31:46 -08:00
Robin Sommer
b8ee425e0d Adding test for per-file rotation. 2011-03-08 16:33:06 -08:00
Robin Sommer
df54cc6e78 New table Log::rotation_control that enables to control rotation
for individual files, overriding defaults.

The interface isn't the greatest but the best I can come up with right
now.
2011-03-08 16:30:53 -08:00
Robin Sommer
eb736a34b3 '[]' is now a valid record ctor. 2011-03-08 16:28:59 -08:00
Robin Sommer
83bd6584ee Enabling record coercion for a table's &default attribute. 2011-03-08 16:27:07 -08:00
Robin Sommer
51f9cb2ac5 Updating test baseline after fixing an older bug in previous commit. 2011-03-08 16:26:31 -08:00
Robin Sommer
d6cef16f77 Rotation support.
This follows rather closely how rotation currently works in
rotate-logs.bro. logging.bro now defines:

        # Default rotation interval; zero disables rotation.
        const default_rotation_interval = 0secs &redef;

        # Default naming suffix format.
        const default_rotation_date_format = "%y-%m-%d_%H.%M.%S" &redef;

        # Default postprocessor for writers outputting into files.
        const default_rotation_postprocessor = "" &redef;

        # Default function to construct the name of the rotated file.
        # The default implementation includes
        # default_rotation_date_format into the file name.
        global default_rotation_path_func: function(info: RotationInfo) : string &redef;

Writer support for rotation is optional, usually it will only make
sense for file-based writers.

TODO: Currently, there's no way to customize rotation on a per file
basis, there are only the global defaults as described above.
Individual customization is coming next.
2011-03-06 19:32:44 -08:00
Robin Sommer
90af0d06c3 A first test for remote logging.
Needs the updated btest, which has two new scripts for running
processes in the background.
2011-03-03 21:47:08 -08:00
Robin Sommer
3f413a2539 Remote logging for the new logging framework.
It works with a simple example, but that's as much testing as it has
seen so far.

Remote::Destination has a new attribute "request_logs: bool"
indicating whether we are interested in the peer's log. Default is
false. If true, Bro will send an explicit "I want your logs" message
over to the other side, which will then start sending log records
back.

When such log records are received, they will be recorded exactly in
the same way as on the remote side, i.e., same fields/writer/path. All
filtering is already performed on the remote side.

Log::Filter has two new attributes, "log_local: bool" and
"log_remote: bool" (both true by default). If log_local is false, this
filter will not record anything locally but still process everything
normally otherwise and potentially forward to remote. If log_remote is
false, this filter will never send anything to remote even if a peer
has requested logs. (Note that with the defaults, requesting logs will
mean getting everything.)

Note that with log forwarding, *both* sides must create the
Filter::Stream. If the remote sends log records for a specific stream,
but the local side hasn't created it, the data will be discarded.
Filtes on the other hand shouldn't created locally; and if they are,
they are ignored for records received from remote).
2011-03-03 16:45:29 -08:00
Robin Sommer
c355f5d1fa Adding options to enable/disable local and remote logging.
Only the local option has an effect right now.

Also moving Log::default_writer out of the bif into logging.bro.
2011-03-01 17:13:50 -08:00
Robin Sommer
d673c8c64c Adding a new RecordVal method LookupWithDefault().
The method honors a potential &default attribute, and refs the Val it
returns.
2011-02-28 22:20:29 -08:00
Robin Sommer
ba2c6f6139 Internal refactoring to provide injection points for remotely received
log records.

Also added some additional type-safety check to later make sure that
what a peer sends is actually of the expected format.
2011-02-28 21:52:56 -08:00
Robin Sommer
e6edc52d54 Moving the map of writers from the filters to the streams.
This is in preparation for remote logging.
2011-02-28 18:24:19 -08:00
Robin Sommer
ab15437339 Working on the logging API exposed to scripts.
- Moving all functions into the Log::* namespace, using the recent
  bifcl updates. Moved logging-specific stuff to logging.bif.

- Log::create_stream() now takes a record Log::Stream as its second
  argument, which specifies columns and (optionally) the event.

- All the internal BiFs are now called "Log::__<something>", with
  script-level wrappers "Log::<something>". That first allows to add
  additional code at the script-level, and second makes things better
  comprehendible as now all relevant functionality is collected (and
  later documetned) in policy/logging.bro.

- New function Log::flush(id), which does the obvious assuming the
  writer supports it.

- add_default_filter() is now called implicitly with every
  create_stream(). Seems that we usually want that functionality, and
  when not, remove_default_filter() gets rid of it.

- The namespace of a stream's ID is now used as the default "path"
  (e.g., if the namespace is SSH, the default log file is "ssh.log").

- Updated policy/test-logging.bro as well as the btest tests according
  to these changes.
2011-02-27 15:09:37 -08:00
Robin Sommer
e3b2748fa6 Merge remote branch 'origin/master' into topic/robin/logging-internals
Conflicts:
	policy/bro.init
	src/builtin-func.l
2011-02-25 16:11:03 -08:00
Robin Sommer
226eeb9729 Updating submodule(s). 2011-02-25 15:49:08 -08:00
Robin Sommer
3527daea5b Updating submodule(s). 2011-02-25 15:48:31 -08:00
Robin Sommer
12139e9faf Merge remote branch 'origin/topic/gregor/bif-tuning'
* origin/topic/gregor/bif-tuning:
  Refactor: BifTypePtr --> BifType
  Bif const: make sure const is indeed a constant.
  Support any type in bif const declaration.
  Tweak for bifcl
  Fix to bifcl wrt namespaces.
  Enable declaration of set, vector, and table types in bifs.
  Moving type declarations into its own bif file
  Support namespaces / modules in bif. Checkpoint.
  Support namespaces / modules in bif. Checkpoint.
  Remove leftovers from removing "declare enum" from bifcl
  Use namespaces for NetVar type pointers.
  Remove unused and unnecessary "declare enum" from bifcl
  Bif: add record type declaration.
  Minor tweaks for bif language.
  enum type: don't allow mixing of explicit value and auto-increment.
  Add support for enum with explicit enumerator values.

Closes #403.
2011-02-25 15:41:56 -08:00
Robin Sommer
a3e1dd5261 Merge remote branch 'origin/topic/gregor/reassmbler-hotfix'
Closes #404.

* origin/topic/gregor/reassmbler-hotfix:
  TCP Reassembler hotfix for conns > 2GB.
2011-02-25 10:54:43 -08:00
Robin Sommer
9b94218c63 Merge remote branch 'origin/topic/gregor/fix-val-64bit'
* origin/topic/gregor/fix-val-64bit:
  Fixing endianess error in XDR when data is not 4-byte aligned.
  Fix for Val constructor with new int64 typedefs.
  New fix for OS X 10.5 compile error wrt llabs()
  Revert "Fix for OS X 10.5 compile error wrt llabs()"
2011-02-25 10:41:51 -08:00
Gregor Maier
6af01a205f Merge remote branch 'origin/master' into topic/gregor/reassmbler-hotfix 2011-02-22 16:58:05 -08:00
Gregor Maier
e310119ffa Refactor: BifTypePtr --> BifType
As per our mail discussion renaming BifTypePtr namespace to
BifType.
2011-02-22 14:52:21 -08:00
Robin Sommer
f02e465c92 Tests for the logging functionality, including a basic btest
configuration.
2011-02-21 18:29:41 -08:00
Robin Sommer
ac936feb95 Consistent error handling. 2011-02-21 18:18:12 -08:00
Robin Sommer
cf148c8a25 New bif log_set_buf() to set the buffering state for a stream. 2011-02-21 17:33:29 -08:00
Robin Sommer
434f57f85f Implementing dynamic paths via $path_func.
One change to original plan: the function also receives the standard
$path argument.
2011-02-21 17:12:58 -08:00
Robin Sommer
28216c84a2 Implemented removing of log filters. 2011-02-21 16:45:35 -08:00
Robin Sommer
a798200dbf Adding predicate support.
Now the filter's $pred field controls what is being logged.
2011-02-21 14:30:40 -08:00
Robin Sommer
f6da93992c A log write now raises the corresponding event.
The event has moved from the filters to the streams, and must now be
specificed when creating the stream. (Not clear yet whether that is a
indeed the right interface).
2011-02-21 14:14:33 -08:00
Robin Sommer
091547de4f Preparing LogWriter API for rotation and flushing. 2011-02-21 14:13:49 -08:00
Robin Sommer
b98d5adcd5 Fixing case of an event handler having no bodies. To sqash later. 2011-02-21 14:13:06 -08:00