Commit graph

13262 commits

Author SHA1 Message Date
Seth Hall
ef5aba6f5f Merge branch 'master' into topic/logging-framework
Conflicts:
	src/builtin-func.l
2011-03-01 17:36:05 -05:00
Seth Hall
04c5c21705 Adapted known-hosts for the logging framework. 2011-03-01 17:34:27 -05:00
Seth Hall
fe85a3e4d2 Adapted for API similarity with Robin's logging framework code. 2011-03-01 17:17:48 -05:00
Seth Hall
e7079aefab Added site as a dependency in the functions file. 2011-03-01 11:47:48 -05:00
Seth Hall
60e66f3227 Added site.bro as another dependency to policy directory. 2011-03-01 11:45:23 -05:00
Seth Hall
600c5bedea Moved some base script dependencies to the new policy directory. 2011-03-01 11:41:07 -05:00
Seth Hall
517456a2df Added and rewrote known-hosts.bro
Created a functions.bro file to add various
reusable functionality.
2011-03-01 11:31:42 -05:00
Seth Hall
62613ec1a0 Moved policy script directory to policy.old 2011-03-01 10:51:44 -05:00
Robin Sommer
d673c8c64c Adding a new RecordVal method LookupWithDefault().
The method honors a potential &default attribute, and refs the Val it
returns.
2011-02-28 22:20:29 -08:00
Robin Sommer
ba2c6f6139 Internal refactoring to provide injection points for remotely received
log records.

Also added some additional type-safety check to later make sure that
what a peer sends is actually of the expected format.
2011-02-28 21:52:56 -08:00
Robin Sommer
e6edc52d54 Moving the map of writers from the filters to the streams.
This is in preparation for remote logging.
2011-02-28 18:24:19 -08:00
Robin Sommer
e5d3654266 Updating submodule(s). 2011-02-28 15:39:24 -08:00
Robin Sommer
a770d80f6b Merge remote branch 'origin/fastpath'
* origin/fastpath:
  Fixing compiler warnings
  Revert "Updating submodule to current master"
  Remvoing leftover local variables that caused compile error on Mac OS X.
  Updating submodule to current master
2011-02-28 15:38:29 -08:00
Robin Sommer
ab15437339 Working on the logging API exposed to scripts.
- Moving all functions into the Log::* namespace, using the recent
  bifcl updates. Moved logging-specific stuff to logging.bif.

- Log::create_stream() now takes a record Log::Stream as its second
  argument, which specifies columns and (optionally) the event.

- All the internal BiFs are now called "Log::__<something>", with
  script-level wrappers "Log::<something>". That first allows to add
  additional code at the script-level, and second makes things better
  comprehendible as now all relevant functionality is collected (and
  later documetned) in policy/logging.bro.

- New function Log::flush(id), which does the obvious assuming the
  writer supports it.

- add_default_filter() is now called implicitly with every
  create_stream(). Seems that we usually want that functionality, and
  when not, remove_default_filter() gets rid of it.

- The namespace of a stream's ID is now used as the default "path"
  (e.g., if the namespace is SSH, the default log file is "ssh.log").

- Updated policy/test-logging.bro as well as the btest tests according
  to these changes.
2011-02-27 15:09:37 -08:00
Gregor Maier
fc6fcded07 Fixing compiler warnings 2011-02-25 21:15:22 -08:00
Gregor Maier
c54c1e0dce Revert "Updating submodule to current master"
This reverts commit 8b42bfed7c.
Master seems to have outdated submodules. Reverting to previous ones on
fastpath.
2011-02-25 21:11:06 -08:00
Gregor Maier
7d822e232d Remvoing leftover local variables that caused compile error on Mac OS X. 2011-02-25 21:09:55 -08:00
Gregor Maier
b6fd317dcc Merge remote branch 'origin/master' into fastpath 2011-02-25 20:44:05 -08:00
Gregor Maier
8b42bfed7c Updating submodule to current master 2011-02-25 20:43:53 -08:00
Robin Sommer
ff740f153c Fixing file detector leak in remote communication module.
This addresses #400.
2011-02-25 17:04:29 -08:00
Robin Sommer
2a82e0bd9b Updating indenpdentn state tests to work with new setyp.
Note that the broccoli test does currently not pass because of the
64-bit changes.
2011-02-25 17:01:11 -08:00
Robin Sommer
e3b2748fa6 Merge remote branch 'origin/master' into topic/robin/logging-internals
Conflicts:
	policy/bro.init
	src/builtin-func.l
2011-02-25 16:11:03 -08:00
Robin Sommer
226eeb9729 Updating submodule(s). 2011-02-25 15:49:08 -08:00
Robin Sommer
3527daea5b Updating submodule(s). 2011-02-25 15:48:31 -08:00
Robin Sommer
12139e9faf Merge remote branch 'origin/topic/gregor/bif-tuning'
* origin/topic/gregor/bif-tuning:
  Refactor: BifTypePtr --> BifType
  Bif const: make sure const is indeed a constant.
  Support any type in bif const declaration.
  Tweak for bifcl
  Fix to bifcl wrt namespaces.
  Enable declaration of set, vector, and table types in bifs.
  Moving type declarations into its own bif file
  Support namespaces / modules in bif. Checkpoint.
  Support namespaces / modules in bif. Checkpoint.
  Remove leftovers from removing "declare enum" from bifcl
  Use namespaces for NetVar type pointers.
  Remove unused and unnecessary "declare enum" from bifcl
  Bif: add record type declaration.
  Minor tweaks for bif language.
  enum type: don't allow mixing of explicit value and auto-increment.
  Add support for enum with explicit enumerator values.

Closes #403.
2011-02-25 15:41:56 -08:00
Jon Siwek
30209b56bb Bro script documentation framework checkpoint
* New bro runtime options: -Z or --doc-scripts enables documentation mode
* New BroDoc, BroBifDoc, and BroDocObj interfaces to support script
  documentation
* Modifications to the bro scanner (scan.l) to get it to keep track of
  which script is being scanned/parsed and which document is being generated
* Modifications to scan.l and the bro parser (parse.y) to produce/consume
  script comments denoted with "##"
* Documentation is currently generated for the following
** Script author
** Script summary
** @load's
** capture_filters
** modules (namespaces)

Most of the remaining framework/infrastructure work should be in extracting
the interesting BroObj objects as the parser sees them and better formatting
the reST documents.
2011-02-25 15:30:18 -06:00
Robin Sommer
a3e1dd5261 Merge remote branch 'origin/topic/gregor/reassmbler-hotfix'
Closes #404.

* origin/topic/gregor/reassmbler-hotfix:
  TCP Reassembler hotfix for conns > 2GB.
2011-02-25 10:54:43 -08:00
Robin Sommer
9b94218c63 Merge remote branch 'origin/topic/gregor/fix-val-64bit'
* origin/topic/gregor/fix-val-64bit:
  Fixing endianess error in XDR when data is not 4-byte aligned.
  Fix for Val constructor with new int64 typedefs.
  New fix for OS X 10.5 compile error wrt llabs()
  Revert "Fix for OS X 10.5 compile error wrt llabs()"
2011-02-25 10:41:51 -08:00
Jon Siwek
dec97ab56d Changes to the way user-modifiable config files are installed.
- Duplicates of the distribution's configuration files are now always
  installed with a .example suffix
- Added --binary-package configure option to toggle configure logic
  specific to the creation of binary packages.
- When not in binary packaging mode, `make install` never overwrites
  existing configure files in case they've been modified.  The previous
  behavior (CMake's default) would only avoid overwriting modified files
  if one consistently uses the same build directory and doesn't reconfigure.
- Fixed an issue with Mac package's pre-install script not preserving ACLs
- Minor cleanup/refactor of the make-mac/rpm-packages scripts
2011-02-24 22:02:22 -06:00
Gregor Maier
6af01a205f Merge remote branch 'origin/master' into topic/gregor/reassmbler-hotfix 2011-02-22 16:58:05 -08:00
Gregor Maier
e310119ffa Refactor: BifTypePtr --> BifType
As per our mail discussion renaming BifTypePtr namespace to
BifType.
2011-02-22 14:52:21 -08:00
Robin Sommer
f02e465c92 Tests for the logging functionality, including a basic btest
configuration.
2011-02-21 18:29:41 -08:00
Robin Sommer
ac936feb95 Consistent error handling. 2011-02-21 18:18:12 -08:00
Robin Sommer
cf148c8a25 New bif log_set_buf() to set the buffering state for a stream. 2011-02-21 17:33:29 -08:00
Robin Sommer
434f57f85f Implementing dynamic paths via $path_func.
One change to original plan: the function also receives the standard
$path argument.
2011-02-21 17:12:58 -08:00
Robin Sommer
28216c84a2 Implemented removing of log filters. 2011-02-21 16:45:35 -08:00
Robin Sommer
a798200dbf Adding predicate support.
Now the filter's $pred field controls what is being logged.
2011-02-21 14:30:40 -08:00
Robin Sommer
f6da93992c A log write now raises the corresponding event.
The event has moved from the filters to the streams, and must now be
specificed when creating the stream. (Not clear yet whether that is a
indeed the right interface).
2011-02-21 14:14:33 -08:00
Robin Sommer
091547de4f Preparing LogWriter API for rotation and flushing. 2011-02-21 14:13:49 -08:00
Robin Sommer
b98d5adcd5 Fixing case of an event handler having no bodies. To sqash later. 2011-02-21 14:13:06 -08:00
Robin Sommer
c0cd62a5a5 Enable passing events into bifs.
When an event was globally decleared, previously it did not get
assigned a value initially until the first implementation body was
added. That then triggered an "not used" error when passing such an
event as argument into a bif. Now we always assign a function value
immediately, just without any body inititally.

When globally declaring an event, i
2011-02-21 13:45:44 -08:00
Robin Sommer
3fbb3c0fcd Missing piece for previous patch adding type vals. 2011-02-21 13:45:07 -08:00
Robin Sommer
c015605113 Merge remote branch 'origin/master' into topic/robin/logging-internals 2011-02-20 17:18:06 -08:00
Robin Sommer
56880da779 Adding a TODO for the internal logging stuff. 2011-02-18 13:11:55 -08:00
Robin Sommer
68062e87f1 Lots of infracstructure for the new logging framework.
This pretty much follows the proposal on the projects page.

It includes:

    - A new LogMgr, maintaining the set of writers.

    - The abstract LogWriter API.

    - An initial implementation in the form of LogWriterAscii
      producing tab-separated columns.

Note that things are only partially working right now, things are
subject to change, and it's all not much tested at all. That's why I'm
creating separate branch for now.

Example:

     bro -B logging test-logging && cat debug.log
    1298063168.409852/1298063168.410368 [logging] Created new logging stream 'SSH::LOG_SSH'
    1298063168.409852/1298063168.410547 [logging] Created new filter 'default' for stream 'SSH::LOG_SSH'
    1298063168.409852/1298063168.410564 [logging]    writer    : Ascii
    1298063168.409852/1298063168.410574 [logging]    path      : ssh_log_ssh
    1298063168.409852/1298063168.410584 [logging]    path_func : not set
    1298063168.409852/1298063168.410594 [logging]    event     : not set
    1298063168.409852/1298063168.410604 [logging]    pred      : not set
    1298063168.409852/1298063168.410614 [logging]    field          t: time
    1298063168.409852/1298063168.410625 [logging]    field  id.orig_h: addr
    1298063168.409852/1298063168.410635 [logging]    field  id.orig_p: port
    1298063168.409852/1298063168.410645 [logging]    field  id.resp_h: addr
    1298063168.409852/1298063168.410655 [logging]    field  id.resp_p: port
    1298063168.409852/1298063168.410665 [logging]    field     status: string
    1298063168.409852/1298063168.410675 [logging]    field    country: string
    1298063168.409852/1298063168.410817 [logging] Wrote record to filter 'default' on stream 'SSH::LOG_SSH'
    1298063168.409852/1298063168.410865 [logging] Wrote record to filter 'default' on stream 'SSH::LOG_SSH'
    1298063168.409852/1298063168.410906 [logging] Wrote record to filter 'default' on stream 'SSH::LOG_SSH'
    1298063168.409852/1298063168.410945 [logging] Wrote record to filter 'default' on stream 'SSH::LOG_SSH'
    1298063168.409852/1298063168.411044 [logging] Wrote record to filter 'default' on stream 'SSH::LOG_SSH

> cat ssh_log_ssh.log
1298063168.40985        1.2.3.4 66770   2.3.4.5 65616   success unknown
1298063168.40985        1.2.3.4 66770   2.3.4.5 65616   failure US
1298063168.40985        1.2.3.4 66770   2.3.4.5 65616   failure UK
1298063168.40985        1.2.3.4 66770   2.3.4.5 65616   success BR
1298063168.40985        1.2.3.4 66770   2.3.4.5 65616   failure MX
2011-02-18 13:03:46 -08:00
Robin Sommer
9d407d882c Two small infrastructure extensions for passing information into the
logging framework.

- To enable passing a type into a bif, there's now a new
  BroType-derived class TypeType and a corresponding TYPE_TYPE tag.
  With that, a Val can now have a type as its value.

  This is experimental for now.

- RecordVal's get a new method CoerceTo() to coerce their value into a
  another record type with the usual semantics. Most of the code in
  there was previously in RecordContructorExpr::InitVal(), which is
  now calling the new CoerceTo() method.
2011-02-18 13:01:34 -08:00
Gregor Maier
fab11c44ee Merge remote branch 'origin/master' into topic/gregor/bif-tuning 2011-02-16 16:15:49 -08:00
Robin Sommer
f79a1f6e58 Updating submodule(s). 2011-02-16 08:44:33 -08:00
Gregor Maier
f3aa7b2fac Bif const: make sure const is indeed a constant. 2011-02-14 10:43:26 -08:00
Gregor Maier
782f007b5c Support any type in bif const declaration.
Revamp of const delcaration in bifs:
* Can only declare are const in the bif, but we cannot assign a value
  or attribute to it. One has to do this in a policy file (bro.init)
* Type specification in bif is now mandatory
* Support any type in bifs (previously only bools were supported).

This will also help with automatic documentation generation, since all
const are now defined in the policy layer and thus can be documented
from there. The bif just gives the C++ layer easy access.
2011-02-14 10:10:40 -08:00