Commit graph

3113 commits

Author SHA1 Message Date
Jon Siwek
d8c716ae17 Merge branch 'master' into topic/jsiwek/filter-rotation 2011-09-08 13:01:00 -05:00
Robin Sommer
db8ab89c3a Merge remote branch 'origin/topic/jsiwek/misc-doc-fixes'
* origin/topic/jsiwek/misc-doc-fixes:
  More tweaks to generated script docs.
  Various changes to documentation framework.

Closes #598.
2011-09-08 09:07:20 -07:00
Robin Sommer
c2501cec51 Merge remote branch 'origin/topic/gilbert/rand-pool'
* origin/topic/gilbert/rand-pool:
  Updating tests.
  Updated uid pools to use integer values instead of strings.
  Updating tests.
  Test no longer relevant.  Need a way to generate and test collisions.
  A few minor tweaks to make code less braindead.  Fixed-length piece of pool name now only used to hash when determinism is not required; otherwise, whole pool name is used.  Note that collisions between pool name hashes will lead to sensitivity to initialization order within the UID generator.
  Testing long (>32 character) pool names.
  Simple test to verify various pools are not affecting each other.
  Some working code.  Adds UID pools classified by string.  Just compiles and runs; need to go back through and make sure this code is actually doing what I want it to do.

Note, I've removed the collision detection. Seems unlikely to occur
and even if, it's not really that bad.
2011-09-07 16:17:59 -07:00
Jon Siwek
d2bf33ee19 Merge branch 'master' into topic/jsiwek/filter-rotation 2011-09-07 12:30:47 -05:00
Jon Siwek
95ed192088 More tweaks to generated script docs.
- The "private interface" section is still tracked, but no longer rendered.
  We can judge the usefulness of it later and add back if there's demand.

- Documentation of event handlers was being treated the same as event
  declarations.  Now handlers are tracked separately, but not currently
  rendered in the generated doc output since usefulness is questionable.
2011-09-07 11:33:08 -05:00
Jon Siwek
80e154ba3c Various changes to documentation framework.
- Reorganize top-level 'doc' Makefile target so submodules can easily
  add their own doc-generating routines to it.  e.g. the Bro project
  makes a placeholder 'doc' target, then adds 'restdoc', 'sphinxdoc';
  later Broccoli can add it's own target as a dependency for generating
  API docs.

- Fixed generated docs for BIFs not being organized under a base/
  subdirectory like the original source files.

- Fixed documentation style for function parameters not applying to
  functions declared as record fields.

- Misc. script documentation tweaks to address warnings given by Sphinx.
2011-09-07 10:02:15 -05:00
Gilbert Clark gc355804@ohio.edu
cff47dd98f Updating tests. 2011-09-05 12:12:09 -07:00
Gilbert Clark gc355804@ohio.edu
3953b851e5 Merge branch 'master' into topic/gilbert/rand-pool
Conflicts:
	testing/btest/Baseline/istate.events-ssl/receiver.http.log
	testing/btest/Baseline/istate.events-ssl/sender.http.log
	testing/btest/Baseline/istate.events/receiver.http.log
	testing/btest/Baseline/istate.events/sender.http.log
	testing/btest/Baseline/scripts.base.protocols.smtp.basic/smtp.log
2011-09-05 11:40:34 -07:00
Robin Sommer
32cb6d6bc3 Adding group "comm" to communication tests for parallelizing execution
with new btest version.
2011-09-05 08:22:07 -07:00
Robin Sommer
33584ec721 Updating test baselines for new ASCII log header. 2011-09-04 13:27:46 -07:00
Robin Sommer
c03d034e49 Updating baselines for recent changes. 2011-09-04 10:18:56 -07:00
Robin Sommer
29e9183f5b Merge remote branch 'origin/topic/jsiwek/nested-record-coerce-fix'
* origin/topic/jsiwek/nested-record-coerce-fix:
  Fix for a case where nested records weren't coerced even though possible.

Closes #571.
2011-09-04 09:00:45 -07:00
Robin Sommer
0d202962bc Merge remote branch 'origin/topic/jsiwek/ascii-log-rotate-fix'
* origin/topic/jsiwek/ascii-log-rotate-fix:
  Change ASCII writer to delay creation of log after rotation until next write.

Closes #570.
2011-09-04 09:00:28 -07:00
Robin Sommer
1c75cd5748 Merge remote branch 'origin/fastpath'
* origin/fastpath:
  Move the `make dist` target into a shell script.
  Update baselines for intervals now using constant 6 decimal place format.
2011-09-04 08:23:28 -07:00
Seth Hall
11c437faa3 Logging framework update and mass Log::ID renaming.
- Log path's are generated in the scripting land
  now.  The default Log stream ID to path string
  mapping works like this:
    - Notice::LOG -> "notice"
    - Notice::POLICY_LOG -> "notice_policy"
    - TestModule::LOG -> "test_module"

- Logging streams updated across all of the shipped
  scripts to be more user friendly.  Instead of
  the logging stream ID HTTP::HTTP, we now have
  HTTP::LOG, etc.

- The priorities on some bro_init handlers have
  been adjusted to make the process of applying
  filters or disabling streams easier for users.
2011-09-03 01:10:17 -04:00
Seth Hall
fe53091cd1 Merge commit '2e853a3'
* commit '2e853a3':
  Change presentation of enum val to include namespace.
2011-09-02 15:04:25 -04:00
Jon Siwek
9c0fc5fe48 Merge branch 'master' into fastpath 2011-09-01 16:32:29 -05:00
Seth Hall
1726aee4e9 Merge remote-tracking branch 'origin/topic/jsiwek/reporter-fatal-bif'
* origin/topic/jsiwek/reporter-fatal-bif:
  Add a Reporter::fatal BIF.
2011-09-01 10:52:39 -04:00
Jon Siwek
02ce833c49 Update baselines for intervals now using constant 6 decimal place format. 2011-08-31 11:17:55 -05:00
Jon Siwek
2e853a3892 Change presentation of enum val to include namespace.
Enum vals now add their full identifier string, including module
namespace to ODescs.
2011-08-31 10:57:19 -05:00
Gilbert Clark gc355804@ohio.edu
97b7140332 Merge remote-tracking branch 'origin/master' into topic/gilbert/rand-pool
Conflicts:
	testing/btest/Baseline/core.vlan-mpls/conn.log
	testing/btest/btest.cfg
2011-08-30 11:59:59 -07:00
Jon Siwek
263ef2bfc9 Add an optional Log::RotationControl to Log::Filter records.
This allows for the RotationControl to be automatically added to the
Log::rotation_control table for the filter's (writer, path) when it is
added to a stream via Log::add_filter.  Log::remove_filter now also
removes any RotationControl's associated with the filter from the
Log::rotation_control table.
2011-08-26 14:36:56 -05:00
Jon Siwek
f08ef8350a Fix for a case where nested records weren't coerced even though possible. 2011-08-26 13:59:51 -05:00
Seth Hall
005b1505b8 Fixing an accidently abandoned test. 2011-08-25 16:53:39 -04:00
Seth Hall
fc5f22cb5d Merge remote-tracking branch 'origin/topic/jsiwek/reorg-followup' 2011-08-25 16:44:31 -04:00
Jon Siwek
31e70e447d Add a Reporter::fatal BIF.
This can be used from a Bro script to cause Bro to exit with non-zero
status and a message to stderr.  Uses of the exit() BIF in scripts
for terminating Bro because of an error were replaced with this one,
which addresses #564.
2011-08-25 13:13:39 -05:00
Seth Hall
562abfb0d1 Merge remote-tracking branch 'origin/topic/gregor/script-polishing'
* origin/topic/gregor/script-polishing:
  Tune when c$conn is set.
  Set c$conn (for logging) in new_connection() event.
  (Semiautomatically) convert the comments in bare-init.bro into autodoc ones.
  Add ConnSize_Analyzer's fields to conn.log
  SSH base scripts: make sure ConnSizeAnalyzer variables are available before using them.
  Make reference to the other script a link
2011-08-25 13:17:49 -04:00
Jon Siwek
38e562dd30 Change ASCII writer to delay creation of log after rotation until next write.
When a given log was rotated, a new log was opened immediately.  If that
log was never written to again, those empty logs perpetually rotate, too.
This change makes it so a log won't be created for a given rotation interval
if there was nothing written to it in that interval.
2011-08-25 10:00:40 -05:00
Seth Hall
be4c9cbef9 Merge branch 'master' of ssh://git.bro-ids.org/bro 2011-08-21 00:32:15 -04:00
Seth Hall
a7f6e4c582 Adding metrics framework intermediate updates.
- Since each host in a cluster has it's own view of the metrics
  the only time the manager would get a chance for a global view
  is the break_interval.  This update improves that time.  If a
  worker crosses 10% of the full threshold, it will send it's
  value to the manager which can then ask the rest of the cluster
  for a global view.  The manager then adds all of the values for
  each workers metric indexes together and will do the notice
  if it crosses the threshold so that it isn't dependent on
  waiting for the break interval to hit.  This functionality
  works completely independently of the break_interval too.  Logging
  will happen as normal.

- Small update for SSH bruteforcer detection to match additions in
  the metrics framework API.

- The hope is that this update is mostly invisible from anyone's
  perspective.  The only affect it should have on users is to better
  the detection of metric values crossing thresholds on cluster
  deployments.
2011-08-21 00:32:00 -04:00
Jon Siwek
1730496d12 Remove the 'net' type from Bro (addresses #535).
Incremented the serialization data format version in the process.
2011-08-17 19:03:34 -05:00
Seth Hall
82f94881c0 Improvements to metrics. SSH bruteforcing detection now done with metrics framework. 2011-08-16 11:47:49 -04:00
Gilbert Clark gc355804@ohio.edu
bb0deea52f Merge remote-tracking branch 'origin/master' into topic/gilbert/rand-pool 2011-08-15 15:55:10 -07:00
Gilbert Clark gc355804@ohio.edu
7bea71a2c2 Updated uid pools to use integer values instead of strings. 2011-08-15 15:47:39 -07:00
Jon Siwek
33fce8a71d Misc. doc/script/test cleanup.
- fixing some Metrics::add_data() call signatures
- slight refactors to cluster framework @if and adding a NONE NodeType for
  so local_node_type() will return that instead of just emitting an error
  when cluster mode isn't enabled
- `make restdoc` target now uses bro's bare-mode
- bro scripts generated from bifs now really only live in the build/src/base/
  directory and changed the DocSourcesList.cmake to dynamically figure out
  what bifs exist by looking in src/ instead of build/src/
- add some missing @load dependencies
2011-08-15 17:29:41 -05:00
Jon Siwek
41dd0b98e9 Merge branch 'master' into topic/jsiwek/reorg-followup
Conflicts:
	scripts/base/frameworks/cluster/setup-connections.bro
	scripts/base/frameworks/metrics/main.bro
	scripts/base/frameworks/notice/actions/email_admin.bro
	scripts/base/frameworks/notice/weird.bro
	scripts/base/protocols/mime/file-hash.bro
	scripts/base/protocols/mime/file-ident.bro
	scripts/policy/frameworks/communication/listen-clear.bro
	scripts/policy/frameworks/communication/listen-ssl.bro
	scripts/policy/frameworks/control/controller.bro
	scripts/policy/frameworks/metrics/http-example.bro
	scripts/policy/frameworks/metrics/ssl-example.bro
	scripts/policy/protocols/conn/scan.bro
	src/CMakeLists.txt
2011-08-15 15:34:25 -05:00
Seth Hall
3919a35b9b Metrics framework update. Mostly to make metrics work on clusters.
- Metrics now work on cluster deployments with no caveats.  It should be
  completely transparent.  Intermediate updates to speed some detection
  will come later.
2011-08-15 15:57:48 -04:00
Gilbert Clark gc355804@ohio.edu
d46525dcca Merge branch 'master' into topic/gilbert/rand-pool 2011-08-14 22:21:59 -07:00
Gilbert Clark gc355804@ohio.edu
066ca80ab4 Updating tests. 2011-08-14 22:16:46 -07:00
Gilbert Clark gc355804@ohio.edu
ad35902bdb Merge branch 'master' into topic/gilbert/rand-pool 2011-08-14 21:56:05 -07:00
Robin Sommer
1ca9def9e6 Merge remote-tracking branch 'origin/topic/jsiwek/autodoc-fixes'
* origin/topic/jsiwek/autodoc-fixes:
  Update doc sources and touch up a few script comments.
  Fixes for script auto-documentation.

Conflicts:
	scripts/base/frameworks/logging/main.bro
2011-08-13 12:09:13 -07:00
Jon Siwek
2a9ea6b8ba Merge branch 'master' into topic/jsiwek/autodoc-fixes
Conflicts:
	scripts/CMakeLists.txt
	scripts/base/frameworks/cluster/setup-connections.bro
	scripts/base/frameworks/communication/__load__.bro
	scripts/base/frameworks/metrics/conn-example.bro
	scripts/base/frameworks/metrics/http-example.bro
	scripts/site/local.bro
2011-08-13 09:31:06 -05:00
Robin Sommer
045fd4690b Merge remote-tracking branch 'origin/topic/seth/piped_exec-update'
* origin/topic/seth/piped_exec-update:
  piped_exec can now write nulls in the "to_write" argument.
2011-08-12 23:35:15 -07:00
Jon Siwek
6f060a58d6 Fix vector initialization for lists of records with optional types.
If possible the list elements now get promoted to the yield type of the
vector.  There was also a problem with the value returned by the record
constructor expression's eval being completely unref'd since the vector
element assignment function doesn't ref the element -- so I changed it
to ref values if they just constructed before assigning them to the
vector.

Addresses #485.
2011-08-12 23:25:01 -07:00
Jon Siwek
560685f1c6 Fix redef'ing records with &default empty set fields.
Attributes have state to track whether they're in a record and should
apply to a record field, but this state wasn't being set for TypeDecls
that are part of a redef'd record.

Closes #460
2011-08-12 23:24:49 -07:00
Jon Siwek
00de88f4cb Fix reporter using part of the actual message as a format string
When not reporting via events, the final contents of the message buffer
after formatting was being used as a format string to fprintf instead of
writing out the actual string.
2011-08-12 23:24:44 -07:00
Robin Sommer
cb31fd3bb9 Logging's path_func now receives the log record as argument.
Closes #555.
2011-08-12 22:18:45 -07:00
Robin Sommer
c436930acf Functions can now be logged.
The function's code is rendered as ASCII and included as a string.
Closes #506.

Note that I'm not sure if the formatting is as desired: should the LFs
and tabs be rendered as \xXX or removed?.
2011-08-12 22:15:57 -07:00
Jon Siwek
521f54c4f0 Fix check_for_unused_event_handlers option.
The format string given to the reporter warning call wasn't printing
the handler names.  Also changed it so that each warning message has
the full context of the warning.
2011-08-11 11:34:42 -05:00
Jon Siwek
c3fb0ea035 Reorganizing btest/policy directory to match new scripts/ organization
Addresses #545
2011-08-11 10:43:11 -05:00