Commit graph

2993 commits

Author SHA1 Message Date
Gregor Maier
ae1eb5379b Merge branch 'master', remote-tracking branch 'origin' into topic/gregor/tunnel 2011-08-15 13:33:14 -07:00
Seth Hall
3919a35b9b Metrics framework update. Mostly to make metrics work on clusters.
- Metrics now work on cluster deployments with no caveats.  It should be
  completely transparent.  Intermediate updates to speed some detection
  will come later.
2011-08-15 15:57:48 -04:00
Robin Sommer
1ca9def9e6 Merge remote-tracking branch 'origin/topic/jsiwek/autodoc-fixes'
* origin/topic/jsiwek/autodoc-fixes:
  Update doc sources and touch up a few script comments.
  Fixes for script auto-documentation.

Conflicts:
	scripts/base/frameworks/logging/main.bro
2011-08-13 12:09:13 -07:00
Jon Siwek
a3147033e2 Update doc sources and touch up a few script comments. 2011-08-13 09:45:42 -05:00
Jon Siwek
2a9ea6b8ba Merge branch 'master' into topic/jsiwek/autodoc-fixes
Conflicts:
	scripts/CMakeLists.txt
	scripts/base/frameworks/cluster/setup-connections.bro
	scripts/base/frameworks/communication/__load__.bro
	scripts/base/frameworks/metrics/conn-example.bro
	scripts/base/frameworks/metrics/http-example.bro
	scripts/site/local.bro
2011-08-13 09:31:06 -05:00
Robin Sommer
cb31fd3bb9 Logging's path_func now receives the log record as argument.
Closes #555.
2011-08-12 22:18:45 -07:00
Gregor Maier
95d84da1b0 Set c$conn (for logging) in new_connection() event.
c$conn was originally set in connection_established(), which is only
used by TCP connections, and in connection_state_remove(). Using
new_connection() allows us to have c$conn available for any connection
and for any script that wants to extend logging to conn.log.
2011-08-11 15:29:54 -07:00
Gregor Maier
eacfbea3fe (Semiautomatically) convert the comments in bare-init.bro into autodoc ones.
With manual tuning were it made sense.
2011-08-11 14:25:34 -07:00
Gregor Maier
fe5624fb06 Add ConnSize_Analyzer's fields to conn.log 2011-08-11 12:21:42 -07:00
Gregor Maier
88f7db927e SSH base scripts: make sure ConnSizeAnalyzer variables are available before using them. 2011-08-11 12:20:38 -07:00
Seth Hall
cc258b29aa Single character bugfix for hostname notice email extension. 2011-08-11 15:16:32 -04:00
Seth Hall
240ae9790b Small updates for notice framework.
- New ACTION_ADD_GEODATA to add geodata to notices in an extension
  field named remote_location.

- Loading extend-email/hostnames by default now that it only
  does anything when the ACTION_EMAIL action is applied (finally).
2011-08-11 14:59:01 -04:00
Jon Siwek
351b13d1c8 Fix more bare-mode @load dependency problems 2011-08-11 11:47:12 -05:00
Jon Siwek
59e5fc5633 Merge branch 'master' into topic/jsiwek/reorg-followup
Conflicts:
	scripts/base/frameworks/cluster/setup-connections.bro
	scripts/base/protocols/ssh/main.bro
2011-08-11 10:56:20 -05:00
Jon Siwek
ed281fb634 Fixing some more bare-mode @load dependency issues 2011-08-11 10:49:41 -05:00
Seth Hall
b45c175147 Split out more SMTP analysis functionality. 2011-08-11 08:26:20 -04:00
Seth Hall
9c2273b7a7 Updates for SSH scripts. 2011-08-11 01:35:50 -04:00
Seth Hall
9a06cece67 ConnSize analyzer is turned on by default now. 2011-08-11 01:32:31 -04:00
Seth Hall
6f50e362db Updates for the build system and site local scripts for cluster. 2011-08-11 01:32:05 -04:00
Jon Siwek
f517d0e0ad Merge branch 'master' into topic/jsiwek/reorg-followup 2011-08-10 19:59:18 -05:00
Jon Siwek
7b07a19cd6 bro scripts generated from bifs now install to $prefix/share/bro/base/
Addresses #545
2011-08-10 19:05:03 -05:00
Jon Siwek
ac257c7a6d Change/fix some cluster script error reporting.
The main script now uses the Reporter::error BIF instead of scheduling
a reporter_error event to report an invalid node name.  The later only
works if the reporter framework is loaded, but the BIF will do the right
thing and use stderr if there's no event handler.

I also @if'd out most of the setup-connections script when the node is
invalid because that's what the cluster/__load__.bro would normally do.
2011-08-10 16:29:52 -05:00
Jon Siwek
9e7934dc32 Normalize some intra-package @loads 2011-08-10 16:06:05 -05:00
Gregor Maier
d0a67dc8bf Tweaking tunnel decapsulation.
Changing names to comply with "Bro Scripting Conventions"
Tweaking documentation.
2011-08-10 13:47:02 -07:00
Jon Siwek
fae54dc8f0 Normalizing base.bro script naming as main.bro 2011-08-10 15:40:28 -05:00
Jon Siwek
47500ceef4 Add a test that checks each individual script can be loaded in bare-mode.
Fixed most @load dependency issues in the process.  The test is still
failing in a "known" way due to hot.conn.bro and scan.bro.

Adressess #545
2011-08-10 15:38:21 -05:00
Gregor Maier
1a565fadfe Merge remote-tracking branch 'origin/topic/jsiwek/autodoc-fixes' into topic/gregor/tunnel 2011-08-10 12:26:13 -07:00
Seth Hall
8e7a76b548 HTTP now uses the extract_filename_from_content_disposition function. 2011-08-10 13:37:58 -04:00
Seth Hall
adc486c673 Merge remote-tracking branch 'origin/topic/jsiwek/smtp-refactor'
- While updating, I did some further work on the branch.

- New function in the base/utils/files for extracting filenames
  from content-dispositions.

- New script for entity excerpt extraction if you aren't interested
  in full extraction.  The data goes a log field too.

- Some renaming and reorganization of types.

- Updated tests to work with new code.

* origin/topic/jsiwek/smtp-refactor:
  Make the doc.coverage test happy.
  SMTP script refactor. (addresses #509)

Conflicts:
	doc/scripts/DocSourcesList.cmake
	policy/protocols/smtp/__load__.bro
	policy/protocols/smtp/base/__load__.bro
2011-08-10 13:34:31 -04:00
Seth Hall
78e69a0bc8 Cluster nodes aren't in fact inherently controllable.
- This @load needs to move to broctl.
2011-08-09 11:15:49 -04:00
Seth Hall
5f61c9bde9 New variable in utils/site for mapping address to defined local subnet.
- Site::local_nets_table[1.2.3.4] might yield "1.2.0.0/16" if that
  subnet is defined in the Site::local_nets variable.
2011-08-09 11:15:07 -04:00
Seth Hall
29425c29e2 Communication framework updates now that communications can be enabled dynamically. 2011-08-09 11:13:36 -04:00
Seth Hall
9fa3bcffdd Metrics framework checkpoint.
- Metrics API is much more similar to the Logging framework's API now.

- Filters define all output and metrics collection now.

- Initial attempt at thresholding and generating notices.
2011-08-09 11:12:02 -04:00
Jon Siwek
4ac6d0ae2e Fixes for script auto-documentation.
- Fixing the parts of the `make restdoc` and `make doc` process that were
  broken by the last Bro script re-organization

- Generated documentation for Bro scripts derived from BiFs now use the
  original BiF source file as the "original source file" link

- Renaming of the internal POLICYDEST definition and other misc places that
  refer to "policy" scripts; that terminology doesn't make total sense now

- Added a documentation blacklist reminder test that will fail if there's
  scripts that are blacklisted from being documentated because they're still
  in progress

- Some minor Bro script changes to fix small @load dependency errors

Addresses #543
2011-08-08 19:50:45 -05:00
Robin Sommer
bbb0b855a2 Merge branch 'topic/robin/rotation-pp'
* topic/robin/rotation-pp:
  Adding a default_path_func that makes the default naming scheme script-level controlled.
  Reworking logging's postprocessor logic.

Conflicts:
	scripts/base/frameworks/logging/main.bro
	testing/btest/policy/frameworks/logging/rotate-custom.bro
2011-08-08 16:14:54 -07:00
Robin Sommer
103396f6d3 Communication must now be enabled explicitly.
The communication subsystem is now disabled until a new BiF,
enable_communication(), is called. The base scripts do this
automatically when either a Communication::Node is defined, or Bro is
asked to listen for incoming connections.
2011-08-08 13:49:50 -07:00
Seth Hall
c4f4df6a79 Renaming the default loaded "init" scripts and added command line arg for "bare-mode"
- bro.init was renamed to base/init-bare.bro and base/all.bro
  was renamed to init-default.bro.

- To run in "bare mode" with only the init-bare.bro and no other
  scripts from base/, use either -b or --bare-mode.

- The environment variable to run in "bare mode" has been removed.
2011-08-08 13:40:43 -04:00
Gregor Maier
cd592203a0 Remvoing old tunnel code (encap_hdr_size and udp_tunnels). 2011-08-07 13:55:46 -07:00
Gregor Maier
32f37c9f6d Documenting tunnel decapsulation.
Haven't tested the autodoc output yet.
2011-08-07 12:26:19 -07:00
Gregor Maier
b0ac4882bd Merge branch 'master' into topic/gregor/tunnel
* master: (21 commits)
  Fixing default policy search path.
  Small tweaks for doc generation.
  The hostname notice email extension now only add hostnames for emailed noticed.
  Attempt at cleaning up doc generation.
  Hopefully the last major script reorganization.
  Updating submodule(s).
  Fixing compiler warning.
  Fixing crash in memory profiling.
  Fix compiler warning.
  Fixing memory (and CPU) leak in log writer.
  Fixing missing sync in cluster setup.
  Update script doc-generation README.
  Omission of bro.init from doc generation fixed.
  Fixing docstrings to make docutils happy.
  Updated submodules to current master
  Updating submodule(s).
  Updating submodule(s).
  Fix genDocSourcesList script lack of explicit sorting
  Fix &raw_output and enable_raw_output interpretation of NUL characters
  Fix PktSrc::Statistics returning bogus stats.
  ...

Conflicts:
	scripts/base/bro.init
2011-08-07 09:59:30 -07:00
Seth Hall
048369fbeb The hostname notice email extension now only add hostnames for emailed noticed. 2011-08-06 00:33:58 -04:00
Seth Hall
d6a67f7c1f Merge branch 'master' of ssh://git.bro-ids.org/bro
Conflicts:
	scripts/base/frameworks/control/main.bro
2011-08-05 23:11:40 -04:00
Seth Hall
597a4d6704 Hopefully the last major script reorganization.
- policy/ renamed to scripts/

- By default BROPATH now contains:
	- scripts/
	- scripts/policy
	- scripts/site

- *Nearly* all tests pass.

- All of scripts/base/ is loaded by main.cc
	- Can be disabled by setting $BRO_NO_BASE_SCRIPTS
	- Scripts in scripts/base/ don't use relative path loading to ease use of BRO_NO_BASE_SCRIPTS (to copy and paste that script).

- The scripts in scripts/base/protocols/ only (or soon will only) do logging and state building.

- The scripts in scripts/base/frameworks/ add functionality without causing any additional overhead.

- All "detection" activity happens through scripts in scripts/policy/.

- Communications framework modified temporarily to need an environment variable to actually enable (ENABLE_COMMUNICATION=1)
	- This is so the communications framework can be loaded as part
	  of the base without causing trouble when it's not needed.
	- This will be removed once a resolution to ticket #540 is reached.
2011-08-05 23:09:53 -04:00