Commit graph

1029 commits

Author SHA1 Message Date
Seth Hall
af6c7c8b1a HTTP body size measurement added to http log.
- The value of the content-length headers has now been removed
  but it could be added back locally at an installation by a user.

- Added fields to indicate if some parsing interruption happened
  during the body transfer.

- Closes #581
2011-09-13 21:34:29 -04:00
Seth Hall
ee1884ca93 Another change to possibly fix the SSL memory consumption problem.
- If a protocol violation happens, Bro now logs what it has seen
  up until the protocol violation and deletes the c$ssl record
  so that a long lived connection with a protocol violation does
  continue to hold the memory.
2011-09-13 09:09:55 -04:00
Seth Hall
c87704cc25 Small usability and correctness updates.
- Removed an notice definition from the base SSL scripts.

- Moved a logging stream ID into the export section for known-services
  and bumped priority for creating the stream.

- Adding configuration knobs for the SQL injection attack detection
  script and renaming the HTTP::SQL_Injection_Attack notice to
  HTTP::SQL_Injection_Attack_Against

- Bumped priority when creating Known::CERTS_LOG.
2011-09-12 16:14:28 -04:00
Robin Sommer
db8ab89c3a Merge remote branch 'origin/topic/jsiwek/misc-doc-fixes'
* origin/topic/jsiwek/misc-doc-fixes:
  More tweaks to generated script docs.
  Various changes to documentation framework.

Closes #598.
2011-09-08 09:07:20 -07:00
Seth Hall
4931aa815f Delete SSL certificates from memory after ssl_established event.
- This is an attempt at fixing the memory issues brought about by
  the introduction of the new SSL analyzer.  My initial testing
  shows a hefty memory saving.
2011-09-08 01:52:25 -04:00
Seth Hall
e07e4ca117 Cleaned up some small SSL mistakes. 2011-09-07 15:11:01 -04:00
Jon Siwek
80e154ba3c Various changes to documentation framework.
- Reorganize top-level 'doc' Makefile target so submodules can easily
  add their own doc-generating routines to it.  e.g. the Bro project
  makes a placeholder 'doc' target, then adds 'restdoc', 'sphinxdoc';
  later Broccoli can add it's own target as a dependency for generating
  API docs.

- Fixed generated docs for BIFs not being organized under a base/
  subdirectory like the original source files.

- Fixed documentation style for function parameters not applying to
  functions declared as record fields.

- Misc. script documentation tweaks to address warnings given by Sphinx.
2011-09-07 10:02:15 -05:00
Seth Hall
11c437faa3 Logging framework update and mass Log::ID renaming.
- Log path's are generated in the scripting land
  now.  The default Log stream ID to path string
  mapping works like this:
    - Notice::LOG -> "notice"
    - Notice::POLICY_LOG -> "notice_policy"
    - TestModule::LOG -> "test_module"

- Logging streams updated across all of the shipped
  scripts to be more user friendly.  Instead of
  the logging stream ID HTTP::HTTP, we now have
  HTTP::LOG, etc.

- The priorities on some bro_init handlers have
  been adjusted to make the process of applying
  filters or disabling streams easier for users.
2011-09-03 01:10:17 -04:00
Seth Hall
fc5f22cb5d Merge remote-tracking branch 'origin/topic/jsiwek/reorg-followup' 2011-08-25 16:44:31 -04:00
Seth Hall
562abfb0d1 Merge remote-tracking branch 'origin/topic/gregor/script-polishing'
* origin/topic/gregor/script-polishing:
  Tune when c$conn is set.
  Set c$conn (for logging) in new_connection() event.
  (Semiautomatically) convert the comments in bare-init.bro into autodoc ones.
  Add ConnSize_Analyzer's fields to conn.log
  SSH base scripts: make sure ConnSizeAnalyzer variables are available before using them.
  Make reference to the other script a link
2011-08-25 13:17:49 -04:00
Seth Hall
9160898d03 The SSH::Login notice type has returned. 2011-08-24 12:55:28 -04:00
Gregor Maier
89d5e01d38 Tune when c$conn is set.
After discussion with Seth we now set c$conn in
connection_state_remove() with priority 5, and log it with priority -5.
If users want to extend c$conn before connection_state_remove, they can
just create c$conn and add custom fields.

Addresses: #554
2011-08-19 14:42:48 -07:00
Jon Siwek
2a9ea6b8ba Merge branch 'master' into topic/jsiwek/autodoc-fixes
Conflicts:
	scripts/CMakeLists.txt
	scripts/base/frameworks/cluster/setup-connections.bro
	scripts/base/frameworks/communication/__load__.bro
	scripts/base/frameworks/metrics/conn-example.bro
	scripts/base/frameworks/metrics/http-example.bro
	scripts/site/local.bro
2011-08-13 09:31:06 -05:00
Gregor Maier
95d84da1b0 Set c$conn (for logging) in new_connection() event.
c$conn was originally set in connection_established(), which is only
used by TCP connections, and in connection_state_remove(). Using
new_connection() allows us to have c$conn available for any connection
and for any script that wants to extend logging to conn.log.
2011-08-11 15:29:54 -07:00
Gregor Maier
fe5624fb06 Add ConnSize_Analyzer's fields to conn.log 2011-08-11 12:21:42 -07:00
Gregor Maier
88f7db927e SSH base scripts: make sure ConnSizeAnalyzer variables are available before using them. 2011-08-11 12:20:38 -07:00
Jon Siwek
351b13d1c8 Fix more bare-mode @load dependency problems 2011-08-11 11:47:12 -05:00
Jon Siwek
59e5fc5633 Merge branch 'master' into topic/jsiwek/reorg-followup
Conflicts:
	scripts/base/frameworks/cluster/setup-connections.bro
	scripts/base/protocols/ssh/main.bro
2011-08-11 10:56:20 -05:00
Jon Siwek
ed281fb634 Fixing some more bare-mode @load dependency issues 2011-08-11 10:49:41 -05:00
Seth Hall
b45c175147 Split out more SMTP analysis functionality. 2011-08-11 08:26:20 -04:00
Seth Hall
9c2273b7a7 Updates for SSH scripts. 2011-08-11 01:35:50 -04:00
Jon Siwek
f517d0e0ad Merge branch 'master' into topic/jsiwek/reorg-followup 2011-08-10 19:59:18 -05:00
Jon Siwek
9e7934dc32 Normalize some intra-package @loads 2011-08-10 16:06:05 -05:00
Jon Siwek
fae54dc8f0 Normalizing base.bro script naming as main.bro 2011-08-10 15:40:28 -05:00
Jon Siwek
47500ceef4 Add a test that checks each individual script can be loaded in bare-mode.
Fixed most @load dependency issues in the process.  The test is still
failing in a "known" way due to hot.conn.bro and scan.bro.

Adressess #545
2011-08-10 15:38:21 -05:00
Seth Hall
8e7a76b548 HTTP now uses the extract_filename_from_content_disposition function. 2011-08-10 13:37:58 -04:00
Seth Hall
adc486c673 Merge remote-tracking branch 'origin/topic/jsiwek/smtp-refactor'
- While updating, I did some further work on the branch.

- New function in the base/utils/files for extracting filenames
  from content-dispositions.

- New script for entity excerpt extraction if you aren't interested
  in full extraction.  The data goes a log field too.

- Some renaming and reorganization of types.

- Updated tests to work with new code.

* origin/topic/jsiwek/smtp-refactor:
  Make the doc.coverage test happy.
  SMTP script refactor. (addresses #509)

Conflicts:
	doc/scripts/DocSourcesList.cmake
	policy/protocols/smtp/__load__.bro
	policy/protocols/smtp/base/__load__.bro
2011-08-10 13:34:31 -04:00
Jon Siwek
4ac6d0ae2e Fixes for script auto-documentation.
- Fixing the parts of the `make restdoc` and `make doc` process that were
  broken by the last Bro script re-organization

- Generated documentation for Bro scripts derived from BiFs now use the
  original BiF source file as the "original source file" link

- Renaming of the internal POLICYDEST definition and other misc places that
  refer to "policy" scripts; that terminology doesn't make total sense now

- Added a documentation blacklist reminder test that will fail if there's
  scripts that are blacklisted from being documentated because they're still
  in progress

- Some minor Bro script changes to fix small @load dependency errors

Addresses #543
2011-08-08 19:50:45 -05:00
Seth Hall
597a4d6704 Hopefully the last major script reorganization.
- policy/ renamed to scripts/

- By default BROPATH now contains:
	- scripts/
	- scripts/policy
	- scripts/site

- *Nearly* all tests pass.

- All of scripts/base/ is loaded by main.cc
	- Can be disabled by setting $BRO_NO_BASE_SCRIPTS
	- Scripts in scripts/base/ don't use relative path loading to ease use of BRO_NO_BASE_SCRIPTS (to copy and paste that script).

- The scripts in scripts/base/protocols/ only (or soon will only) do logging and state building.

- The scripts in scripts/base/frameworks/ add functionality without causing any additional overhead.

- All "detection" activity happens through scripts in scripts/policy/.

- Communications framework modified temporarily to need an environment variable to actually enable (ENABLE_COMMUNICATION=1)
	- This is so the communications framework can be loaded as part
	  of the base without causing trouble when it's not needed.
	- This will be removed once a resolution to ticket #540 is reached.
2011-08-05 23:09:53 -04:00