Commit graph

121 commits

Author SHA1 Message Date
Jon Siwek
8c3cf8921a Disable all default AppStat plugins except facebook.
The scripts for the others still remain and can be loaded explicitly,
but they reportedly may produce figures that are far from correct.

Addresses BIT-1171.
2014-05-16 14:15:39 -05:00
Robin Sommer
dadfcde70e Merge branch 'topic/robin/event-dumper'
Changes:

   - Changing semantics of the new_event() meta event: it's raised
     only for events that have a handler defined. There are too many
     checks in Bro that prevent events wo/ handler from being even
     prepared to raise to do that differently.

   - Adding test case.

* topic/robin/event-dumper:
  New script misc/dump-events.bro, along with core support, that dumps events Bro is raising in an easily readable form.
  Prettyfing Describe() for record types.
2013-12-04 12:13:07 -08:00
Robin Sommer
de9f03b0bf New script misc/dump-events.bro, along with core support, that dumps events Bro is raising
in an easily readable form.

This is for debugging purposes, obviously.

Example, including only SMTP events:

    > bro -r smtp.trace misc/dump-events.bro DumpEvents::include=/smtp/

    [...]
    1254722768.219663 smtp_reply
                  [0] c: connection   = [id=[orig_h=10.10.1.4, orig_p=1470/tcp, resp_h=74.53.140.153, [...]
                  [1] is_orig: bool   = F
                  [2] code: count     = 220
                  [3] cmd: string     = >
                  [4] msg: string     = xc90.websitewelcome.com ESMTP Exim 4.69 #1 Mon, 05 Oct 2009 01:05:54 -0500
                  [5] cont_resp: bool = T

    1254722768.219663 smtp_reply
                  [0] c: connection   = [id=[orig_h=10.10.1.4, orig_p=1470/tcp, resp_h=74.53.140.153, [...]
                  [1] is_orig: bool   = F
                  [2] code: count     = 220
                  [3] cmd: string     = >
                  [4] msg: string     = We do not authorize the use of this system to transport unsolicited,
                  [5] cont_resp: bool = T
    [...]
2013-11-22 14:20:00 -08:00
Jon Siwek
9967aea52c Integrate new Broxygen functionality into Sphinx.
Add a "broxygen" domain Sphinx extension w/ directives to allow
on-the-fly documentation to be generated w/ Bro and included in files.

This means all autogenerated reST docs are now done by Bro.  The odd
CMake/Python glue scipts which used to generate some portions are now
gone.  Bro and the Sphinx extension handle checking for outdated docs
themselves.

Parallel builds of `make doc` target should now work (mostly because
I don't think there's any tasks that can be done in parallel anymore).

Overall, this seems to simplify things and make the Broxygen-generated
portions of the documentation visible/traceable from the main Sphinx
source tree.  The one odd thing still is that per-script documentation
is rsync'd in to a shadow copy of the Sphinx source tree within the
build dir.  This is less elegant than using the new broxygen extension
to make per-script docs, but rsync is faster and simpler.  Simpler as in
less code because it seems like, in the best case, I'd need to write a
custom Sphinx Builder to be able to get that to even work.
2013-11-21 14:34:32 -06:00
Jon Siwek
1fabbd441c Remove unneeded Broxygen comments in scan.bro.
They just duplicated the text from where the events are originally
declared and also it's not generally useful to Broxygen-style comment
event *handlers* (they're more of an implementation detail of a script,
not a user-facing element).
2013-11-05 10:54:05 -06:00
Vlad Grigorescu
e1d2f6d82f Change percent_lost in capture-loss from a string to a double. 2013-10-29 06:03:34 -07:00
Daniel Thayer
72a4a90416 Add more script package README files
The text from these README files appears on the "Bro Script Packages"
page after building the documentation.
2013-10-23 16:36:14 -05:00
Daniel Thayer
1b26c05c2c Add more script package README files
The text from these README files appears on the "Bro Script Packages"
page after building the documentation.
2013-10-22 14:44:59 -05:00
Daniel Thayer
181f180374 Fix doc comment so the text gets included in the HTML 2013-10-22 14:42:23 -05:00
Daniel Thayer
02d7e16997 Fix typos and formatting in the other policy docs 2013-10-21 02:37:00 -05:00
Daniel Thayer
9d2668a3f4 Fix incorrect uses of reST directives
These did not generate any warning messages unfortunately, so had to
find them using grep.
2013-10-17 01:50:42 -05:00
Seth Hall
2b78922af8 Added a $unique_max field to Reducers for the SumStats::UNIQUE calculation
- Using the new option in scan.bro and the FTP bruteforce detection.
2013-08-28 00:57:44 -04:00
Robin Sommer
e7aefcdf36 Merge branch 'master' into topic/robin/dhcp-merge 2013-08-03 19:14:01 -07:00
Robin Sommer
308db797c3 Merge remote-tracking branch 'origin/topic/vladg/dhcp'
* origin/topic/vladg/dhcp:
  DHCP: Adding unit tests.
  DHCP:
  Rework the DHCP analyzer to make it compatible again.
2013-08-03 19:14:00 -07:00
Seth Hall
d6edbd27b1 Merge remote-tracking branch 'origin/master' into topic/seth/sumstats-updates 2013-08-02 13:17:48 -04:00
Seth Hall
4f8100774c Updates for SumStats API to deal with high memory stats.
- The code is a mess and will need to be cleaned up, but the
    tests do pass.
2013-08-02 12:44:33 -04:00
Vlad Grigorescu
0f1b0e75b7 DHCP:
- Added the known-devices log, and the script to log DHCP hostname per MAC address,
 - Added DPD sig,
 - Removed some scripts following a discussion with Seth.
2013-07-31 16:32:20 -04:00
Seth Hall
7db531e162 Merge remote-tracking branch 'origin/master' into topic/seth/sumstats-updates
Conflicts:
	doc/scripts/DocSourcesList.cmake
	scripts/test-all-policy.bro
2013-07-30 11:48:03 -04:00
Jon Siwek
d84f6e012c Fix various documentation, mostly related to file analysis.
- Fix examples/references in the file analysis how-to/usage doc.

- Add Broxygen-generated docs for file analyzer plugins.

- Break FTP::Info type declaration out in to its own file to get
  rid of some circular dependencies (between s/b/p/ftp/main and
  s/b/p/ftp/utils).
2013-07-29 16:15:37 -05:00
Robin Sommer
b62927e9de Merge remote-tracking branch 'origin/topic/seth/packet-filter-updates'
Closes #1030.

* origin/topic/seth/packet-filter-updates:
  Missed a test fix.
  Updating test baselines.
  Updates for the PacketFilter framework to simplify it.
  Last test update for PacketFilter framework.
  Several final fixes for PacketFilter framework.
  Packet filter framework checkpoint.
  Checkpoint on the packet filter framework.
  Initial rework of packet filter framework.
2013-07-07 21:09:28 -07:00
Seth Hall
5f8ee93ef0 Merge remote-tracking branch 'origin/master' into topic/seth/analyzer-framework
Conflicts:
	scripts/base/init-default.bro
	scripts/base/protocols/dns/main.bro
	scripts/base/protocols/ftp/main.bro
	scripts/base/protocols/http/main.bro
	scripts/base/protocols/irc/main.bro
	scripts/base/protocols/smtp/main.bro
	scripts/base/protocols/ssh/main.bro
	scripts/base/protocols/ssl/main.bro
	scripts/base/protocols/syslog/main.bro
	src/main.cc
	testing/btest/Baseline/coverage.default-load-baseline/canonified_loaded_scripts.log
2013-07-04 23:07:52 -04:00
Robin Sommer
c049c758c3 Merge remote-tracking branch 'origin/master' into topic/robin/plugins
Conflicts:
	aux/bro-aux
	aux/broctl
	src/DPM.cc
2013-05-30 17:43:50 -07:00
Jon Siwek
e45933562e Fix broken/missing documentation. 2013-05-23 16:53:42 -05:00
Seth Hall
bec965b66f Large update for the SumStats framework.
- On-demand access to sumstats results through "return from"
   functions named SumStats::request and Sumstats::request_key.
   Both functions are tested in standalone and clustered modes.

 - $name field has returned to SumStats which simplifies cluster
   code and makes the on-demand access stuff possible.

 - Clustered results can only be collected for 1 minute from their
   time of creation now instead of time of last read.

 - Thresholds use doubles instead of counts everywhere now.

 - Calculation dependency resolution occurs at start up time now
   instead of doing it at observation time which provide a minor
   cpu performance improvement.  A new plugin registration mechanism
   was created to support this change.

 - AppStats now has a minimal doc string and is broken into hook-based
   plugins.

 - AppStats and traceroute detection added to local.bro
2013-05-21 15:52:59 -04:00
Robin Sommer
eb637f9f3e Merge remote-tracking branch 'origin/master' into topic/robin/plugins
Thanks to git this merge was less troublesome that I was afraid it
would be. Not all tests pass yet though (and file hashes have changed
unfortunately).

Conflicts:
	cmake
	doc/scripts/DocSourcesList.cmake
	scripts/base/init-bare.bro
	scripts/base/protocols/ftp/main.bro
	scripts/base/protocols/irc/dcc-send.bro
	scripts/test-all-policy.bro
	src/AnalyzerTags.h
	src/CMakeLists.txt
	src/analyzer/Analyzer.cc
	src/analyzer/protocol/file/File.cc
	src/analyzer/protocol/file/File.h
	src/analyzer/protocol/http/HTTP.cc
	src/analyzer/protocol/http/HTTP.h
	src/analyzer/protocol/mime/MIME.cc
	src/event.bif
	src/main.cc
	src/util-config.h.in
	testing/btest/Baseline/coverage.bare-load-baseline/canonified_loaded_scripts.log
	testing/btest/Baseline/coverage.default-load-baseline/canonified_loaded_scripts.log
	testing/btest/Baseline/istate.events-ssl/receiver.http.log
	testing/btest/Baseline/istate.events-ssl/sender.http.log
	testing/btest/Baseline/istate.events/receiver.http.log
	testing/btest/Baseline/istate.events/sender.http.log
2013-05-16 17:58:48 -07:00
Robin Sommer
b9249ecf9d Layout tweaks for the sumstats code, and preliminary updates for NEWS.
The layout changes are mostly whitespace and some comment rewrapping.
No functional changes.
2013-04-28 15:35:21 -07:00
Seth Hall
04410237c2 Added protocol to the traceroute detection script. 2013-04-26 12:24:46 -04:00
Seth Hall
2c689b7f40 Removed some dead code in scan.bro 2013-04-22 15:27:14 -04:00
Seth Hall
1cac89e4f8 SumStats test checkpoint. 2013-04-16 00:54:41 -04:00
Seth Hall
fbe967e16a Checkpoint for SumStats rename. 2013-04-15 15:12:28 -04:00
Robin Sommer
52cd02173d Removing event groups. 2013-04-09 16:49:47 -07:00
Seth Hall
f1d165956a Fix path compression to include removing "/./".
- This involved a fix to the FTP scripts that relied on the old behavior.
2013-04-02 00:16:56 -04:00
Seth Hall
b477d2b02d Measurement framework is ready for testing.
- New, expanded API.
 - Calculations moved into plugins.
 - Scripts using measurement framework ported.
 - Updated the script-land queue implementation to make it more generic.
 -
2013-04-01 17:04:15 -04:00
Seth Hall
6dc204b385 Checkpoint, don't try running this. It's broken all over the place. 2013-03-19 11:39:58 -04:00
Seth Hall
8778761c07 Checkpoint 2013-03-13 22:55:03 -04:00
Seth Hall
9e1d5d87de New script to detect FTP bruteforcing.
- Created a new time utils library.
2013-01-04 23:34:57 -05:00
Seth Hall
ab7087f953 Hooks work now, enabling the hooks in the scan.bro script. 2013-01-04 21:01:49 -05:00
Seth Hall
6e9e3a5e88 Small fixes.
- Slight wording change in scan notices.

 - AppMetrics no longer writes to metrics.log.
2012-12-18 10:31:38 -05:00
Seth Hall
ed36f37643 A few more small fixes. 2012-12-18 02:23:56 -05:00
Seth Hall
7edef1f2c4 Disable the hook execution in the scan.bro script. It's not working like I expected. 2012-12-18 01:31:52 -05:00
Seth Hall
69b7ce12d2 API updates for metrics framework.
- Removed default logging.  Now a function is available for the new
  $period_finished filter field to get the same behavior for logging
  named Metrics::write_log.

- Added index rollups for getting multiple metrics result values
  as the same time.
2012-12-18 01:08:59 -05:00
Seth Hall
d0e8a6eef3 Comment updates and revised scan detection duration logging.
- Detection duration tracking is now logged in notices as 2m43s and
  only goes down to seconds.  Previously is was proceeding to milli-
  and micro seconds which aren't particularly useful.

- Inline docu-comment updates from Vlad Grigorescu.
2012-12-04 11:54:39 -05:00
Seth Hall
3af4517e2a Adding an $end time for result values to measure the length of time a measurement took. 2012-12-04 11:04:01 -05:00
Seth Hall
e769ab469f Comment and indentation cleanup. 2012-12-04 00:15:49 -05:00
Seth Hall
4bb8babb45 Small change to load the correct scan file in local.bro. 2012-12-03 14:58:11 -05:00
Seth Hall
1542b3696e Changed how traceroute detection works by having it check for low ttl packets after detecting time exceeded messages. 2012-11-30 11:27:09 -05:00
Seth Hall
bb7db64841 Fixed Sheharbano's name. 2012-11-30 09:51:20 -05:00
Seth Hall
96f850ca4e Moving scan.bro to a more appropriate place. 2012-11-30 09:49:16 -05:00
Seth Hall
47f5d256d8 Added a script module for detecting hosts doing traceroutes. 2012-11-20 01:01:37 -05:00
Seth Hall
257b460b18 Updated the app-metrics script to the new metrics api.
- Inconsequential change to scan.bro.
2012-11-16 03:05:43 -05:00