Commit graph

708 commits

Author SHA1 Message Date
Seth Hall
0e3c84e863 Fixed the measurement "sample" plugin. 2013-04-02 00:19:06 -04:00
Seth Hall
f1d165956a Fix path compression to include removing "/./".
- This involved a fix to the FTP scripts that relied on the old behavior.
2013-04-02 00:16:56 -04:00
Seth Hall
d11a1dab73 Removed the example metrics scripts. Better real world examples exist now. 2013-04-02 00:15:55 -04:00
Seth Hall
b477d2b02d Measurement framework is ready for testing.
- New, expanded API.
 - Calculations moved into plugins.
 - Scripts using measurement framework ported.
 - Updated the script-land queue implementation to make it more generic.
 -
2013-04-01 17:04:15 -04:00
Seth Hall
93eca70e6b Merge remote-tracking branch 'origin/master' into topic/seth/metrics-merge 2013-04-01 14:16:46 -04:00
Seth Hall
53f9948b02 Measurement framework tests all pass now. 2013-04-01 14:16:37 -04:00
Seth Hall
6dc204b385 Checkpoint, don't try running this. It's broken all over the place. 2013-03-19 11:39:58 -04:00
Robin Sommer
d58a02aa01 Merge remote-tracking branch 'origin/topic/bernhard/base64'
* origin/topic/bernhard/base64:
  and re-enable caching of extracted certs
  and add bae64 bif tests.
  re-unify classes
  and modernize script.
  add base64-encode functionality and bif.

Closes #965.
2013-03-17 13:00:52 -07:00
Robin Sommer
788c0d547d Merge remote-tracking branch 'origin/topic/seth/software-version-updates2'
* origin/topic/seth/software-version-updates2:
  Correctly handle DNS lookups for software version ranges.
  Improvements to vulnerable software detection.
  Update software version parsing and comparison to account for a third numeric subversion.

Closes #938.
2013-03-17 12:38:25 -07:00
Seth Hall
5734f9ef71 Merge remote-tracking branch 'origin/master' into topic/seth/software-version-updates2 2013-03-14 09:21:13 -04:00
Seth Hall
8778761c07 Checkpoint 2013-03-13 22:55:03 -04:00
Bernhard Amann
457ce10e99 and re-enable caching of extracted certs
I kind of deleted the line by accident...
2013-03-13 00:34:15 -07:00
Seth Hall
09cbaa7ccc Merge remote-tracking branch 'origin/master' into topic/seth/metrics-merge
Conflicts:
	testing/btest/Baseline/coverage.default-load-baseline/canonified_loaded_scripts.log
	testing/btest/Baseline/scripts.base.frameworks.notice.cluster/manager-1.notice.log
	testing/btest/Baseline/scripts.base.frameworks.notice.suppression-cluster/manager-1.notice.log
	testing/btest/Baseline/scripts.base.protocols.ftp.gridftp/notice.log
2013-03-13 00:26:55 -04:00
Bernhard Amann
a1896fde90 Merge remote-tracking branch 'origin/master' into topic/bernhard/base64 2013-03-12 09:28:07 -07:00
Robin Sommer
f830ed3edf s/bro-ids.org/bro.org/g 2013-03-07 19:33:04 -08:00
Robin Sommer
8a6d68e00f Merge remote-tracking branch 'origin/topic/bernhard/remove-length'
Closes #955.

* origin/topic/bernhard/remove-length:
  forgot to remove the baselines for the now unnecessary bifs
  remove the byte_len and length bifs
2013-03-06 16:46:20 -08:00
Robin Sommer
1bd2f26df3 Merge remote-tracking branch 'origin/topic/seth/notice-framework-updates'
So much nicer!

Closes #954.

* origin/topic/seth/notice-framework-updates:
  Update notice framework documentation to represent the new reality.
  Complete removal of the old table based notice policy mechanism.
  Updates for the notices framework.
2013-03-06 16:45:30 -08:00
Bernhard Amann
986b346e3f remove the byte_len and length bifs 2013-03-06 13:45:42 -08:00
Bernhard Amann
cfada61672 and modernize script.
thanks Seth.
2013-03-06 13:30:13 -08:00
Seth Hall
00eeadf2f5 Complete removal of the old table based notice policy mechanism. 2013-03-06 09:44:25 -05:00
Bernhard Amann
5e8e12182a add base64-encode functionality and bif.
This allows replacing an ugly openssl-call from one of
the policy scripts. The openssl call is now replaced with
a still-but-less-ugly call to base64_encode.

I do not know if I split the Base64 classes in a "smart" way... :)
2013-03-05 16:05:07 -08:00
Seth Hall
e3856d7681 Removing a field that is no longer logged through the standard metrics log. 2013-02-27 11:25:01 -05:00
Seth Hall
9f8ba408ba Updates for the notices framework.
- Moved the Notice::notice event and Notice::policy table to both be hooks.

 - Renamed the old Notice::policy to Notice::policy_table and documented it as deprecated.
2013-02-11 14:36:14 -05:00
Jon Siwek
26bf99c5a3 Add parsing for GTPv1 extension headers and control messages.
Added a generic gtpv1_message event generated for any GTP message type.

Added specific events for the create/update/delete PDP context
request/response messages.

Addresses #934.
2013-02-07 14:59:02 -06:00
Seth Hall
f8be65ca56 Merge remote-tracking branch 'origin/master' into topic/seth/metrics-merge 2013-02-07 10:01:13 -05:00
Seth Hall
e974950c6d Correctly handle DNS lookups for software version ranges. 2013-02-06 13:59:17 -05:00
Seth Hall
ff3ec04f32 Improvements to vulnerable software detection.
- Add a DNS based updating method.  This needs to be tested still.

 - Vulnerable version ranges are used now instead
   of only single versions.  This can deal with
   software with multiple stable major versions.
2013-02-06 13:59:12 -05:00
Seth Hall
ad9e2a143c Update software version parsing and comparison to account for a third numeric subversion.
- $addl is also compared numerically now if the value is actually numeric.
2013-02-06 13:59:05 -05:00
Seth Hall
a2556642e6 Merge remote-tracking branch 'origin/topic/matthias/notary'
* origin/topic/matthias/notary:
  Small cosmetic changes.
  Give log buffer the correct name.
  Simplify delayed logging of SSL records.
  Implement delay-token style SSL logging.
  More style tweaks: replace spaces with tabs.
  Factor notary code into separte file.
  Adhere to Bro coding style guidelines.
  Enhance ssl.log with information from notary.

Closes #928
2013-02-05 02:06:33 -05:00
Robin Sommer
52f52f60a5 Merge branch 'topic/robin/exit-after-terminate'
* topic/robin/exit-after-terminate:
  Updating submodule(s).
  Fixing exit-after-terminate when used with bare mode.
  New option exit_only_after_terminate to prevent Bro from exiting.
2013-02-04 09:44:33 -08:00
Jon Siwek
69afc4a882 Add an error for record coercions that would orphan a field.
These cases should be avoidable by fixing scripts where they occur and
they can also help catch typos that would lead to unintentional runtime
behavior.

Adding this already revealed several scripts where a field in an inlined
record was never removed after a code refactor.
2013-01-24 09:56:19 -06:00
Robin Sommer
762c034ec2 Merge remote-tracking branch 'origin/topic/bernhard/input-logging-commmon-functions'
* origin/topic/bernhard/input-logging-commmon-functions:
  add the last of Robins suggestions (separate info-struct for constructors).
  port memory leak fix from master
  harmonize function naming
  move AsciiInputOutput over to threading
  and thinking about it, ascii-io doesn't need the separator
  change constructors
  and factor stuff out the input framework too.
  factor out ascii input/output.
  std::string accessors to escape_sequence functionality
  intermediate commit - it has been over a month since I touched this...

I cleaned up the AsciiInputOutput class somewhat, including renaming
it to AsciiFormatter, renaming some of its methods, and turning the
static methods into members for consistency.

Closes #929.
2013-01-23 16:51:54 -08:00
Robin Sommer
4f39470c1b New option exit_only_after_terminate to prevent Bro from exiting.
If set, the main loop won't terminate before somebody calls
terminate().

This should make input framework testing more reliable I'd hope.
2013-01-18 17:34:33 -08:00
Jon Siwek
fdd11428c1 Change reporter messages to more reliably print to stderr.
Moved this functionality to be internal instead of in the script-layer
event handlers.  The issue with the later is that bad things can happen
between the time a reporter event handler is dispatched and the time it
is executed, and if bro crashes in that time, the message may never be
seen/logged.

Addressed #930 (and revisits #836).
2013-01-18 12:46:00 -06:00
Matthias Vallentin
a635f96518 Small cosmetic changes. 2013-01-15 14:56:02 -08:00
Bernhard Amann
1b0bb5063a Merge remote-tracking branch 'origin/master' into topic/bernhard/input-logging-commmon-functions
Conflicts:
	src/input/readers/Ascii.cc
2013-01-11 09:48:53 -08:00
Seth Hall
720089c03f Add a feature to better select the value threshold will apply to in the metrics framework.
- The feature was primarily added to allow the value to be
   modified for cluster based intermediate threshold checks
   without requiring the user to write the metrics filter
   differently for cluster consideration.  It's also a nice
   way to calculate some related information to the metric
   without accidently applying thresholds to that value.

 - Fixed a few small bugs in ftp detect-bruteforcing script
   and adapted it to the new threshold value selection feature.
2013-01-05 22:37:19 -05:00
Seth Hall
bcd7fe114d Fixed an issue causing reporter messages from the metrics framework. 2013-01-05 22:27:17 -05:00
Seth Hall
9366411cf4 Fix the FTP bruteforce threshold to what it's really supposed to be. 2013-01-04 23:49:09 -05:00
Seth Hall
283f7840b3 Removing some debugging print statements I accidently left behind. 2013-01-04 23:38:10 -05:00
Seth Hall
9e1d5d87de New script to detect FTP bruteforcing.
- Created a new time utils library.
2013-01-04 23:34:57 -05:00
Seth Hall
ab7087f953 Hooks work now, enabling the hooks in the scan.bro script. 2013-01-04 21:01:49 -05:00
Seth Hall
89bc65f23c Merge remote-tracking branch 'origin/master' into topic/seth/metrics-merge 2013-01-04 20:49:37 -05:00
Seth Hall
c3a6916572 More cluster tuning for the metrics framework.
- Fixed several state maintenance issues for intermediate updates.

 - Added a new tuning variable Metrics::max_outstanding_global_views
   which limits the number of in-flight intermediate updates per
   metric filter.

 - Changed the default global view threshold percent to 20% (up from 10%)
2013-01-04 16:54:13 -05:00
Matthias Vallentin
32a0ead698 Give log buffer the correct name. 2012-12-24 23:06:56 -08:00
Matthias Vallentin
7ff15f4599 Simplify delayed logging of SSL records. 2012-12-24 22:57:49 -08:00
Matthias Vallentin
9e81342c92 Implement delay-token style SSL logging.
This commit moves the notary script into the policy directory, along with some
architectural changes: the main SSL script now has functionality to add and
remove tokens for a given record. When adding a token, the script delays the
logging until the token has been removed or until the record exceeds a maximum
delay time.

As before, the base SSL script stores all records sequentially and buffers even
non-delayed records for the sake of having an ordered log file. If this turns
out to be not so important, we can easily revert to a simpler logic.

(This is still WiP, some debuggin statements still linger.)
2012-12-22 20:30:17 -08:00
Seth Hall
50827d8df0 Implement an option to disable intermediate updates for the metrics framework.
- There are some large sites having trouble and I'm thinking it
   might be overload from intermediate updates.
2012-12-21 23:17:27 -05:00
Matthias Vallentin
8a569facd6 More style tweaks: replace spaces with tabs. 2012-12-21 18:04:19 -08:00
Matthias Vallentin
382262e286 Factor notary code into separte file.
There exists one complication: the new file notary.bro requires the definition
of the SSL::Info record, but as does main.bro. Because I did not really know
where to put the common code (it's not a constant, so ssl/const.bro does not
really fit), I put it into __load.bro__ so that it sticks out for now. If
anybody has an idea how to solve this elegantly, please let me know.
2012-12-21 17:56:31 -08:00