Commit graph

2206 commits

Author SHA1 Message Date
Robin Sommer
2818bb3dbc Merge remote-tracking branch 'origin/fastpath'
* origin/fastpath:
  make the documentation of the SQLite reader/writer a bit nicer.
  Wrong example file was included - reported by Michael Auger @LM4K
2013-10-25 11:12:51 -07:00
Robin Sommer
c980d1055e Fix for input readers occasionally dead-locking.
Bernhard and I tracked it down we believe: the thread queue could
deadlock in certain cases. As a fix we tuned the heuristic for telling
if a queue might have input to occasionaly err on the safe side by
flagging "yes", so that processing will proceed.

It's a bit unfortunate to apply this fix last minute before the
release as it could potentially impact performance if the heuristic
fails to often. We believe the chosen parmaterization should be fine ...
2013-10-24 18:21:24 -07:00
Robin Sommer
42c4a51da3 Updating submodule(s).
[nomail]
2013-10-24 16:59:31 -07:00
Robin Sommer
5c80fd169f Merge remote-tracking branch 'origin/fastpath'
* origin/fastpath:
  change temporary workaround to permanent fix.

Conflicts:
	src/input/readers/Ascii.cc
2013-10-24 16:33:11 -07:00
Robin Sommer
a7cebff99d Merge remote-tracking branch 'origin/topic/dnthayer/doc-changes-for-2.2'
* origin/topic/dnthayer/doc-changes-for-2.2:
  Add gawk to list of optional packages
  Add more script package README files
  Add NEWS about new features of broctl and upgrade info
2013-10-24 14:35:21 -07:00
Robin Sommer
1f91b2936e Merge remote-tracking branch 'origin/fastpath'
* origin/fastpath:
  rework libc++ patch to use AvailabilityMacros.h instead of Availability.h

Conflicts:
	src/input/readers/Ascii.cc
2013-10-24 14:01:44 -07:00
Robin Sommer
26c01468f5 Merge remote-tracking branch 'origin/fastpath'
* origin/fastpath:
  temporary osx mavericks libc++ issue workaround.
  Change test of identify_data BIF to ignore charset.
2013-10-24 13:07:49 -07:00
Robin Sommer
08e6b6b4b4 Merge remote-tracking branch 'origin/fastpath'
* origin/fastpath:
  Fix record coercion tolerance of optional fields.
  Add NEWS about incompatible local.bro changes, addresses BIT-1047.
  Fix minor formatting problem in NEWS.
2013-10-23 09:50:47 -07:00
Robin Sommer
02a414d1ba Merge remote-tracking branch 'origin/topic/dnthayer/doc-changes-for-2.2'
* origin/topic/dnthayer/doc-changes-for-2.2:
  Document which Bro script vars are set by BroControl
  Add more script package README files
  Fix doc comment so the text gets included in the HTML
  Do not include documentation of commented-out items
  Make RFC links in the docs more consistent
  Fix typos and formatting in the signature framework docs
  Fix typos and formatting in the notice framework docs
  Fix typos and formatting in the events docs
  Fix typos and formatting in the other policy docs
  Fix typos and formatting in the policy/protocols docs
  Fix typos and formatting in the policy/frameworks docs
  Correct more typos in init-bare.bro
  Fix typos and formatting in init-bare.bro and init-default.bro
  Fix formatting in the protocol BiFs docs
2013-10-23 09:48:22 -07:00
Robin Sommer
8bfb81ca6f Moving the SQLite examples into separate Bro files to turn them into
sphinx-btest tests.

It's hard to make sure these are actually working as expected, but the
tests now at least make sure things parse correctly.
2013-10-18 14:18:33 -07:00
Robin Sommer
615cca8baa Merge remote-tracking branch 'origin/topic/dnthayer/doc-changes-for-2.2'
Snapshotting the work in this branch. I'll merge it again later as we
get closer to the release.

* origin/topic/dnthayer/doc-changes-for-2.2: (29 commits)
  Add README files for base/protocols
  Fix incorrect uses of reST directives
  Fix typos and formatting in the BiFs docs
  Fix typos and formatting in the base/utils docs
  Fix typos and formatting in the other protocol docs
  Fix typos and formatting in the ssl protocol docs
  Fix typos and formatting in the http protocol docs
  Fix typos and formatting in the ftp protocol docs
  Fix typos and formatting in the dns protocol docs
  Fix typos and formatting in the dhcp protocol docs
  Adjust line numbers to match changes in conn/main.bro
  Fix typos and formatting in the conn protocol docs
  Update FreeBSD install instructions
  Improvements to file analysis docs
  Add README files for most Bro frameworks
  Fix typos and formatting in various other framework docs
  Fix typos and formatting in the software framework docs
  Fix typos and formatting in the sumstats docs
  Fix typos and formatting in the packet filter docs
  Fix typos and formatting in the logging framework docs
  ...
2013-10-18 13:49:49 -07:00
Robin Sommer
27a8c5e568 Merge remote-tracking branch 'origin/topic/bernhard/input-documentation'
* origin/topic/bernhard/input-documentation:
  and provide a bit of motivation to try the last example.
  and restructure it a bit
  First try at sqlite reader/writer documentation
  add check that the SQLite reader is only used in MANUAL reading mode
  rename the dbname configuration option to tablename.
2013-10-18 13:19:37 -07:00
Robin Sommer
2562b517b6 Merge remote-tracking branch 'origin/fastpath'
* origin/fastpath:
  Fix more Coverity-reported issues (introduced by internal error audit).
  Add check for sqlite3 command to tests that require it
  Fix misc. issues reported by Coverity.
2013-10-14 14:30:57 -07:00
Robin Sommer
2e05978210 Merge remote-tracking branch 'origin/fastpath'
* origin/fastpath:
  Add check for curl command to active-http.test

I've moved the check from TEST-EXEC to TEST-REQUIRES. I think it's
fine to just skip the test if curl is not available.
2013-10-14 09:27:14 -07:00
Robin Sommer
e5ad3c38c9 Merge branch 'topic/robin/internal-errors-merge'
* topic/robin/internal-errors-merge:
  Reverting one of the my internal errors tweaks.
  Renaming InternalAnalyzerError to AnalyzerError.
  Pass over the InternalError() changes.
  Review usage of Reporter::InternalError, addresses BIT-1045.
2013-10-14 09:21:05 -07:00
Robin Sommer
1a9835a00a Merge remote-tracking branch 'origin/fastpath'
* origin/fastpath:
  Improve Broxygen end-of-sentence detection.
2013-10-11 17:23:41 -07:00
Robin Sommer
3d010f3bb6 Merge remote-tracking branch 'origin/fastpath'
* origin/fastpath:
  IANA assigned a couple of new tls extension numbers
  add 3 more really new ciphers.
2013-10-10 13:37:19 -07:00
Robin Sommer
8c68161bea Merge remote-tracking branch 'origin/master' 2013-10-07 17:34:45 -07:00
Robin Sommer
d127d8d01d Merge remote-tracking branch 'origin/topic/jsiwek/coverity'
* origin/topic/jsiwek/coverity:
  Fix uninitialized (or unused) fields.
  Remove logically dead code.
  Remove dead/unfinished code in unary not expr.
  Fix logic for failed DNS TXT lookups.
  A couple null ptr checks.
  Improve return value checking and error handling.
  Remove unused variable assignments, dead code.
  Prevent division/modulo by zero in scripts.
  Fix unintentional always-false condition.
  Fix invalidated iterator usage.
  Fix DNS_Mgr iterator mismatch.
  Set safe umask when creating script profiler tmp files.
  Fix nesting/indent level whitespace mismatch.
  Add checks to avoid improper negative values use.

BIT-1085 #merged
2013-10-02 11:05:49 -07:00
Robin Sommer
ecd30a3d85 Merge remote-tracking branch 'origin/topic/bernhard/new-ciphers'
BIT-1086 #merged

* origin/topic/bernhard/new-ciphers:
  add ciphers to cipher_desc table that were already present but apparently forgotten.
  ...and a few more ciphers.
  add new tls cipher suites
2013-10-02 10:29:34 -07:00
Robin Sommer
6c8cbef997 Updating bifcl usage message. 2013-10-01 11:32:37 -07:00
Robin Sommer
5ddbe9e75f Merge remote-tracking branch 'origin/fastpath'
* origin/fastpath:
  Fix a "make doc" warning
2013-09-28 11:17:10 -07:00
Robin Sommer
bb4b262813 Updating submodule(s).
[nomail]
2013-09-24 13:24:06 -07:00
Robin Sommer
abba39c261 Updating submodule(s).
[nomail]
2013-09-23 20:31:57 -07:00
Robin Sommer
00a3ad03dd Updating submodule(s).
[nomail]
2013-09-23 11:55:12 -07:00
Robin Sommer
d070b01828 Updating baselines for doc changes. 2013-09-20 14:38:45 -07:00
Robin Sommer
589a0239be Merge remote-tracking branch 'origin/topic/srunnels/documentation'
* origin/topic/srunnels/documentation:
  Spelling corrections.
  Include a better description for detect-MHR.bro
  Rewrite the MHR detection description.
  Spelling corrections.
  Update the lines included from events.bif.bro.
2013-09-20 14:36:55 -07:00
Robin Sommer
97cb04aae2 Merge remote-tracking branch 'origin/topic/jsiwek/raw-exec-pgrp'
* origin/topic/jsiwek/raw-exec-pgrp:
  Kill raw input reader's child by process group.

BIT-1081 #merged
2013-09-20 14:12:59 -07:00
Robin Sommer
0316bd77f5 Merge remote-tracking branch 'origin/fastpath'
* origin/fastpath:
  Add more links in the GeoLocation document
2013-09-19 20:08:22 -07:00
Robin Sommer
d123d95b46 Merge remote-tracking branch 'origin/fastpath'
* origin/fastpath:
  Add links to Intelligence Framework documentation
  update mozilla root ca list
  Update Mozilla root CA certs.
  Update documentation of required packages
2013-09-19 15:12:46 -07:00
Robin Sommer
3738b4136b Merge remote-tracking branch 'origin/topic/bernhard/ticket950'
* origin/topic/bernhard/ticket950:
  make client and server random available on script-level.

BIT-950 #merged
2013-09-18 15:02:55 -07:00
Robin Sommer
c6de23ebe1 Merge remote-tracking branch 'origin/topic/bernhard/ticket1072'
* origin/topic/bernhard/ticket1072:
  and const 2 more functions
  update hll documentation, make a few functions private and create a new copy constructor.
  fix case where hll_error_margin could be undefined (thanks John)

BIT-1072 #merged
2013-09-18 15:00:06 -07:00
Robin Sommer
6e1f01f04c Merge remote-tracking branch 'origin/topic/dnthayer/compilerwarn'
* origin/topic/dnthayer/compilerwarn:
  Fix compiler warning (time_t is not a pointer type)
  Fix cmake warning about ENABLE_PERFTOOLS not being used
  Fix another compiler warning
  Fix compiler warnings

BIT-1079 #merged
2013-09-18 14:42:58 -07:00
Robin Sommer
9d3fc1e7ed Updating CHANGES and VERSION. 2013-09-18 14:36:59 -07:00
Jon Siwek
bcf79fe11a Merge branch 'topic/dnthayer/doc-fix-links'
BIT-1076 #merged
2013-09-12 16:40:28 -05:00
Robin Sommer
ac66db286a Merge remote-tracking branch 'origin/topic/documentation'
I'm merging in the documentation branch to avoid that it keeps getting
out of sync. We still need to work on this a bit more, I'll summarize
in a mail.

* origin/topic/documentation: (68 commits)
  Going over initial parts of the scripting overview.
  Adding lines in scripting/index.txt. No other change.
  Updating submodule.
  Pass over the Using Bro section.
  Applying doc updates from BIT-1063. (Anthony Verez)
  Breaking lines in using/*.
  More doc reorg, and a light pass over the first 3 sections.
  Starting to put a new structure in place.
  Updating submodule(s).
  Updating submodule(s).
  Adding some temporary testing/demonstration to front page.
  Switching btest to topic/robin/parts branch.
  Basic cross-referencing UIDs between files, btests, and baselines.
  A truly embarassing number of spelling corrections.
  Using redirection with bro-cut.  Include initial btests for this document.
  Corrected a mis-typed RST include.
  Initial start for "Using Bro".
  Spelling corrections.
  Include Notice Policy shortcuts in the Scripting User Manual.
  Notice::policy hooks and tests.
  ...

Conflicts:
	src/3rdparty
2013-08-31 16:07:44 -07:00
Robin Sommer
6f9d28cc18 Merge branch 'topic/robin/hyperloglog-merge'
* topic/robin/hyperloglog-merge: (35 commits)
  Making the confidence configurable.
  Renaming HyperLogLog->CardinalityCounter.
  Fixing bug introduced during merging.
  add clustered leak test for hll. No issues.
  make gcc happy
  (hopefully) fix refcounting problem in hll/bloom-filter opaque vals. Thanks Robin.
  re-use same hash class for all add operations
  get hll ready for merging
  and forgot a file...
  adapt to new structure
  fix opaqueval-related memleak.
  make it compile on case-sensitive file systems and fix warnings
  make error rate configureable
  add persistence test not using predetermined random seeds.
  update cluster test to also use hll
  persistence really works.
  well, with this commit synchronizing the data structure should work.. ...if we had consistent hashing.
  and also serialize the other things we need
  ok, this bug was hard to find.
  serialization compiles.
  ...
2013-08-31 10:42:42 -07:00
Robin Sommer
23144e44a7 Merge remote-tracking branch 'origin/fastpath'
* origin/fastpath:
  Fix global opaque val segfault, addresses BIT-1071
  Fix malloc/delete mismatch.
  Fix invalid pointer dereference in AsciiFormatter.
2013-08-30 08:28:18 -07:00
Robin Sommer
c8a2fbeadd Merge remote-tracking branch 'origin/fastpath'
Changed two deletes to Unrefs.

* origin/fastpath:
  Fix input framework memory leaks.
  Fix mem leak in socks analyzer for bad addr types.
  Fix bloom filter memory leaks.
2013-08-29 14:20:59 -07:00
Robin Sommer
6373d817a5 Merge remote-tracking branch 'origin/topic/jsiwek/uid'
Thanks!

* origin/topic/jsiwek/uid:
  Add bits_per_uid unit test, addressing BIT0-1016.
  UID optimizations addressing BIT-1016.

BIT-1016 #merged
2013-08-28 18:57:00 -07:00
Robin Sommer
f46f692185 Merge remote-tracking branch 'origin/topic/bernhard/hexstr'
BIT-1069 #merged

* origin/topic/bernhard/hexstr:
  add hexstr_to_bytestring bif that does exactly the opposite of bytestring_to_hexstr.
2013-08-27 13:30:25 -07:00
Robin Sommer
f6b689db81 Merge remote-tracking branch 'origin/topic/jsiwek/uid'
* origin/topic/jsiwek/uid:
  Fix UID compiler warning/error & missed baselines.
  Increase UIDs to 96 bits w/ C/F prefix - BIT-1016
2013-08-27 12:36:12 -07:00
Robin Sommer
df84083227 Merge remote-tracking branch 'origin/fastpath'
* origin/fastpath:
  Remove code relict pointed out by Bernhard.
  Fix wrong documentation for mkdir BIF.
  File extraction tweaks.
2013-08-24 10:29:07 -07:00
Robin Sommer
391bfdc317 Merge remote-tracking branch 'origin/topic/hui/dnp3-merge-v2'
* origin/topic/hui/dnp3-merge-v2:
  fixed number of object bug in dnp3-protocol pac; update two base test trases
2013-08-23 16:34:26 -07:00
Robin Sommer
927f534833 Merge remote-tracking branch 'origin/topic/jsiwek/extract-limit'
* origin/topic/jsiwek/extract-limit:
  Use macros to create file analyzer plugin classes.
  Add options to limit extracted file sizes w/ 100MB default.
2013-08-22 16:53:06 -07:00
Robin Sommer
40d849a2c5 Updating CHANGES and VERSION. 2013-08-22 08:47:06 -07:00
Robin Sommer
ab8d13889e Merge remote-tracking branch 'origin/topic/matthias/bloom-filter'
* origin/topic/matthias/bloom-filter:
  Use Bro-style platform-independent integer types.
  Change bloom filter's dependence on size_t.
  Remove debugging code.
  Update baseline with now correct FP tests.
  Add debugging code to find FP inconsistency.

Conflicts:
	src/3rdparty
2013-08-19 11:26:29 -07:00
Robin Sommer
95f74313d0 Merge branch 'master' of https://github.com/anthonykasza/bro
* 'master' of https://github.com/anthonykasza/bro:
  levenshtein distance function unit test
  levenshtein distance

Conflicts:
	src/3rdparty
2013-08-19 11:20:50 -07:00
Robin Sommer
be1f582508 Merge remote-tracking branch 'origin/topic/bernhard/3rdparty'
Note this is another submodule change, you'll need to init the new
module:

    git submodule update --recursive --init

BIT-1059 #merged

* origin/topic/bernhard/3rdparty:
  add external 3rdparty submodule
  step 1 - delete 3rdparty
2013-08-19 09:32:47 -07:00
Robin Sommer
f7a31ab004 Merge remote-tracking branch 'origin/fastpath'
* origin/fastpath:
  Fix timer type enum and timer name array mismatch.
2013-08-14 10:30:03 -07:00