Commit graph

40 commits

Author SHA1 Message Date
Jon Siwek
69afc4a882 Add an error for record coercions that would orphan a field.
These cases should be avoidable by fixing scripts where they occur and
they can also help catch typos that would lead to unintentional runtime
behavior.

Adding this already revealed several scripts where a field in an inlined
record was never removed after a code refactor.
2013-01-24 09:56:19 -06:00
Robin Sommer
da90976170 Merge remote-tracking branch 'origin/topic/matthias/opaque'
* origin/topic/matthias/opaque:
  Add new unit test for opaque serialization.
  Migrate entropy testing to opaque.
  C++ify RandTest.*
  Fix a hard-to-spot bug.
  Use more descriptive error message.
  Fix the fix :-/.
  Fix initialization of hash values.
  Be clearer about delegation.
  Implement serialization of opaque types.
  Update hash BiF documentation.
  Migrate free SHA* functions to SHA*Val::digest().
  Add missing type name that caused failing tests.
  Update base scripts and unit tests.
  Simplify hash function BiFs.
  Add support for opaque hash values.
  Adapt BiF & Bro parser to handle opaque types.
  More lexer/parser work.
  Implement equivalence relation for opaque types.
  Support basic serialization of opaque.
  Add opaque type to lexer, parser, and BroType.

Closes #925

Conflicts:
	aux/broccoli
2012-12-20 16:30:22 -08:00
Matthias Vallentin
816965f3c7 Merge remote-tracking branch 'origin/master' into topic/matthias/opaque 2012-12-11 16:32:01 -08:00
Matthias Vallentin
30bab14dbf Update base scripts and unit tests. 2012-12-11 16:26:17 -08:00
Robin Sommer
57510464a1 Adapting the HTTP request line parsing to only accept methods
consisting of letters [A-Za-z].

I had some bogus HTTP sessions now with the test-suite that reported
data as HTTP because it started with "<!... ". Requiring letters seems
a reasonable constraint.
2012-12-05 16:56:54 -08:00
Robin Sommer
177c014cb7 Merge remote-tracking branch 'vlad/topic/vladg/http-verbs'
* vlad/topic/vladg/http-verbs:
  A test for HTTP methods, including some horribly illegal requests.
  Remove hardcoded HTTP verbs from the analyzer (#741)

I added a "bad_HTTP_request" weird for HTTP request lines that don't
have more than a single word.

Closes #741.
2012-12-05 15:27:42 -08:00
Vlad Grigorescu
e98343b562 Remove hardcoded HTTP verbs from the analyzer (#741) 2012-11-30 20:08:20 -05:00
Robin Sommer
ce4b8dd4ac Changing HTTP DPD port 3138 to 3128.
Addresses #857.
2012-07-20 09:57:38 -07:00
Vlad Grigorescu
f43576cff3 Fix some Info:Record field documentation. 2012-07-13 14:04:24 -04:00
Jon Siwek
dd4dd0ca6e Add @load-sigs directive for loading signature files (addresses #551). 2012-06-01 14:10:23 -05:00
Jon Siwek
905e4d3a14 Change IPv6 output format to no longer automatically be bracketed.
Instead, the `addr_to_uri` script-level function can be used to
explicitly add brackets to an address if it's IPv6 and will be
included in a URI or when a ":<port>" needs to be appended to it.
2012-05-07 12:55:54 -05:00
Robin Sommer
3d2dc5f5fc Merge remote-tracking branch 'origin/topic/script-reference'
* origin/topic/script-reference: (50 commits)
  A few updates for the FAQ.
  Fixing some doc warnings.
  Forgot to add protocol identifier support for TLS 1.2
  Finished SSL & syslog autodocs.
  Adding the draft SSL extension type next_protocol_negotiation.
  Fix some documentation errors.
  Tweaks.
  A set of script-reference polishing.
  fixed a couple typos in comments
  Add summary documentation to bif files.
  Add ssl and syslog script documentation
  Add Conn and DNS protocol script documentation. (fixes #731)
  Small updates to the default local.bro.
  Documentation updates for HTTP & IRC scripts.
  SSH&FTP Documentation updates.
  Fixing a warning from the documentation generation.
  This completes framework documentation package 4.
  Minor notice documentation tweaks.
  Fix some malformed Broxygen xref roles.
  Minor doc tweaks to init-bare.bro.
  ...

Conflicts:
	aux/broccoli
	aux/broctl
	src/bro.bif
	src/strings.bif

Includes:

    - Updated baselines for autodoc tests.
    - Now excluding stats.bro from external texts, it's not stable.
2012-01-10 14:00:44 -08:00
Seth Hall
727e626bb4 Added an option for filtering out urls before they are turned into HTTP::Incorrect_File_Type notices 2012-01-10 10:38:12 -05:00
Seth Hall
3be1222532 Documentation updates for HTTP & IRC scripts.
Closes #733
2012-01-08 02:22:52 -05:00
Robin Sommer
fa76330afb Merge remote-tracking branch 'origin/fastpath'
* origin/fastpath:
  Binary packaging script tweaks.
  More default "weird" tuning for the "SYN_with_data" notice.
  Tiny bugfix for http file extraction along with test.
2011-11-15 07:53:36 -08:00
Seth Hall
d14349a6f8 Merge remote-tracking branch 'origin/master' into fastpath 2011-11-14 16:06:44 -05:00
Seth Hall
b12d2c768e Tiny bugfix for http file extraction along with test. 2011-11-14 15:24:15 -05:00
Seth Hall
4753f2aeca Adding extra fields to smtp and http to track transaction depth.
- This will for help linking in analysis scripts and databases later.

- Test baseline updates coming in a few minutes.
2011-10-25 11:34:48 -04:00
Seth Hall
dcc8d8456a Removed some fields from http analysis that weren't commonly needed or were wrong. 2011-10-25 09:32:31 -04:00
Seth Hall
8b56c54348 Slightly restructured http file hashing to fix a bug. 2011-10-21 14:03:31 -04:00
Seth Hall
6d67f7830d Added to the likely_server_ports set for protocols with analyzers.
- Updated some tests since Bro is getting the direction
  correct now.

- Updated BPF filter test since I added a few ports to IRC
  as well.
2011-10-07 13:44:28 -04:00
Jon Siwek
c9a540b992 Add check for optional HTTP::Info status_code. 2011-10-04 14:27:51 -05:00
Robin Sommer
221d1663be Merge branch 'master' of ssh://git.bro-ids.org/bro
Conflicts:
	scripts/base/protocols/http/main.bro
2011-09-29 18:54:50 -07:00
Seth Hall
012d8cfc5f Fix for shutdown bug in http scripts.
- The bug was introduced with the recent 1xx update.

- I updated some tests that seemed be written wrong.
2011-09-29 21:25:00 -04:00
Robin Sommer
f7521ad222 Fixing occasional HTTP crash with new 1xx code.
Sometimes the status_code field isn't set. Adding check for that, hope
that's all that needed.
2011-09-29 16:18:25 -07:00
Robin Sommer
4d6a90ce89 Merge remote-tracking branch 'origin/topic/jsiwek/http-1xx-replies'
* origin/topic/jsiwek/http-1xx-replies:
  Change logging of HTTP 1xx responses to occur in their own columns.
  Fix handling of HTTP 1xx response codes (addresses #411).
2011-09-28 17:10:40 -07:00
Seth Hall
c621da523b Since these now measure "seen" data, set the default to 0.
- A null value no longer fits since if there is no body
  a value of zero makes sense.  Previously, a null value would
  makes sense because the Content-Length header may not have
  been sent which would leave the field null.
2011-09-28 11:18:24 -04:00
Jon Siwek
7af3977a50 Change logging of HTTP 1xx responses to occur in their own columns.
Instead of as entirely new log lines (addresses #411).
2011-09-27 14:15:23 -05:00
Seth Hall
19f1e34408 Deleting scripts that aren't ready to be included.
- scan.bro and hot.conn.bro will be returning soon.

- The rest are going to return as updated protocol analysis
  scripts and new/updated frameworks later.
2011-09-27 14:40:11 -04:00
Jon Siwek
64e821624b Fix handling of HTTP 1xx response codes (addresses #411).
Changed the parser to not treat 1xx response codes as a final answer
to an unanswered request -- a later response is still expected.

The scripting layer will also not finish a request-reply pair when
seeing 1xx's, instead it logs both the 1xx and final response messages
with associated information of the current request as they're seen.
2011-09-26 17:37:29 -05:00
Seth Hall
3449321dc1 Fix a problem with accidental and mistaken HTTP log lines. 2011-09-25 00:56:53 -04:00
Seth Hall
af6c7c8b1a HTTP body size measurement added to http log.
- The value of the content-length headers has now been removed
  but it could be added back locally at an installation by a user.

- Added fields to indicate if some parsing interruption happened
  during the body transfer.

- Closes #581
2011-09-13 21:34:29 -04:00
Seth Hall
11c437faa3 Logging framework update and mass Log::ID renaming.
- Log path's are generated in the scripting land
  now.  The default Log stream ID to path string
  mapping works like this:
    - Notice::LOG -> "notice"
    - Notice::POLICY_LOG -> "notice_policy"
    - TestModule::LOG -> "test_module"

- Logging streams updated across all of the shipped
  scripts to be more user friendly.  Instead of
  the logging stream ID HTTP::HTTP, we now have
  HTTP::LOG, etc.

- The priorities on some bro_init handlers have
  been adjusted to make the process of applying
  filters or disabling streams easier for users.
2011-09-03 01:10:17 -04:00
Seth Hall
fc5f22cb5d Merge remote-tracking branch 'origin/topic/jsiwek/reorg-followup' 2011-08-25 16:44:31 -04:00
Jon Siwek
ed281fb634 Fixing some more bare-mode @load dependency issues 2011-08-11 10:49:41 -05:00
Jon Siwek
f517d0e0ad Merge branch 'master' into topic/jsiwek/reorg-followup 2011-08-10 19:59:18 -05:00
Jon Siwek
9e7934dc32 Normalize some intra-package @loads 2011-08-10 16:06:05 -05:00
Jon Siwek
47500ceef4 Add a test that checks each individual script can be loaded in bare-mode.
Fixed most @load dependency issues in the process.  The test is still
failing in a "known" way due to hot.conn.bro and scan.bro.

Adressess #545
2011-08-10 15:38:21 -05:00
Seth Hall
8e7a76b548 HTTP now uses the extract_filename_from_content_disposition function. 2011-08-10 13:37:58 -04:00
Seth Hall
597a4d6704 Hopefully the last major script reorganization.
- policy/ renamed to scripts/

- By default BROPATH now contains:
	- scripts/
	- scripts/policy
	- scripts/site

- *Nearly* all tests pass.

- All of scripts/base/ is loaded by main.cc
	- Can be disabled by setting $BRO_NO_BASE_SCRIPTS
	- Scripts in scripts/base/ don't use relative path loading to ease use of BRO_NO_BASE_SCRIPTS (to copy and paste that script).

- The scripts in scripts/base/protocols/ only (or soon will only) do logging and state building.

- The scripts in scripts/base/frameworks/ add functionality without causing any additional overhead.

- All "detection" activity happens through scripts in scripts/policy/.

- Communications framework modified temporarily to need an environment variable to actually enable (ENABLE_COMMUNICATION=1)
	- This is so the communications framework can be loaded as part
	  of the base without causing trouble when it's not needed.
	- This will be removed once a resolution to ticket #540 is reached.
2011-08-05 23:09:53 -04:00