Commit graph

360 commits

Author SHA1 Message Date
Johanna Amann
fe60d5e9dd Split dhcp log writing from record creation.
This allows users to customize dhcp.log by changing the record in their own
dhcp_ack event.
2014-08-01 11:07:32 -07:00
Jon Siwek
b4b64c1239 Merge remote-tracking branch 'origin/topic/robin/smtp-fix'
* origin/topic/robin/smtp-fix:
  Fixing SMTP state tracking.

BIT-1203 #merged
2014-06-11 15:38:29 -05:00
Robin Sommer
9301ef5a4f Fixing SMTP state tracking.
This fixes the case that an SMTP session has multiple mails sent from
the originator but we miss the server's response (e.g., because we
don't see server side packets at all).
2014-06-10 18:01:38 -07:00
Bernhard Amann
67c0cc118d Add two more ssl events - one triggered for each handshake message and one
triggered for the tls change cipherspec message.

Also - fix small bug. In case SSL::disable_analyzer_after_detection was set
to F, the ssl_established event would fire after each data packet after the
session is established.
2014-06-06 12:50:54 -07:00
Bernhard Amann
85f5c05b95 add new TLS extension type numbers from IANA 2014-06-05 13:17:52 -07:00
Jon Siwek
7211d73ee6 Merge remote-tracking branch 'origin/fastpath'
* origin/fastpath:
  last ssl fixes - missed three more.
  and more tiny ssl script fixes
  a few more small fixes for chains containing broken certs.
  fix expression errors in x509 policy scrips when unparseable data is in certificate chain.
2014-05-21 15:59:26 -05:00
Bernhard Amann
9a8fc7a47d and more tiny ssl script fixes 2014-05-21 11:16:24 -07:00
Bernhard Amann
ff00c0786a a few more small fixes for chains containing broken certs. 2014-05-21 11:01:33 -07:00
Robin Sommer
ed4cd9352a Merge remote-tracking branch 'origin/topic/bernhard/even-more-ssl-changes'
Good stuff! (but I admit I didn't look at the OpenSSL code too closely :)

* origin/topic/bernhard/even-more-ssl-changes:
  small test update & script fix
  update baselines & add ocsp leak check
  Add policy script adding ocsp validation to ssl.log
  Implement verification of OCSP replies.
  Add tls flag to smtp.log. Will be set if a connection switched to startls.
  add starttls support for pop3
  Add smtp starttls support
  Replace errors when parsing x509 certs with weirds (as requested by Seth).
  move tls content types from heartbleed to consts.bro. Seems better to put them there...
  Add new features from other branch to the heartbleed-detector (and clean them up).
  Let TLS analyzer fail better when no longer in sync with the data stream. The version field in each record-layer packet is now re-checked.

BIT-1190 #merged

Conflicts:
	testing/btest/Baseline/scripts.policy.misc.dump-events/all-events.log
	testing/btest/Baseline/scripts.policy.misc.dump-events/smtp-events.log
2014-05-16 14:45:25 -07:00
Robin Sommer
525e757d2a Merge remote-tracking branch 'origin/topic/vladg/radius' into topic/robin/radius-merge
* origin/topic/vladg/radius:
  Radius functionality and memleak test.
  Update test baselines.
  Move seq to uint64 to match recent changes in seq processing.

BIT-1129 #merged
2014-05-15 11:39:05 -07:00
Robin Sommer
ebc8ebf5f9 Merge remote-tracking branch 'origin/master' into topic/robin/radius-merge
Conflicts:
	scripts/base/init-default.bro
2014-05-15 11:10:11 -07:00
Bernhard Amann
10cc44b37f Add tls flag to smtp.log. Will be set if a connection switched to startls. 2014-05-15 10:53:11 -07:00
Bernhard Amann
6bc914458b Add smtp starttls support 2014-05-15 09:59:43 -07:00
Vlad Grigorescu
a3e00322a2 Update test baselines. 2014-05-15 11:18:00 -04:00
Vlad Grigorescu
df99f87dbf Merge origin/master into topic/vladg/radius 2014-05-14 23:23:08 -04:00
Bernhard Amann
5bd0c3fcaf move tls content types from heartbleed to consts.bro. Seems better to put them there... 2014-05-14 15:45:47 -07:00
Jon Siwek
3905b6fc70 Clean up base SNMP script. Mostly docs, some logic refactors. 2014-05-02 12:36:02 -05:00
Jon Siwek
2e84e1f78c Merge branch 'topic/seth/snmp'
BIT-1142 #merged
2014-05-02 11:39:19 -05:00
Robin Sommer
8d1b47fae6 Merge remote-tracking branch 'origin/topic/bernhard/ec-curve'
BIT-1189 #merged

* origin/topic/bernhard/ec-curve:
  fix broxygen errors
  Polish changes for ecdhe/dhe
  Add DH support to SSL analyzer.
  Add a few more ciphers Bro did not know at all so far.
  Forgot a few ciphers in the EC list...
  Log chosen curve when using ec cipher suite in TLS.
2014-05-01 20:52:50 -07:00
Jon Siwek
5b9d190f2c Fix missing "irc-dcc-data" service field from IRC DCC connections. 2014-05-01 14:08:07 -05:00
Bernhard Amann
b1a2bccdc7 Add a few more ciphers Bro did not know at all so far. 2014-04-26 15:24:28 -07:00
Bernhard Amann
597c373fa0 Log chosen curve when using ec cipher suite in TLS. 2014-04-26 09:48:36 -07:00
Robin Sommer
201fc7b25a Merge remote-tracking branch 'origin/topic/bernhard/ssl-analyzer'
* origin/topic/bernhard/ssl-analyzer:
  Fix a few failing tests
  Add very basic ocsp stapling support.
  Add documentation, consts and tests for the new events.
  Support parsing of several TLS extensions.
  Make SSL/TLS version detection less brittle.
  Nicer notices for heartbleed.
  rip out state handline from ssl analyzer.
  enable detection of encrypted heartbleeds.
  also extract payload data in ssl_heartbeat
  add to local.bro, add disclaimer
  make tls heartbeat messages a bit better.
  fix tabs.
  polish script and probably detect encrypted attacks too.
  detect and alert on simple case of heartbleed
  default to TLS when not being able to determine version
  add is_orig to heartbeat event
  Throw new event for heartbeat messages.

BIT-1178 #merged
2014-04-24 17:04:56 -07:00
Robin Sommer
7f9a6f51ca Merge remote-tracking branch 'origin/topic/jsiwek/bit-1156'
I've added a unit test.

* origin/topic/jsiwek/bit-1156:
  BIT-1156: Fix parsing of DNS TXT RRs w/ multiple character-strings.

BIT-1156 #merged
2014-04-24 16:36:47 -07:00
Robin Sommer
de20b4f0fb Merge remote-tracking branch 'origin/topic/jsiwek/faf-perf'
* origin/topic/jsiwek/faf-perf:
  Adapt HTTP partial content to cache file analysis IDs.
  Adapt SSL analyzer to generate file analysis handles itself.
  Adapt more of HTTP analyzer to use cached file analysis IDs.
  Adapt IRC/FTP analyzers to cache file analysis IDs.
  Refactor regex/signature AcceptingSet data structure and usages.
  Enforce data size limit when checking files for MIME matches.
  Refactor file analysis file ID lookup.
2014-04-24 16:16:14 -07:00
Jon Siwek
de0ce6deed BIT-1156: Fix parsing of DNS TXT RRs w/ multiple character-strings.
The "dns_TXT_reply" event now uses a "vector of strings" as the final
parameter instead of just a "string" in order to support DNS TXT
resource records that contain multiple character-strings.

The format in which the TXT answers are logged by default is now changed
to be a list of strings of the form `fmt("TXT %d %s", |str|, str)`, one
for each character-string in the RR and delimited by a space (' ')
character.
2014-04-24 16:20:01 -05:00
Bernhard Amann
9b7eb293f1 Add documentation, consts and tests for the new events.
This also fixes the heartbleed detector to work for encrypted attacks in this
branch again. It stopped working, because the SSL analyzer now successfully detects
established connections, and the scripts usually disable analyzing after that.

(The heartbeat branch should not have been affected)
2014-04-24 12:05:30 -07:00
Jon Siwek
58efa09426 Adapt SSL analyzer to generate file analysis handles itself. 2014-04-23 16:59:27 -05:00
Bernhard Amann
4ae52d9e1c Support parsing of several TLS extensions.
At the moment, we have support for:
elliptic_curves: client supported elliptic curves
ec_point_formats: list of client supported EC point formats
application_layer_protocol_negotiation: list of supported application layer protocols (used for spdy/http2 negotiation)
server_name: server name sent by client. This was supported before, but... a bit brittle.
2014-04-23 14:34:06 -07:00
Robin Sommer
e24f3f5fd5 Updating CHANGES and VERSION. 2014-04-22 20:01:55 -07:00
Seth Hall
db80947b5f Updated snmp script. Feedback would be welcome! 2014-04-14 15:58:37 -04:00
Bernhard Amann
c741ea7c50 Small logic fix for main ssl script.
Thank you, Jon
2014-04-10 14:35:05 -07:00
Bernhard Amann
aa73d42120 update dpd for tls 1.2
all tests still pass
2014-04-10 08:12:02 -07:00
Robin Sommer
cf7e25643e Merge remote-tracking branch 'origin/topic/jsiwek/snmp'
* origin/topic/jsiwek/snmp:
  Add memory leak unit test for SNMP.
  Fix compiler nitpicks from new SNMP code.
  Add SNMP datagram parsing support.

BIT-1142
2014-04-08 15:31:03 -07:00
Robin Sommer
9efb549236 Merge remote-tracking branch 'origin/topic/jsiwek/file-signatures'
* origin/topic/jsiwek/file-signatures:
  File type detection changes and fix https.log {orig,resp}_fuids fields.
  Various minor changes related to file mime type detection.
  Refactor common MIME magic matching code.
  Replace libmagic w/ Bro signatures for file MIME type identification.

Conflicts:
	scripts/base/init-default.bro
	testing/btest/Baseline/coverage.bare-load-baseline/canonified_loaded_scripts.log
	testing/btest/Baseline/coverage.default-load-baseline/canonified_loaded_scripts.log

BIT-1143 #merged
2014-03-30 22:51:05 +02:00
Jon Siwek
8dad5026fd File type detection changes and fix https.log {orig,resp}_fuids fields.
- Removed "binary" and "octet-stream" mime type detections. They don't
  provide any more information than an uninitialized mime_type field
  which implicitly means no magic signature matches and so the media
  type is unknown to Bro.

- Slight change to "text/plain" signature.  It's still not the most
  accurate, which is reflected in its -20 strength value.

- The logic for adding file ids to {orig,resp}_fuids fields of
  the http.log incorrectly depended on the state of
  {orig,resp}_mime_types fields, so sometimes not all file ids
  associated w/ the session were logged.
2014-03-25 12:44:11 -05:00
Robin Sommer
e8339d5c63 Merge remote-tracking branch 'origin/topic/bernhard/file-analysis-x509'
* origin/topic/bernhard/file-analysis-x509:
  Forgot the preamble for the new leak test
  (hopefully) last change -> return real opaque vec instead of any_vec
  Fix dump-events - it cannot be used with ssl anymore, because openssl does not give the same string results in all versions.
  Finishing touches of the x509 file analyzer.
  Revert change to only log certificates once per hour.
  Change x509 log - now certificates are only logged once per hour.
  Fix circular reference problem and a few other small things.
  X509 file analyzer nearly done. Verification and most other policy scripts work fine now.
  Add verify functionality, including the ability to get the validated chain. This means that it is now possible to get information about the root-certificates that were used to secure a connection.
  Second try on the event interface.
  Backport crash fix that made it into master with the x509_extension backport from here.
  Make x509 certificates an opaque type
  rip out x509 code from ssl analyzer. Note that since at the moment the file analyzer does not yet re-populate the info record that means quite a lot of information is simply not available.
  parse out extension. One event for general extensions (just returns the openssl-parsed string-value), one event for basicconstraints (is a certificate a CA or not) and one event for subject-alternative-names (only DNS parts).
  Very basic file-analyzer for x509 certificates. Mostly ripped from the ssl-analyzer and the topic/bernhard/x509 branch.
2014-03-14 09:53:07 -07:00
Bernhard Amann
4da0718511 Finishing touches of the x509 file analyzer.
Mostly baseline updates and new tests.

addresses BIT-953, BIT-760, BIT-1150
2014-03-13 15:21:30 -07:00
Bernhard Amann
74d728656d Revert change to only log certificates once per hour.
addresses BIT-953, BIT-760, BIT-1150
2014-03-13 13:38:44 -07:00
Bernhard Amann
b0c3486fd6 Merge remote-tracking branch 'origin/master' into topic/bernhard/file-analysis-x509 2014-03-13 00:09:48 -07:00
Bernhard Amann
0d50b8b04f Change x509 log - now certificates are only logged once per hour.
Add parsing of several more types to SAN extension.

Make error messages of x509 file analyzer more useful.

Fix file ID generation.

You apparently have to be very careful which EndOfFile function of
the file analysis framework you call... otherwhise it might try
to close another file id. This took me quite a while to find.

addresses BIT-953, BIT-760, BIT-1150
2014-03-13 00:05:48 -07:00
Jon Siwek
f30d3e635e Fix non-deterministic logging of unmatched DNS msgs, addresses BIT-1153
Unmatched DNS messages may fail to be logged sometimes due to a type of
iterator invalidation.
2014-03-10 11:34:57 -05:00
Robin Sommer
69d52feb18 Merge remote-tracking branch 'origin/topic/seth/dns-srv-fix'
* origin/topic/seth/dns-srv-fix:
  No longer accidentally attempting to parse NBSTAT RRs as SRV RRs.
  Fix DNS SRV responses and a small issue with NBNS queries and label length.

BIT-1147 #merged
2014-03-09 08:59:48 -07:00
Seth Hall
bcdffe3212 No longer accidentally attempting to parse NBSTAT RRs as SRV RRs.
The NetBios name service RFC (1002) specified NBSTAT (NetBios Status)
resource records to have identifier 0x0021.  The DNS SRV RFC specified
SRV records to have identifier 33.  Unfortunately those are the
same number. :)

We now check the resp port to handle this situation better so that
we won't be attempting to parse NBSTAT records as SRV (which
causes several weird messages).
2014-03-06 09:06:23 -05:00
Seth Hall
9743959995 Fix DNS SRV responses and a small issue with NBNS queries and label length.
- DNS SRV responses never had the code written to actually
   generate the dns_SRV_reply event.  Adding this required
   extending the event a bit to add extra information.  SRV responses
   now appear in the dns.log file correctly.

 - Fixed an issue where some Microsoft NetBIOS Name Service lookups
   would exceed the max label length for DNS and cause an incorrect
   "DNS_label_too_long" weird.
2014-03-05 16:11:06 -05:00
Bernhard Amann
f140abc629 only call disable_analyzer if the connection is still open. 2014-03-04 15:09:19 -08:00
Bernhard Amann
ea1616bed5 At the moment, SSL connections where the ssl_established event does not fire are not logged.
That means that, for example, connections that are terminated with an alert during the
handshake never appear in the ssl.log.

This patch changes this behavior - now all ssl connections that fire any event are logged.

The protocol confirmation of the ssl analyzer is moved to the client_hello instead to
the server hello. Furthermore, an additional field is added to ssl.log, which indicates
if a connection has been established or not (which probably indicates a handshake problem).
2014-03-04 14:23:49 -08:00
Bernhard Amann
7eb6b5133e Fix circular reference problem and a few other small things.
SSL::Info now holds a reference to Files::Info instead of the
fa_files record.

Everything should work now, if everyone thinks that the interface is
ok I will update the test baselines in a bit.

addresses BIT-953, BIT-760
2014-03-04 05:30:32 -08:00
Bernhard Amann
110d9fbd6a X509 file analyzer nearly done. Verification and most other policy scripts
work fine now.

Todo:
 * update all baselines
 * fix the circular reference to the fa_file structure I introduced :)
   Sadly this does not seem to be entirely straightforward.

addresses BIT-953, BIT-760
2014-03-03 17:07:50 -08:00
Robin Sommer
d6d26a3ea7 Merge branch 'topic/robin/http-connect'
* topic/robin/http-connect:
  HTTP fix for output handlers.
  Expanding the HTTP methods used in the signature to detect HTTP traffic.
  Updating submodule(s).
  Fixing removal of support analyzers, plus some tweaking and cleanup of CONNECT code.
  HTTP CONNECT proxy support.

BIT-1132 #merged
2014-03-03 16:53:46 -08:00