Good stuff! (but I admit I didn't look at the OpenSSL code too closely :)
* origin/topic/bernhard/even-more-ssl-changes:
small test update & script fix
update baselines & add ocsp leak check
Add policy script adding ocsp validation to ssl.log
Implement verification of OCSP replies.
Add tls flag to smtp.log. Will be set if a connection switched to startls.
add starttls support for pop3
Add smtp starttls support
Replace errors when parsing x509 certs with weirds (as requested by Seth).
move tls content types from heartbleed to consts.bro. Seems better to put them there...
Add new features from other branch to the heartbleed-detector (and clean them up).
Let TLS analyzer fail better when no longer in sync with the data stream. The version field in each record-layer packet is now re-checked.
BIT-1190 #merged
Conflicts:
testing/btest/Baseline/scripts.policy.misc.dump-events/all-events.log
testing/btest/Baseline/scripts.policy.misc.dump-events/smtp-events.log
* origin/topic/vladg/radius:
Radius functionality and memleak test.
Update test baselines.
Move seq to uint64 to match recent changes in seq processing.
BIT-1129 #merged
BIT-1189 #merged
* origin/topic/bernhard/ec-curve:
fix broxygen errors
Polish changes for ecdhe/dhe
Add DH support to SSL analyzer.
Add a few more ciphers Bro did not know at all so far.
Forgot a few ciphers in the EC list...
Log chosen curve when using ec cipher suite in TLS.
* origin/topic/bernhard/ssl-analyzer:
Fix a few failing tests
Add very basic ocsp stapling support.
Add documentation, consts and tests for the new events.
Support parsing of several TLS extensions.
Make SSL/TLS version detection less brittle.
Nicer notices for heartbleed.
rip out state handline from ssl analyzer.
enable detection of encrypted heartbleeds.
also extract payload data in ssl_heartbeat
add to local.bro, add disclaimer
make tls heartbeat messages a bit better.
fix tabs.
polish script and probably detect encrypted attacks too.
detect and alert on simple case of heartbleed
default to TLS when not being able to determine version
add is_orig to heartbeat event
Throw new event for heartbeat messages.
BIT-1178 #merged
* origin/topic/jsiwek/faf-perf:
Adapt HTTP partial content to cache file analysis IDs.
Adapt SSL analyzer to generate file analysis handles itself.
Adapt more of HTTP analyzer to use cached file analysis IDs.
Adapt IRC/FTP analyzers to cache file analysis IDs.
Refactor regex/signature AcceptingSet data structure and usages.
Enforce data size limit when checking files for MIME matches.
Refactor file analysis file ID lookup.
The "dns_TXT_reply" event now uses a "vector of strings" as the final
parameter instead of just a "string" in order to support DNS TXT
resource records that contain multiple character-strings.
The format in which the TXT answers are logged by default is now changed
to be a list of strings of the form `fmt("TXT %d %s", |str|, str)`, one
for each character-string in the RR and delimited by a space (' ')
character.
This also fixes the heartbleed detector to work for encrypted attacks in this
branch again. It stopped working, because the SSL analyzer now successfully detects
established connections, and the scripts usually disable analyzing after that.
(The heartbeat branch should not have been affected)
At the moment, we have support for:
elliptic_curves: client supported elliptic curves
ec_point_formats: list of client supported EC point formats
application_layer_protocol_negotiation: list of supported application layer protocols (used for spdy/http2 negotiation)
server_name: server name sent by client. This was supported before, but... a bit brittle.
This prevents the worker nodes from crashing, when request_key is used
in cluster mode and called on the worker and the manager nodes (i.e. when
a non-cluster-aware script is used).
Addresses BIT-1177
* origin/topic/jsiwek/file-signatures:
File type detection changes and fix https.log {orig,resp}_fuids fields.
Various minor changes related to file mime type detection.
Refactor common MIME magic matching code.
Replace libmagic w/ Bro signatures for file MIME type identification.
Conflicts:
scripts/base/init-default.bro
testing/btest/Baseline/coverage.bare-load-baseline/canonified_loaded_scripts.log
testing/btest/Baseline/coverage.default-load-baseline/canonified_loaded_scripts.log
BIT-1143 #merged
* origin/topic/bernhard/file-analysis-x509:
x509 version always has to be incremented. Baselines already committed.
Fix x509 certificate Version (the +1 was missing...).
Add policy script to suppress non host-certificate logging in x509.log
Forgot the preamble for the new leak test
(hopefully) last change -> return real opaque vec instead of any_vec
Fix dump-events - it cannot be used with ssl anymore, because openssl does not give the same string results in all versions.
Finishing touches of the x509 file analyzer.
Revert change to only log certificates once per hour.
Change x509 log - now certificates are only logged once per hour.
Fix circular reference problem and a few other small things.
X509 file analyzer nearly done. Verification and most other policy scripts work fine now.
Add verify functionality, including the ability to get the validated chain. This means that it is now possible to get information about the root-certificates that were used to secure a connection.
Second try on the event interface.
Backport crash fix that made it into master with the x509_extension backport from here.
Make x509 certificates an opaque type
rip out x509 code from ssl analyzer. Note that since at the moment the file analyzer does not yet re-populate the info record that means quite a lot of information is simply not available.
parse out extension. One event for general extensions (just returns the openssl-parsed string-value), one event for basicconstraints (is a certificate a CA or not) and one event for subject-alternative-names (only DNS parts).
Very basic file-analyzer for x509 certificates. Mostly ripped from the ssl-analyzer and the topic/bernhard/x509 branch.
BIT-1150 #merged
* origin/topic/jsiwek/improve-type-checks:
Add unit tests covering vector/set/table ctors/inits.
Fix parsing of "local" named table constructors.
Improve type checking of records, addresses BIT-1159.
BIT-1159 #merged
- Removed "binary" and "octet-stream" mime type detections. They don't
provide any more information than an uninitialized mime_type field
which implicitly means no magic signature matches and so the media
type is unknown to Bro.
- Slight change to "text/plain" signature. It's still not the most
accurate, which is reflected in its -20 strength value.
- The logic for adding file ids to {orig,resp}_fuids fields of
the http.log incorrectly depended on the state of
{orig,resp}_mime_types fields, so sometimes not all file ids
associated w/ the session were logged.
