* origin/topic/bernhard/ssl-analyzer:
Fix a few failing tests
Add very basic ocsp stapling support.
Add documentation, consts and tests for the new events.
Support parsing of several TLS extensions.
Make SSL/TLS version detection less brittle.
Nicer notices for heartbleed.
rip out state handline from ssl analyzer.
enable detection of encrypted heartbleeds.
also extract payload data in ssl_heartbeat
add to local.bro, add disclaimer
make tls heartbeat messages a bit better.
fix tabs.
polish script and probably detect encrypted attacks too.
detect and alert on simple case of heartbleed
default to TLS when not being able to determine version
add is_orig to heartbeat event
Throw new event for heartbeat messages.
BIT-1178 #merged
This also fixes the heartbleed detector to work for encrypted attacks in this
branch again. It stopped working, because the SSL analyzer now successfully detects
established connections, and the scripts usually disable analyzing after that.
(The heartbeat branch should not have been affected)
SSL::Info now holds a reference to Files::Info instead of the
fa_files record.
Everything should work now, if everyone thinks that the interface is
ok I will update the test baselines in a bit.
addresses BIT-953, BIT-760
work fine now.
Todo:
* update all baselines
* fix the circular reference to the fa_file structure I introduced :)
Sadly this does not seem to be entirely straightforward.
addresses BIT-953, BIT-760
The previous method of matching queries with replies was still
unreliable in cases where the reply contains no answers. The new code
also takes extra measures to avoid pending state growing too large in
cases where the condition to match a query with a corresponding reply is
never met, but yet DNS messages continue to be exchanged over the same
connection 5-tuple (preventing cleanup of the pending state).
The scope of dns.log is now only standard queries (OPCODE == 0). Other
kinds of queries (e.g. inverse query) were not handled correctly and
could interfere with the state tracking of the default DNS scripts.
- Added the known-devices log, and the script to log DHCP hostname per MAC address,
- Added DPD sig,
- Removed some scripts following a discussion with Seth.
* origin/topic/seth/faf-updates: (27 commits)
Undoing the FTP tests I updated earlier.
Update the last two btest FAF tests.
File analysis fixes and test updates.
Fix a bug with getting analyzer tags.
A few test updates.
Some tests work now (at least they all don't fail anymore!)
Forgot a file.
Added protocol description functions that provide a super compressed log representation.
Fix a bug where orig file information in http wasn't working right.
Added mime types to http.log
Clean up queued but unused file_over_new_connections event args.
Add jar files to the default MHR lookups.
Adding CAB files for MHR checking.
Improve malware hash registry script.
Fix a small issue with finding smtp entities.
Added support for files to the notice framework.
Make the custom libmagic database a git submodule.
Add an is_orig parameter to file_over_new_connection event.
Make magic for emitting application/msword mime type less strict.
Disable more libmagic builtin checks that override the magic database.
...
Conflicts:
doc/scripts/DocSourcesList.cmake
scripts/base/init-bare.bro
scripts/test-all-policy.bro
testing/btest/Baseline/coverage.bare-load-baseline/canonified_loaded_scripts.log
testing/btest/Baseline/coverage.default-load-baseline/canonified_loaded_scripts.log
- Several places were just using old variable names or not loading
scripts correctly after they'd been renamed/moved.
- Revert/adjust a change in how HTTP file handles are generated that
broke partial content responses.
- Turn some libmagic builtin checks back on; seems some are actually
useful (e.g. text detection seems to be a builtin). The rule going
forward probably will be only to turn off a builtin if we confirm it
causes issues.
- Removed some tests that are redundant or not necessary anymore because
the generic file analysis tests cover them.
- A couple FTP tests still fail that I think need an actual solution via
script changes.
- On-demand access to sumstats results through "return from"
functions named SumStats::request and Sumstats::request_key.
Both functions are tested in standalone and clustered modes.
- $name field has returned to SumStats which simplifies cluster
code and makes the on-demand access stuff possible.
- Clustered results can only be collected for 1 minute from their
time of creation now instead of time of last read.
- Thresholds use doubles instead of counts everywhere now.
- Calculation dependency resolution occurs at start up time now
instead of doing it at observation time which provide a minor
cpu performance improvement. A new plugin registration mechanism
was created to support this change.
- AppStats now has a minimal doc string and is broken into hook-based
plugins.
- AppStats and traceroute detection added to local.bro
Thanks to git this merge was less troublesome that I was afraid it
would be. Not all tests pass yet though (and file hashes have changed
unfortunately).
Conflicts:
cmake
doc/scripts/DocSourcesList.cmake
scripts/base/init-bare.bro
scripts/base/protocols/ftp/main.bro
scripts/base/protocols/irc/dcc-send.bro
scripts/test-all-policy.bro
src/AnalyzerTags.h
src/CMakeLists.txt
src/analyzer/Analyzer.cc
src/analyzer/protocol/file/File.cc
src/analyzer/protocol/file/File.h
src/analyzer/protocol/http/HTTP.cc
src/analyzer/protocol/http/HTTP.h
src/analyzer/protocol/mime/MIME.cc
src/event.bif
src/main.cc
src/util-config.h.in
testing/btest/Baseline/coverage.bare-load-baseline/canonified_loaded_scripts.log
testing/btest/Baseline/coverage.default-load-baseline/canonified_loaded_scripts.log
testing/btest/Baseline/istate.events-ssl/receiver.http.log
testing/btest/Baseline/istate.events-ssl/sender.http.log
testing/btest/Baseline/istate.events/receiver.http.log
testing/btest/Baseline/istate.events/sender.http.log
- New, expanded API.
- Calculations moved into plugins.
- Scripts using measurement framework ported.
- Updated the script-land queue implementation to make it more generic.
-
