* origin/topic/seth/rdp: (31 commits)
Improved transition into SSL/TLS from RDP.
Fixes tests in RDP branch.
add a special case to the X509 code that deals with RDP certificates.
A few more changes to handling encryption in RDP.
Adds some comments and fixes a broxygen warning.
Fixes another optional part of an RDP unit.
Support RDP negotiation requests optionally and support zero length cookies.
Changed UTF-16 to UTF-8 conversion to be more lenient.
Fixed an issue with parse failure on an optional field.
Removing a stray printf from RDP analyzer.
Another big RDP update.
New script to add a field to rdp.log when the connection is upgraded to SSL.
Huge updates to the RDP analyzer from Josh Liburdi.
FreeRDP test trace showing SSL encryption -- RDP analyzer does not currently handle this and SSL analyzer does not identify it either
Wireshark test trace for native encryption -- generates a binpac error
Delete RDP-004.pcap
Delete nla_win7_win2k8r2.pcap
Update dpd.sig
Fixed typo
Added check for connection existence
...
BIT-1340 #merged
The new loader scripts should not try to @load .bif.bro scripts that
were part of a build of an earlier Bro version, but have since had their
.bif file removed.
* origin/topic/jsiwek/file-signatures:
File type detection changes and fix https.log {orig,resp}_fuids fields.
Various minor changes related to file mime type detection.
Refactor common MIME magic matching code.
Replace libmagic w/ Bro signatures for file MIME type identification.
Conflicts:
scripts/base/init-default.bro
testing/btest/Baseline/coverage.bare-load-baseline/canonified_loaded_scripts.log
testing/btest/Baseline/coverage.default-load-baseline/canonified_loaded_scripts.log
BIT-1143 #merged
Notable changes:
- libmagic is no longer used at all. All MIME type detection is
done through new Bro signatures, and there's no longer a means to get
verbose file type descriptions (e.g. "PNG image data, 1435 x 170").
The majority of the default file magic signatures are derived
from the default magic database of libmagic ~5.17.
- File magic signatures consist of two new constructs in the
signature rule parsing grammar: "file-magic" gives a regular
expression to match against, and "file-mime" gives the MIME type
string of content that matches the magic and an optional strength
value for the match.
- Modified signature/rule syntax for identifiers: they can no longer
start with a '-', which made for ambiguous syntax when doing negative
strength values in "file-mime". Also brought syntax for Bro script
identifiers in line with reality (they can't start with numbers or
include '-' at all).
- A new Built-In Function, "file_magic", can be used to get all
file magic matches and their corresponding strength against a given
chunk of data
- The second parameter of the "identify_data" Built-In Function
can no longer be used to get verbose file type descriptions, though it
can still be used to get the strongest matching file magic signature.
- The "file_transferred" event's "descr" parameter no longer
contains verbose file type descriptions.
- The BROMAGIC environment variable no longer changes any behavior
in Bro as magic databases are no longer used/installed.
- Reverted back to minimum requirement of CMake 2.6.3 from 2.8.0
(it's back to being the same requirement as the Bro v2.2 release).
The bump was to accomodate building libmagic as an external project,
which is no longer needed.
Addresses BIT-1143.
This supports parsing of SNMPv1 (RFC 1157), SNMPv2 (RFC 1901/3416), and
SNMPv2 (RFC 3412). An event is raised for each SNMP PDU type, though
there's not currently any event handlers for them and not a default
snmp.log either. However, simple presence of SNMP is currently visible
now in conn.log service field and known_services.log.
If reading a trace file w/ only TCP control packets, a warning is
emitted to suggest the 'detect_filtered_traces' option if the user
doesn't desire Bro to report missing TCP segments for such a trace file.
BIT-1054 #merged
* origin/topic/seth/unified2-analyzer:
Fixes in case a packet isn't seen that matches an event.
Finished work on unified2 analyzer.
Fixed some tests.
Working unified2 analyzer.
Unified2 file analyzer updated to new plugin style.
Adding the unified2 analyzer.
Conflicts:
testing/btest/Baseline/coverage.bare-load-baseline/canonified_loaded_scripts.log
testing/btest/Baseline/coverage.default-load-baseline/canonified_loaded_scripts.log
Includes a bit more docs/comments cleanup. We should eventually
document the events further but it should suffice for now.
* topic/robin/dnp3-merge-v3:
Tiny bit of cleanup and adapting the new test.
added a test case for dnp3 packets with only link layer
added condition to check DNP3 packet without app layer data
Fixing well-known port.
Pluginizing the DNP3 analyzer, plus a basic script logging requests and replies.
* origin/topic/jsiwek/exec-module:
Exec module changes/fixes.
Coverage test fixes and whitespace/doc tweaks.
Update to make Dir::monitor watch inodes instead of file names.
Updates to use new input framework mechanism to execute command line programs.
Added Exec, Dir, and ActiveHTTP modules.
BIT-1046 #merged.
Conflicts:
magic
testing/btest/Baseline/coverage.default-load-baseline/canonified_loaded_scripts.log
* origin/topic/seth/faf-updates: (27 commits)
Undoing the FTP tests I updated earlier.
Update the last two btest FAF tests.
File analysis fixes and test updates.
Fix a bug with getting analyzer tags.
A few test updates.
Some tests work now (at least they all don't fail anymore!)
Forgot a file.
Added protocol description functions that provide a super compressed log representation.
Fix a bug where orig file information in http wasn't working right.
Added mime types to http.log
Clean up queued but unused file_over_new_connections event args.
Add jar files to the default MHR lookups.
Adding CAB files for MHR checking.
Improve malware hash registry script.
Fix a small issue with finding smtp entities.
Added support for files to the notice framework.
Make the custom libmagic database a git submodule.
Add an is_orig parameter to file_over_new_connection event.
Make magic for emitting application/msword mime type less strict.
Disable more libmagic builtin checks that override the magic database.
...
Conflicts:
doc/scripts/DocSourcesList.cmake
scripts/base/init-bare.bro
scripts/test-all-policy.bro
testing/btest/Baseline/coverage.bare-load-baseline/canonified_loaded_scripts.log
testing/btest/Baseline/coverage.default-load-baseline/canonified_loaded_scripts.log
- Fix examples/references in the file analysis how-to/usage doc.
- Add Broxygen-generated docs for file analyzer plugins.
- Break FTP::Info type declaration out in to its own file to get
rid of some circular dependencies (between s/b/p/ftp/main and
s/b/p/ftp/utils).
in.
No more manual includes to pull them in.
(It doesn't quite work fully automatically yet for some bifs that need
script-level types defined, like the input and logging frameworks.
They still do a manual "@load foo.bif" in their main.bro to get the
order right. It's a bit tricky to fix that and would probably need
splitting main.bro into two parts; not sure that's worth it.)
