* topic/robin/intel-framework-merge: (22 commits)
Fixing tests after intel-framework merge.
Extracting URLs from message bodies over SMTP and sending them to Intel framework.
Small comment updates in the Intel framework CIF support.
Intelligence framework documentation first draft.
Only the manager tries to read files with the input framework now.
Initial support for Bro's Intel framework with the Collective Intelligence Framework.
Initial API for Intel framework is complete.
Fixed an issue with cluster data distribution.
Updating some intel framework test baselines.
Reworked cluster intelligence data distribution mechanism and fixed tests.
Lots more intelligence checking in SMTP traffic.
Added intelligence check for "Received" path checking and a bit of reshuffling.
Added sources to the intel log.
Fixing a problem with intel distribution on clusters.
Updated intel framework test to include matching.
Restructuring the scripts that feed data into the intel framework slightly.
One test for cluster transparency of the intel framework.
Fixed a cluster support bug.
Intelligence framework checkpoint
Major updates to fix the Intel framework API.
...
Closes#914.
* vlad/topic/vladg/dns_txt_queries:
Add detection rate threshold for MHR.
Make sure lookup_hostname_txt isn't checking lookup_hostname's cache for answers.
A couple of lookup_hostname_txt fixes.
Reverting the earlier function-join during merging.
* origin/topic/bernhard/software:
change software framework interface again. At the moment everything should worl.
start reworking interface of software framework. working apart from detect-webapps.bro, which direcly manipulates a no longer available interface...
after talking to seth - change host_a field in record back to host.
forgotten policy files.
Software framework stores ports for server software.
* origin/fastpath:
Fix missing action in notice policy for looking up GeoIP data.
Better persistent state config warning messages (fixes#433).
A few updates for SQL injection detection.
Fixed some DPD signatures for IRC. Fixes ticket #311.
Removing Off_Port_Protocol_Found notice.
SSH::Interesting_Hostname_Login cleanup. Fixes#664.
Teach Broxygen to more generally reference attribute values by name.
Fixed a really dumb bug that was causing the malware hash registry script to break.
Fix Broxygen confusing scoped id at start of line as function parameter.
Remove remnant of libmagic optionality
- The biggest change is the change in notice names from
HTTP::SQL_Injection_Attack_Against to
HTTP::SQL_Injection_Victim
- A few new SQL injection attacks in the tests that we need to
support at some point.
- With the software-browser-plugins script you can watch for Omniture
advertising servers to grab the list of installed plugins.
- I reorganized the plugin detection a bit too to abstract it better.
- Removed the WEB_ prefix from all of the Software::Type HTTP enums.
They were essentially redundant due to the full name already being
HTTP::SERVER (for example).
- Removed an notice definition from the base SSL scripts.
- Moved a logging stream ID into the export section for known-services
and bumped priority for creating the stream.
- Adding configuration knobs for the SQL injection attack detection
script and renaming the HTTP::SQL_Injection_Attack notice to
HTTP::SQL_Injection_Attack_Against
- Bumped priority when creating Known::CERTS_LOG.
* origin/topic/gregor/script-polishing:
Tune when c$conn is set.
Set c$conn (for logging) in new_connection() event.
(Semiautomatically) convert the comments in bare-init.bro into autodoc ones.
Add ConnSize_Analyzer's fields to conn.log
SSH base scripts: make sure ConnSizeAnalyzer variables are available before using them.
Make reference to the other script a link
- fixing some Metrics::add_data() call signatures
- slight refactors to cluster framework @if and adding a NONE NodeType for
so local_node_type() will return that instead of just emitting an error
when cluster mode isn't enabled
- `make restdoc` target now uses bro's bare-mode
- bro scripts generated from bifs now really only live in the build/src/base/
directory and changed the DocSourcesList.cmake to dynamically figure out
what bifs exist by looking in src/ instead of build/src/
- add some missing @load dependencies
- Metrics API is much more similar to the Logging framework's API now.
- Filters define all output and metrics collection now.
- Initial attempt at thresholding and generating notices.
- policy/ renamed to scripts/
- By default BROPATH now contains:
- scripts/
- scripts/policy
- scripts/site
- *Nearly* all tests pass.
- All of scripts/base/ is loaded by main.cc
- Can be disabled by setting $BRO_NO_BASE_SCRIPTS
- Scripts in scripts/base/ don't use relative path loading to ease use of BRO_NO_BASE_SCRIPTS (to copy and paste that script).
- The scripts in scripts/base/protocols/ only (or soon will only) do logging and state building.
- The scripts in scripts/base/frameworks/ add functionality without causing any additional overhead.
- All "detection" activity happens through scripts in scripts/policy/.
- Communications framework modified temporarily to need an environment variable to actually enable (ENABLE_COMMUNICATION=1)
- This is so the communications framework can be loaded as part
of the base without causing trouble when it's not needed.
- This will be removed once a resolution to ticket #540 is reached.
