* topic/robin/intel-framework-merge: (22 commits)
Fixing tests after intel-framework merge.
Extracting URLs from message bodies over SMTP and sending them to Intel framework.
Small comment updates in the Intel framework CIF support.
Intelligence framework documentation first draft.
Only the manager tries to read files with the input framework now.
Initial support for Bro's Intel framework with the Collective Intelligence Framework.
Initial API for Intel framework is complete.
Fixed an issue with cluster data distribution.
Updating some intel framework test baselines.
Reworked cluster intelligence data distribution mechanism and fixed tests.
Lots more intelligence checking in SMTP traffic.
Added intelligence check for "Received" path checking and a bit of reshuffling.
Added sources to the intel log.
Fixing a problem with intel distribution on clusters.
Updated intel framework test to include matching.
Restructuring the scripts that feed data into the intel framework slightly.
One test for cluster transparency of the intel framework.
Fixed a cluster support bug.
Intelligence framework checkpoint
Major updates to fix the Intel framework API.
...
Closes#914.
Instead, the `addr_to_uri` script-level function can be used to
explicitly add brackets to an address if it's IPv6 and will be
included in a URI or when a ":<port>" needs to be appended to it.
* origin/fastpath:
Add two more TLS extension values that we see in live traffic.
Fixed IPv6 link local unicast CIDR and added IPv6 loopback to private address space.
Fixed a problem where cluster workers were still processing notices in some cases.
Added Carrier Grade NAT CIDR and link local IPv6 to "private_address_space"
- Metrics now work on cluster deployments with no caveats. It should be
completely transparent. Intermediate updates to speed some detection
will come later.
- While updating, I did some further work on the branch.
- New function in the base/utils/files for extracting filenames
from content-dispositions.
- New script for entity excerpt extraction if you aren't interested
in full extraction. The data goes a log field too.
- Some renaming and reorganization of types.
- Updated tests to work with new code.
* origin/topic/jsiwek/smtp-refactor:
Make the doc.coverage test happy.
SMTP script refactor. (addresses #509)
Conflicts:
doc/scripts/DocSourcesList.cmake
policy/protocols/smtp/__load__.bro
policy/protocols/smtp/base/__load__.bro
- policy/ renamed to scripts/
- By default BROPATH now contains:
- scripts/
- scripts/policy
- scripts/site
- *Nearly* all tests pass.
- All of scripts/base/ is loaded by main.cc
- Can be disabled by setting $BRO_NO_BASE_SCRIPTS
- Scripts in scripts/base/ don't use relative path loading to ease use of BRO_NO_BASE_SCRIPTS (to copy and paste that script).
- The scripts in scripts/base/protocols/ only (or soon will only) do logging and state building.
- The scripts in scripts/base/frameworks/ add functionality without causing any additional overhead.
- All "detection" activity happens through scripts in scripts/policy/.
- Communications framework modified temporarily to need an environment variable to actually enable (ENABLE_COMMUNICATION=1)
- This is so the communications framework can be loaded as part
of the base without causing trouble when it's not needed.
- This will be removed once a resolution to ticket #540 is reached.
