* topic/robin/intel-framework-merge: (22 commits)
Fixing tests after intel-framework merge.
Extracting URLs from message bodies over SMTP and sending them to Intel framework.
Small comment updates in the Intel framework CIF support.
Intelligence framework documentation first draft.
Only the manager tries to read files with the input framework now.
Initial support for Bro's Intel framework with the Collective Intelligence Framework.
Initial API for Intel framework is complete.
Fixed an issue with cluster data distribution.
Updating some intel framework test baselines.
Reworked cluster intelligence data distribution mechanism and fixed tests.
Lots more intelligence checking in SMTP traffic.
Added intelligence check for "Received" path checking and a bit of reshuffling.
Added sources to the intel log.
Fixing a problem with intel distribution on clusters.
Updated intel framework test to include matching.
Restructuring the scripts that feed data into the intel framework slightly.
One test for cluster transparency of the intel framework.
Fixed a cluster support bug.
Intelligence framework checkpoint
Major updates to fix the Intel framework API.
...
Closes#914.
* remotes/origin/topic/seth/modbus-merge:
Small modbus documentation update and tiny refactoring.
Final touches to modbus analyzer for now.
Major revisions to Modbus analyzer support (not quite done yet).
put some make-up on Modbus analyser
Modbus analyser, added support: FC=20,21
Modbus analyzer,added support: FC=1,2,15,24
Modbus analyzer, current support: FC=3,4,5,6,7,16,22,23
Closes#915.
* vlad/topic/vladg/dns_txt_queries:
Add detection rate threshold for MHR.
Make sure lookup_hostname_txt isn't checking lookup_hostname's cache for answers.
A couple of lookup_hostname_txt fixes.
Reverting the earlier function-join during merging.
- There are still some broken events in the modbus analyzer because
I don't have traffic to test with (coil and record related events primarily).
- There are a few example scripts in policy/protocols/modbus
- Basic API seems to works, but tests aren't updated yet.
- Several scripts are available in policy/frameworks/intel that
call the "seen" function to provide data into the intel
framework to be tested.
- Intel::policy is not done yet and needs to be discussed to
figure out what it needs to have.
- Running the intel framework and having it do something finally
is really cool!
This usually requires specifying an additional zone identifier
(see RFC 4007). The connect() and listen() BIFs have been
changed to accept this zone identifier as an argument.
- Communication::listen_ipv6 needs to be redef'd to true in order
for IPv6 listening sockets to be opened.
- Added Communication::listen_retry option as an interval at which
to retry binding to socket addresses that were already in use.
- Added some explicit baselines to check in the istate.events
and istate.events-ssl tests -- the SSL test was incorrectly
passing because it compared two empty files. (The files being
empty because "http/base" was given as an argument to Bro which
it couldn't handle because that script doesn't exist anymore).
- New script measures a couple of aspects of SMTP traffic.
- Existing metrics scripts had a small amount of work done
to make them work with changes to metrics framework.
- Large rework on packet filter framework to make many things easier.
- Removed the PacketFilter::all_packets variable because it was confusing.
- New variable (PacketFilter::enable_auto_protocol_capture_filters) to re-enable the old filtering model of only sniffing ports for analyzed protocols.
- In progress plugin model for adding filtering mechanisms.
- New default single item for capture_filters = { ["default"] = PacketFilter::default_capture_filter };
- Mechanism and helper functions to "shunt" traffic with filters.
- Created the Protocols framework to assist with reworking how base protocol scripts are registered with DPD and other things.
- Protocols framework creates BPF filters for registered analyzers. (if using PacketFilter framework in that mode).
* origin/topic/bernhard/software:
change software framework interface again. At the moment everything should worl.
start reworking interface of software framework. working apart from detect-webapps.bro, which direcly manipulates a no longer available interface...
after talking to seth - change host_a field in record back to host.
forgotten policy files.
Software framework stores ports for server software.
* origin/topic/script-reference: (50 commits)
A few updates for the FAQ.
Fixing some doc warnings.
Forgot to add protocol identifier support for TLS 1.2
Finished SSL & syslog autodocs.
Adding the draft SSL extension type next_protocol_negotiation.
Fix some documentation errors.
Tweaks.
A set of script-reference polishing.
fixed a couple typos in comments
Add summary documentation to bif files.
Add ssl and syslog script documentation
Add Conn and DNS protocol script documentation. (fixes#731)
Small updates to the default local.bro.
Documentation updates for HTTP & IRC scripts.
SSH&FTP Documentation updates.
Fixing a warning from the documentation generation.
This completes framework documentation package 4.
Minor notice documentation tweaks.
Fix some malformed Broxygen xref roles.
Minor doc tweaks to init-bare.bro.
...
Conflicts:
aux/broccoli
aux/broctl
src/bro.bif
src/strings.bif
Includes:
- Updated baselines for autodoc tests.
- Now excluding stats.bro from external texts, it's not stable.
