and fix up the hll scripts for it.
Conflicts:
scripts/base/frameworks/sumstats/plugins/__load__.bro
testing/btest/scripts/base/frameworks/measurement/basic.bro
- FileAnalysis::Info is now just a record used for logging, the fa_file
record type is defined in init-bare.bro as the analogue to a
connection record.
- Starting to transfer policy hook triggers and analyzer results to
events.
I've used the opportunity to also cleanup DPD's expect_connection()
infrastructure, and renamed that bif to schedule_analyzer(), which
seems more appropiate. One can now also schedule more than one
analyzer per connection.
TODOs:
- "make install" is probably broken.
- Broxygen is probably broken for plugin-defined events.
- event groups are broken (do we want to keep them?)
- parallel btest is broken, but I'm not sure why ...
(tests all pass individually, but lots of error when running
in parallel; must be related to *.bif restructuring).
- Document API for src/plugin/*
- Document API for src/analyzer/Analyzer.h
- Document API for scripts/base/frameworks/analyzer
Other misc:
- Remove HTTP::MD5 notice.
- Add "last_active" field to FileAnalysis::Info record.
- Replace "conn_uids", "conn_ids" fields in FileAnalysis::Info record
with just a "conns" fields containing full connection records.
- The http-methods unit test is failing now, but I think it will be
fixed once I change the file handle callback mechanism to use events
instead.
* send end_of_data event for all kind of streams
* send process_finished event containing exit code of child process for executed programs
* move raw-tests to separate directory
* expose name of input stream to readers
* better handling of some error cases in raw reader
* new force_kill option for raw reader which SIGKILLs progesses on exit
The ordering of events how they arrive in the main loop is a bit peculiar at the moment.
The process_finished event arrives in scriptland before all of the other events, even though
it should be sent last. I have not yet fully figured that out.
- Moved the Notice::notice event and Notice::policy table to both be hooks.
- Renamed the old Notice::policy to Notice::policy_table and documented it as deprecated.
Sadly there also seems to be another deadlock issue which I am currently
not really able to figure out - on shutdown sometimes (too often) the main
thread + all sqlite threads wait for semaphores or mutexes.
- Removed default logging. Now a function is available for the new
$period_finished filter field to get the same behavior for logging
named Metrics::write_log.
- Added index rollups for getting multiple metrics result values
as the same time.
'only_single_header_row' that turns the output into CSV format.
In that mode all meta data is skipped except for a single header line
with the fields names. Example:
local my_filter: Log::Filter = [$name = "my-filter", $writer = Log::WRITER_ASCII, $config = table(["only_single_header_row"] = "T")];
Contributed by Carsten Langer.
* origin/topic/seth/intel-framework: (21 commits)
Extracting URLs from message bodies over SMTP and sending them to Intel framework.
Small comment updates in the Intel framework CIF support.
Intelligence framework documentation first draft.
Only the manager tries to read files with the input framework now.
Initial support for Bro's Intel framework with the Collective Intelligence Framework.
Initial API for Intel framework is complete.
Fixed an issue with cluster data distribution.
Updating some intel framework test baselines.
Reworked cluster intelligence data distribution mechanism and fixed tests.
Lots more intelligence checking in SMTP traffic.
Added intelligence check for "Received" path checking and a bit of reshuffling.
Added sources to the intel log.
Fixing a problem with intel distribution on clusters.
Updated intel framework test to include matching.
Restructuring the scripts that feed data into the intel framework slightly.
One test for cluster transparency of the intel framework.
Fixed a cluster support bug.
Intelligence framework checkpoint
Major updates to fix the Intel framework API.
Checkpoint commit. This is all a huge mess right now. :)
...
Closes#914.
