This allows to enable/disable file analyzers through the same interfaces
as packet and protocol analyzers, specifically Analyzer::disable_analyzer
could be interesting.
This adds machinery to the packet_analysis manager for disabling
and enabling packet analyzers and implements two low-level bifs
to use it.
Extend Analyzer::enable_analyzer() and Analyzer::disable_analyzer()
to transparently work with packet analyzers, too. This also allows
to add packet analyzers to Analyzer::disabled_analyzers.
Introduce two new events for analyzer confirmation and analyzer violation
reporting. The current analyzer_confirmation and analyzer_violation
events assume connection objects and analyzer ids are available which
is not always the case. We're already passing aid=0 for packet analyzers
and there's not currently a way to report violations from file analyzers
using analyzer_violation, for example.
These new events use an extensible Info record approach so that additional
(optional) information can be added later without changing the signature.
It would allow for per analyzer extensions to the info records to pass
analyzer specific info to script land. It's not clear that this would be
a good idea, however.
The previous analyzer_confirmation and analyzer_violation events
continue to exist, but are deprecated and will be removed with Zeek 6.1.
...the only known cases where the `-` for `connection$service` was
handled is to skip/ignore these analyzers.
Slight suspicion that join_string_set() should maybe become a bif
now determine_service() runs once for each connection.
Closes#2388
* origin/topic/awelzel/dpd-analyzer-merger:
analyzer/dpd: Address review comments
Remove @load base/frameworks/dpd from tests
frameworks/dpd: Move to frameworks/analyzer/dpd, load by default
scripts/dce-rpc,ntlm: Do not load base/frameworks/dpd
btest: Remove unnecessary loading of frameworks/dpd
In supervised nodes, the Supervisor's NodeConfig$scripts vector adds scripts to
the end of the user-provided scripts (options.scripts_to_load), so they load
_after_ any user-provided ones. This can cause confusing redef pitfalls when
users expect their customizations to run last, as they normally do.
This adds two members in Supervisor::NodeConfig, `addl_base_scripts` and
`addl_user_scripts`, to store scripts to load before and after the user scripts,
respectively. The latter serves the same purpose as the old `scripts` member,
which is still there but deprecated (in scriptland only). It functions as
before, after any scripts added via `addl_user_scripts`.
* Because frameworks/analyzer is loaded via init-frameworks-and-bifs the
dpd functionality (really just dpd.log and disabling of analyzers) is
now enabled even in bare mode.
* Not sure we need to keep frameworks/base/dpd/__load__.zeek around
or can just remove it right away.
This is a script-only change that unrolls File::Info records into
multiple files.log entries if the same file was seen over different
connections by single worker. Consequently, the File::Info record
gets the commonly used uid and id fields added. These fields are
optional for File::Info - a file may be analyzed without relation
to a network connection (e.g by using Input::add_analysis()).
The existing tx_hosts, rx_hosts and conn_uids fields of Files::Info
are not meaningful after this change and removed by default. Therefore,
files.log will have them removed, too.
The tx_hosts, rx_hosts and conn_uids fields can be revived by using the
policy script frameworks/files/deprecated-txhosts-rxhosts-connuids.zeek
included in the distribution. However, with v6.1 this script will be
removed.
Adds base/frameworks/telemetry with wrappers around telemetry.bif
and updates telemetry/Manager to support collecting metrics from
script land.
Add policy/frameworks/telemetry/log for logging of metrics data
into a new telemetry.log and telemetry_histogram.log and add into
local.zeek by default.
When setting up the DPD info we previously would get the
`transport_proto` for the connection with `get_conn_transport_proto`.
This function takes a `conn_id` and would fail fatally if the connection
for the given ID was unknown. It seems it was possible to run into such
scenarios when the `analyzer_violation` event was processed after the
connection had been cleaned up.
We now get the `transport_proto` directly from the ports in the
`connection` passed into `analyzer_violation` via
`get_port_transport_proto` which cannot fail.
* origin/topic/awelzel/2120-logdir-leftover:
sqlite default-logdir test: Remove ls ./logs baseline
logging/sqlite: Recognize Log::default_logdir and place files there if set
logging: Introduce Log::default_logdir deprecate LogAscii::logdir and per writer logdir
logging/ascii: Fix .shadow paths when using LogAscii::logdir
Also modify FormatRotationPath to keep rotated logs within
Log::default_logdir unless the rotation function explicitly
set dir, e.g. by when the user redef'ed default_rotation_interval.
* topic/christian/gh-2134-fix-intel-test-races:
Expand scripts.base.frameworks.intel.cluster-transparency test
Fix races in scripts.base.frameworks.intel.cluster-transparency-with-proxy test
Add Intel::send_store_on_node_up boolean to control min_data_store delivery
This exposes Broker's new WebSocket support in Zeek. To enable it,
call `Broker::listen_websocket()`. Zeek will then start listening on
port 9997 for incoming WebSocket connections.
See the Broker documentation for a description of the message format
expected over these WebSocket connections.
This adds a redefinable const to the internals of the Intel framework, to allow
suppression of the manager sending its current min_data_store when a worker
connects. This feature is desirable for nodes that check in "late" to bring them
up to speed, but during testing it introduces nondeterminism.
The Supervisor generates this event every time it receives a status update from
the stem, meaning a node got created or re-created. A corresponding
SupervisorControl::node_status event relays the same information for users
interacting with the Supervisor over Broker.
* origin/topic/vern/script-profiling:
tidy up after generating profile
test suite updates for refined script coverage, use of new BiF to speed startup
fix for coverage reporting for functions that use "when" statements
new global_options() BiF to speed up startup, plus a micro-preen
hooks for new --profile-scripts option
classes for managing script profiles
address some holes in script coverage
fix for script coverage missing on-exit activity
memory management fixes for loggers
make curr_CPU_time() broadly available rather than just isolated to ZAM
I needed to figure out which exact algorithm we use for our
probabilistic top-k measurements. It turns out that we do not mention
this in our source tree at all so far.
These allow packet analyzers to register ports as identifiers to forward from
parent analyzers, while also adding those ports to the now-global
Analyzer::ports table at the same time.
