I removed `deprecated-txhosts-rxhosts-connuids.zeek` from
`local.zeek`, seems preferable not to have a script-to-go-away in the
standard configuration for new users. Also tweaked `NEWS` just a tiny
bit.
* origin/topic/awelzel/files-log-unrolling:
files.log: Unroll and introduce uid and id fields
This is a script-only change that unrolls File::Info records into
multiple files.log entries if the same file was seen over different
connections by single worker. Consequently, the File::Info record
gets the commonly used uid and id fields added. These fields are
optional for File::Info - a file may be analyzed without relation
to a network connection (e.g by using Input::add_analysis()).
The existing tx_hosts, rx_hosts and conn_uids fields of Files::Info
are not meaningful after this change and removed by default. Therefore,
files.log will have them removed, too.
The tx_hosts, rx_hosts and conn_uids fields can be revived by using the
policy script frameworks/files/deprecated-txhosts-rxhosts-connuids.zeek
included in the distribution. However, with v6.1 this script will be
removed.
* origin/topic/awelzel/1678-disabling-analyzer-hook:
Add NEWS entry and zeekygen-smithing for disabling_analyzer()
Introduce global disabling_analyzer() hook to veto disable_analyzer()
ssl: Only delete c$ssl$analyzer_id when disabling the analyzer was successful
Add new syntax for adding and removing attributes from record fields:
redef RecordType$field_name += { &log };
redef RecordType$field_name -= { &log };
For now this only allowed for the &log attribute as the semantics are clear.
For &default and &optional the semantics aren't obvious and no use-cases have
been identified where those would make sense to change.
This enables a mechanism to add potentially interesting fields to the typical
Info records in base scripts, but letting users opt-into actually including
them into their log. At the same time, users that find specific fields in a
standard log uninteresting can opt-out without using `Log::Filter$exclude`
which can be difficult to use correctly. Patching or forking external packages
to remove columns from a log can also be avoided with this mechanism.
Closes#2000.
The low-level singleton Telemetry BIFs have been removed with the that there
haven't been any users. Singleton metrics can be instantiated by providing
an empty label vector instead and aren't in any way a special concept.
Closes#2262.
Adds base/frameworks/telemetry with wrappers around telemetry.bif
and updates telemetry/Manager to support collecting metrics from
script land.
Add policy/frameworks/telemetry/log for logging of metrics data
into a new telemetry.log and telemetry_histogram.log and add into
local.zeek by default.
Due to the asynchronous behavior of the input framework and broker
communication, change handlers were previously called even after
zeek_done() event processing completed and also broker shutdown.
Accessing broker store handles within change handlers this late
triggered invalid Broker store handle messages:
error in ././my_option_store.zeek, line 13: invalid Broker store handle (Broker::put(Test::store, to_any_coercemy_option, to_any_coerceTest::new_value, 0 secs) and broker::store::{})
Fixes#2010
* origin/topic/awelzel/2120-logdir-leftover:
sqlite default-logdir test: Remove ls ./logs baseline
logging/sqlite: Recognize Log::default_logdir and place files there if set
logging: Introduce Log::default_logdir deprecate LogAscii::logdir and per writer logdir
logging/ascii: Fix .shadow paths when using LogAscii::logdir
* 'update' of https://github.com/leres/zeek:
Return false on error from the other place we call fstat()
Address concerns raised by @0xxon; avoid the new code path when reading from a pipe and return false if fstat() fails after sucessfully opening the file (unlikely).
Tweak some new comments
Remove child program check, it's probably wrong given the test failures it causes
Conform to style police
Only set mtime and ino in Raw::OpenInput() do this for MODE_STREAM and avoid breaking MODE_REREAD
Implement tail -F semantics for input framework MODE_STREAM Open /dev/null if the file is missing during init and wait for file to be created Collect initial ino, dev, and mtime when first opening the file Detect if the file has been replaced and open the new version
Only set mtime and ino in Raw::OpenInput() do this for MODE_STREAM and avoid breaking MODE_REREAD
Implement tail -F semantics for input framework MODE_STREAM Open /dev/null if the file is missing during init and wait for file to be created Collect initial ino, dev, and mtime when first opening the file Detect if the file has been replaced and open the new version
The Supervisor generates this event every time it receives a status update from
the stem, meaning a node got created or re-created. A corresponding
SupervisorControl::node_status event relays the same information for users
interacting with the Supervisor over Broker.
* origin/topic/bbannier/spicy:
Add NEWS item for new requirements due to default-enabled Spicy support.
Include headers in binary dir before other headers.
Increase memory in CI.
Add tests for bundled Spicy infrastructure.
Include spicy in build.
Add `bare_mode` bif.
Change test so included plugins can load notice framework.
Do not log function arguments in test.
* origin/topic/vern/type-names:
regularize deprecation warning
make DoDescribe protected for a tidier interface mark use of DESC_PORTABLE as deprecated
when printing a description of a type, use its name if available
* topic/christian/gh-1963:
Add cmake-time reporting of bifcl, binpac, and gen-zam used for build
Build Gen-ZAM from a submodule and support use of pre-existing executable
This separates Gen-ZAM from the main Zeek distribution and places it in a
submodule at auxil/gen-zam, reflecting binpac and bifcl. Configuring with
--with-gen-zam=... allws reuse of an existing executable.
* origin/topic/christian/gh-1982:
The is_num(), is_alpha(), and is_alnum() BiFs now return F on empty string
Expand testcases around is_num(), is_alpha(), is_alnum(), is_ascii() BiFs
Clarify is_ascii() BiF docstring re behavior on empty strings
Fix minor indentation bugs in strings.bif
Documentation is missing and will be added in the next couple of hours.
* origin/topic/johanna/tls12-decryption: (24 commits)
TLS decryption: add test, fix small issues
Address PR feedback
TLS decryption: refactoring, more comments, less bare pointers
Small code fix and test baseline update.
SSL decryption: refactor TLS12_PRF
SSL decryption: small style changes, a bit of documentation
Deprecation and warning fixes
Clang-format updates
add missing call to EVP_KDF_CTX_set_params
TLS decryption: remove payload from ssl_encrypted_data again.
TLS 1.2 decryption: adapt OpenSSL 3.0 changes for 1.1
ssl: adapt TLS-PRF to openSSL 3.0
ssl/analyzer: potentially fix memory leaks caused by bytestrings
analyzer/ssl: several improvements
analyzer/ssl: defensive key length check + more debug logging
testing: feature gate ssl/decryption test
testing: add ssl/decryption test
analyzer/ssl: handle missing <openssl/kdf.h>
analyzer/ssl: silence warning in DTLS analyzer
analyzer/ssl: move proc-{client,server}-hello into the respective analyzers
...
