* origin/topic/awelzel/analyzer-log:
btest/net-control: Use different expiration times for rules
analyzer: Add analyzer.log for logging violations/confirmations
In certain deployment scenarios, all analyzers are disabled by default.
However, conditionally/optionally loaded scripts may rely on analyzers
functioning and declare a request for them.
Add a global set set to the Analyzer module where external scripts can record
their requirement/request for a certain analyzer. Analyzers found in this
set are enabled at zeek_init() time.
This commit adds an optional event_groups field to the Logging::Stream record
to associated event groups with logging streams.
This can be used to disable all event groups of a logging stream when it is
disabled. It does require making an explicit connection between the
logging stream and the involved groups, however.
* origin/topic/awelzel/2629-notice-file-info:
analyzer/files: handle non-analyzer names in describe_file()
frameworks/notice: Handle fa_file with no or more than a single connection better
* When a file is transferred over multiple connection, have
create_file_info() just pick the first one instead of none.
* Do not unconditionally assume cid and cuid as set on a
Notice::FileInfo object.
* 'topic/fox-ds/ssh-key-init-events' of github.com:fox-ds/zeek:
Added several events for detailed info on the SSH2 key init directions
* Straightened out the zeek:see lines in events.bif to be the same across all events.
Python, Ruby, Javascript, Go, ..., allow use of trailing commas and is even
recommended in some style-guides as it keeps diffs smaller. The black formatter
for Python even goes as far to take a trailing comma as an indication to
format a list one-item on a line.
It has been a bit unusual to not be able to put trailing commas in Zeek
scripts, so this change allows for it.
It explicitly prevents trailing commas in list expressions on the left
hand side. Concretely, this disallows trailing commas in the key list
expression during table initializations.
It probably allows for commas in more places that I haven't fully grasped.
Maybe we should tighten those down again if we find them surprising.
I've skipped treating overflows as warnings, as ++ wrapping around at 0
doesn't currently trigger a runtime error and might be expected to be
quiet and silently wrap.
Closes#2486
* origin/topic/vern/vec-construct-compat:
provide deprecated version of merge_type_list()
keep merge_types() externally available; address reviewing suggestion
change vector constructors to require direct type equivalence for non-arithmetics
Two new BIFs get_identifier_declaring_script() and get_record_field_declaring_script()
mirroring existing functions for comments. This allows to query the declaring
script information from Zeek scripts and further determine if a redef operations
were involved for record fields or enum names by comparing the declaring script
with the one of the involved type. See the tests.
Yet another one of @stevesmoot's requests.
* origin/topic/robin/gh-2426-flipping:
Fixing productive connections with missing SYN still considered partial after flipping direction.
Add some missing bits when flipping endpoints.
Seemed easiest to do it via the traversal infrastructure as we do not
otherwise track enough context/scope when instantiating break or next
statements.
Might be worth moving this out of src/parse.y, but didn't exactly know
where. Or maybe we wait until there's more such trivial validations
popping up
Fixes#2440
Using positional and vararg arguments for BIFs, it's not possible to do
proper runtime type checking on them as discussed in #2425. The bifcl produced
code unconditionally attempts to convert the positional arguments to StringVals,
but nothing ever type checks them. Instead of improving the vararg support in
Zeek script and bifcl, align cat_sep() with fmt() in making it fully vararg
and do implement type checks by hand.
With this change, passing wrong types for the separator and default argument
isn't a fatal error anymore and the error messages are also more descriptive.
It's a bit of a crutch working around varargs limitations.
Fixes#2425
One more from @stevesmoot. The record_fields() BIF produced "enum" as
type_name for fields of type enum.
Extend container_type_name() to append the actual name of the enum.
This is changing the format and may break consumers, but those are
likely in a category that are happy to adapt. Not having the actual
enum name available wasn't very helpful.
We could alternatively render only the actual type_name without the
prefixed "enum", but that isn't how it's done for record types currently
and it would make it more difficult to decide which subsequent BIFs to
use for further introspection, like enum_names().
This started with reverting commit 52cd02173d
and then rewriting it to be per handler rather than handler identifier
and adding support for hooks as well as adding implicit module groups.
Mostly: Do not instantiate variables within for loops and allow
reusing differently typed blanks which previously wasn't possible.
This may be missing some corner-cases, but the added tests seem
to work as expected and nothing else fell apart it seems.
In his ZeekWeek 2022 presentation, @stevesmoot mentioned that he had a
difficult time looking up enum names when all he had was a string
naming the type.
Add support to enum_names() to transparently lookup the type if a string
is provided. This is similar in how record_fields() behaves when being
passed a string.
This has come up a few times and the motivation is mainly better "first timer"
experience with Zeek. Concretely, if one wants to run a Zeek cluster with
multiple workers and reasonable load balancing on Linux, AF_PACKET is a decent
start. Without AF_PACKET support being built into Zeek, however, a new user's
next experience is that of setting up a development environment in order
to compile an external plugin (think compiler, kernel headers, zkg, ...).
Only to get what could be termed basic functionality.
This is using the ZEEK_INCLUDE_PLUGINS infrastructure. I've used the all
upper case spelling of AF_PACKET in the help output because it seems everyone
else references/writes it like that. I think we should also write it
like that in the docs.
This allows to enable/disable file analyzers through the same interfaces
as packet and protocol analyzers, specifically Analyzer::disable_analyzer
could be interesting.
This adds machinery to the packet_analysis manager for disabling
and enabling packet analyzers and implements two low-level bifs
to use it.
Extend Analyzer::enable_analyzer() and Analyzer::disable_analyzer()
to transparently work with packet analyzers, too. This also allows
to add packet analyzers to Analyzer::disabled_analyzers.
Add a test parsing a malformed PE file showing that analyzer_violation_info
is raised with the fa_file object set.
It could be interesting to pass through an optional connection if one
exists, but access is provided through f$conns, too.
Introduce two new events for analyzer confirmation and analyzer violation
reporting. The current analyzer_confirmation and analyzer_violation
events assume connection objects and analyzer ids are available which
is not always the case. We're already passing aid=0 for packet analyzers
and there's not currently a way to report violations from file analyzers
using analyzer_violation, for example.
These new events use an extensible Info record approach so that additional
(optional) information can be added later without changing the signature.
It would allow for per analyzer extensions to the info records to pass
analyzer specific info to script land. It's not clear that this would be
a good idea, however.
The previous analyzer_confirmation and analyzer_violation events
continue to exist, but are deprecated and will be removed with Zeek 6.1.
...the only known cases where the `-` for `connection$service` was
handled is to skip/ignore these analyzers.
Slight suspicion that join_string_set() should maybe become a bif
now determine_service() runs once for each connection.
Closes#2388
* topic/christian/supervisor-pre-user-scripts:
Update Management framework to new Supervisor::NodeConfig script fields
Suppress deprecation warnings for the Supervisor's own NodeConfig::script access
Expand Supervisor to support loading additional scripts before user scripts
Avoid the issue outlined in #2289 where the @if or @else is taken as the
statement of an `if`, `for` or `while` by rejecting such constructs.
Effectively this means the following scripts are now rejected:
# Print's "cond true" with Zeek 5.0 even though the `if ( F )`
# should be in effect.
if ( F )
@if ( T )
print "cond true";
@else
print "cond false";
@endif
or
# Print's "hello" once with Zeek 5.0
local v = vector( 1, 2, 3 );
for ( i in v )
@if ( T )
print("hello")
@endif
To make above work as intended, additional braces can be used.
if ( T )
{
@if ( cond )
print "cond true";
@else
print "cond false";
@endif
}
for ( i in v )
{
@if ( T )
print("hello")
@endif
}
