This reverts commit 957825441a, reversing
changes made to c8cdc75f2b.
Caused spurious CI failures in the external testing baselines. See zeek/zeek#2842.
This change exposes the signature tyope inside the signed portion of an
X.509 certificate. In the past, we only exposed the signature type that
is contained inside the signature, which is outside the signed portion
of the X.509 certificate.
In theory, both signature fields should have the same value; it is,
however, possible to encode differing values in both fields. The new
field is not logged by default.
This adds one metric per log stream and one metric per log writer (path based)
to track the number of writes on a stream level as well as on a writer level.
$ curl -sSf localhost:8181/metrics | grep Conn
zeek_log_writer_writes_total{endpoint="",filter-name="default",module="HTTP",path="http",stream="HTTP::LOG",writer="Log::WRITER_SQLITE"} 1 1677497572770
zeek_log_stream_writes_total{endpoint="",module="HTTP",stream="HTTP::LOG"} 1 1677497572770
The initial version of this change also included metrics around log
write vetoes, but given no log policies exist in the default configuration
and they are mostly interesting for a few streams/writers only, skip this
for now. These can always be added by the script writer, too.
The difference between the stream level writes and concrete writers can
be used to deduce the number of vetoes (or errors) as a starting point.
When bumping the VERSION file, everything that includes the auto-generated
zeek/zeek-config.h needs to rebuild and ccache usage is voided due the file
changing. Split the mutable version information into a new zeek-version.h
file to avoid this.
Further, do not include zeek-version.h within Plugin.h and avoid the reference
to ZEEK_VERSION_FUNCTION unless we're building an external plugin.
Closes#2776.
Mostly for consistency with &default, &expire_func and other attributes
being propagated through a copy(). Seems this was just missed during
the implementation and/or was never tested for.
When an analyzer calls DataIn(), there's a costly callback construct
going through the event queue. If an analyzer does not have a
get_file_handle() handler installed, the produced file_id would
end up empty and ignored. Consequently, the get_file_handle() callback
was invoked for every new DataIn() invocations.
This is surprising and costly. Log a warning when this happens and
instead set a generically generated file handle value instead to
prevent the repeated get_file_handle() invocations.
* origin/topic/awelzel/2674-arm64-containers-on-cirrus:
cirrus/containers: Do not install btest pcaps
cirrus: Replace curl with _cache
cirrus: Add cluster testing
cirrus: Polish container_image_manifest_docker_builder
cirrus: Use ccache for faster container builds
cirrus: Add docker_builder tasks to build and push images
github: No more docker workflow
Add configurability of synchronous and journal_mode for SQLite backed
Broker data stores. Setting these to synchronous=normal and journal_mode=wal
can significantly improve throughput at the cost of some durability in
the presence of power loss or OS crash. In the context of Zeek, this is
likely more than acceptable.
Additionally, add integrity_check and failure_mode options to support deleting
and re-opening a corrupted SQLite database at store creation.
Closes#2698
This instantiates the SSL analyzer when the client requests SSL
so that Zeek now has a bit more visibility into encrypted MySQL
connections.
The pattern used is the same as in the IMAP, POP or XMPP analyzer.
* topic/awelzel/2696-mysql-analyzer-issues:
testing/mysql: Add traces recorded with a free-tier MySQL instance
MySQL: Fix endianness, introduce mysql_eof() event
When disabling_analyzer() was introduced, it was added to the GLOBAL
module. The awkward side-effect is that implementing a hook handler
in another module requires to prefix it with GLOBAL. Alternatively, one
can re-open the GLOBAL module and implement the handler in that scope.
Both are not great, and prefixing with GLOBAL is ugly, so move the
identifier to the Analyzer module and ask users to prefix with Analyzer.
* origin/topic/awelzel/analyzer-log:
btest/net-control: Use different expiration times for rules
analyzer: Add analyzer.log for logging violations/confirmations
In certain deployment scenarios, all analyzers are disabled by default.
However, conditionally/optionally loaded scripts may rely on analyzers
functioning and declare a request for them.
Add a global set set to the Analyzer module where external scripts can record
their requirement/request for a certain analyzer. Analyzers found in this
set are enabled at zeek_init() time.
This commit adds an optional event_groups field to the Logging::Stream record
to associated event groups with logging streams.
This can be used to disable all event groups of a logging stream when it is
disabled. It does require making an explicit connection between the
logging stream and the involved groups, however.
* origin/topic/awelzel/2629-notice-file-info:
analyzer/files: handle non-analyzer names in describe_file()
frameworks/notice: Handle fa_file with no or more than a single connection better
* When a file is transferred over multiple connection, have
create_file_info() just pick the first one instead of none.
* Do not unconditionally assume cid and cuid as set on a
Notice::FileInfo object.
* 'topic/fox-ds/ssh-key-init-events' of github.com:fox-ds/zeek:
Added several events for detailed info on the SSH2 key init directions
* Straightened out the zeek:see lines in events.bif to be the same across all events.
Python, Ruby, Javascript, Go, ..., allow use of trailing commas and is even
recommended in some style-guides as it keeps diffs smaller. The black formatter
for Python even goes as far to take a trailing comma as an indication to
format a list one-item on a line.
It has been a bit unusual to not be able to put trailing commas in Zeek
scripts, so this change allows for it.
It explicitly prevents trailing commas in list expressions on the left
hand side. Concretely, this disallows trailing commas in the key list
expression during table initializations.
It probably allows for commas in more places that I haven't fully grasped.
Maybe we should tighten those down again if we find them surprising.
I've skipped treating overflows as warnings, as ++ wrapping around at 0
doesn't currently trigger a runtime error and might be expected to be
quiet and silently wrap.
Closes#2486
* origin/topic/vern/vec-construct-compat:
provide deprecated version of merge_type_list()
keep merge_types() externally available; address reviewing suggestion
change vector constructors to require direct type equivalence for non-arithmetics
Two new BIFs get_identifier_declaring_script() and get_record_field_declaring_script()
mirroring existing functions for comments. This allows to query the declaring
script information from Zeek scripts and further determine if a redef operations
were involved for record fields or enum names by comparing the declaring script
with the one of the involved type. See the tests.
Yet another one of @stevesmoot's requests.
* origin/topic/robin/gh-2426-flipping:
Fixing productive connections with missing SYN still considered partial after flipping direction.
Add some missing bits when flipping endpoints.
Seemed easiest to do it via the traversal infrastructure as we do not
otherwise track enough context/scope when instantiating break or next
statements.
Might be worth moving this out of src/parse.y, but didn't exactly know
where. Or maybe we wait until there's more such trivial validations
popping up
Fixes#2440
Using positional and vararg arguments for BIFs, it's not possible to do
proper runtime type checking on them as discussed in #2425. The bifcl produced
code unconditionally attempts to convert the positional arguments to StringVals,
but nothing ever type checks them. Instead of improving the vararg support in
Zeek script and bifcl, align cat_sep() with fmt() in making it fully vararg
and do implement type checks by hand.
With this change, passing wrong types for the separator and default argument
isn't a fatal error anymore and the error messages are also more descriptive.
It's a bit of a crutch working around varargs limitations.
Fixes#2425
One more from @stevesmoot. The record_fields() BIF produced "enum" as
type_name for fields of type enum.
Extend container_type_name() to append the actual name of the enum.
This is changing the format and may break consumers, but those are
likely in a category that are happy to adapt. Not having the actual
enum name available wasn't very helpful.
We could alternatively render only the actual type_name without the
prefixed "enum", but that isn't how it's done for record types currently
and it would make it more difficult to decide which subsequent BIFs to
use for further introspection, like enum_names().
