Since enabling Spicy, 500M (default) of ccache max-size size is not
sufficient for efficient cache usage of a Zeek build anymore, so we
end up thrashing the configured ccache. Build times have increased
from 5-10 minutes to 1+ hour after enabling Spicy in the docs build.
When inspecting Spicy values for marshalling to Zeek we should receive
field identifiers as owned `std::string` so it should be safe to
directly compare them against `const char[]`. It seems that e.g.,
gcc-9.4.0's analysis sees things differently and warns of a comparision
of `const char*` (which as written would compare pointer values; the
correct API for this would be `::strlen`).
This patch addresses that warning by visiting with type which supports
`==` comparison, in particular, take a `std::string_view` so we could
even visit `const char*` identifiers.
When there are errors reported during yyparse(), Zeek still continued
running initialization functions like init_general_global_var(), init_net_var()
and run_bif_initializers(). These usually call abort() in unexpected
situations causing misleading and confusing errors. This patch prevents
this by exiting earlier.
Closes#3316
These tasks are meant to run nightly on the master branch. Currently,
the external dns, http and dhcp Spicy analyzers are installed via zkg
post building. The build artifact is uploaded to Cirrus and the benchmarker
API triggered.
For the spicy_head task, the auxil/spicy submodule is pulled to the latest
commit. This also provides a bit of a nightly integration test.
* origin/topic/bbannier/btest-issue-98:
Use unique port for test `scripts.base.utils.active-http`
Do not automatically load Spicy modules in dot paths.
Exclude dot directories when searching ZEEK_PLUGIN_PATH
Dot directories rarely contain anything we would want to load as a
dynamic plugin. Even worse, they likely contain files with externally
controlled lifetimes which might be removed while we are using them
(see e.g., zeek/btest#98).
With this patch we do not search _discovered_ dot directories anymore.
We continue to load from a user-specified `ZEEK_PLUGIN_PATH`, even if
its name starts with a dot.
Since this patch changes previous behavior it is a **BREAKING CHANGE**.
On Slack, a user reported "fortify source" aborts within ares_fds() due
to the FDs used by c-ares exceeding 1024 and thereby larger than the
maximum fd value that a fd_set can hold.
Switch to ares_get_socks() and poll() to avoid this.
Closes#3309.
Similar motivation as for RecordVal, save an extra malloc/free
and pointer indirection.
This breaks the `auto& RawVec()` API which previously returned
a reference to the std::vector*. It now returns a reference
to the vector instead. It's commented as intended for internal
and compiled code, so even though it's public API,
The previous `std::vector<std::optional<ZVal>>*&` return type was also very
likely not intended (all consumers just dereference it anyhow). I'm certain
this API was never meant to modify the actual pointer value.
I've switched to explicit typing, too.
A user reported that they ran into a situation where a parse error
caused an assertion failure in `reject_protocol`. printf debugging points
to the `CookieSetter`s in the `try`/`catch` blocks during processing
already clearing the cookie RAII-style; since their `catch` blocks
already send an analyzer violation no further sending of one from
`reject_protocol` is required.
This patch replaces the assert in `reject_protocol` with a runtime check
and an early return if the cookie is already cleared.
* origin/topic/gh-196-spicy-docs:
Add check to docs CI whether Spicy docs are up to date.
Bump zeek-docs.
Enable Spicy when generating docs.
Zeekygen: Merge in Spicy analyzer documentation.
Zeekygen: Refine location tracking for Spicy events.
Zeekygen: Refactor code documenting analyzers.
Add `%doc-*` tags to Syslog and Finger analyzers.
Spicy: Support additional documentation tags inside EVT files.
Fix lack of namespace qualification inside an include file.
Exclude script from documentation that interferes with other locations.
When Spicy defines an event through EVT, it would end up with
more-or-less random, stale Zeek-side location information. Now making
sure it's unset, plus then using any script-side prototype we may
encounter later to update it.
Also fixing a couple location comparisons that I believe were broken
because we do sometimes copy location objects and hence can't compare
by pointer.
No functional change, but we refactor the logic to
1. Untie the function signature from the fact that the information
currently comes only out of plugins.
2. Avoid some redundancies
In the next commit, we'll use this to add information from Spicy EVT
files into the output.
So far we had trouble documenting Spicy analyzers through Zeekygen
because they would show up as components belonging to the
`Zeek::Spicy` plugin; whereas traditional analyzers would be their own
plugins and hence documented individually on their own. This commit
teaches Zeekygen to track Spicy analyzers separately inside their own
`Info` instances. This information isn't further used in this commit
yet, but will be merged with the plugin output in a subsequent change
to get the expected joint output.
To pass additional information to Zeekygen, EVT files now also support
two new tags for Zeekygen purposes:
- `%doc-id = ID;` defines the global ID under which everything inside
the EVT file will be documented by Zeekygen, conceptually comparable
to plugin names (e.g., `Zeek::Syslog`).
- `%doc-description = "text" provides additional text to go into the
documentation (comparable to plugin descriptions).
This information is carried through into the HLTO runtime
initialization code, from where it's registered with Zeekygen.
This commit also removes a couple of previous hacks of how Spicy
integrated with Zeekygen which (1) ended up generating broken doc output
for Spicy components, and (2) don't seem to be necessary anymore
anyways.
