When Spicy defines an event through EVT, it would end up with
more-or-less random, stale Zeek-side location information. Now making
sure it's unset, plus then using any script-side prototype we may
encounter later to update it.
Also fixing a couple location comparisons that I believe were broken
because we do sometimes copy location objects and hence can't compare
by pointer.
No functional change, but we refactor the logic to
1. Untie the function signature from the fact that the information
currently comes only out of plugins.
2. Avoid some redundancies
In the next commit, we'll use this to add information from Spicy EVT
files into the output.
So far we had trouble documenting Spicy analyzers through Zeekygen
because they would show up as components belonging to the
`Zeek::Spicy` plugin; whereas traditional analyzers would be their own
plugins and hence documented individually on their own. This commit
teaches Zeekygen to track Spicy analyzers separately inside their own
`Info` instances. This information isn't further used in this commit
yet, but will be merged with the plugin output in a subsequent change
to get the expected joint output.
To pass additional information to Zeekygen, EVT files now also support
two new tags for Zeekygen purposes:
- `%doc-id = ID;` defines the global ID under which everything inside
the EVT file will be documented by Zeekygen, conceptually comparable
to plugin names (e.g., `Zeek::Syslog`).
- `%doc-description = "text" provides additional text to go into the
documentation (comparable to plugin descriptions).
This information is carried through into the HLTO runtime
initialization code, from where it's registered with Zeekygen.
This commit also removes a couple of previous hacks of how Spicy
integrated with Zeekygen which (1) ended up generating broken doc output
for Spicy components, and (2) don't seem to be necessary anymore
anyways.
I wonder if we hit a query string limitation and should instead switch
to post body here. It's the `curl -G -X POST` part making this a POST
with a long query string.
In 36a6770e98 we changed the way Spicy
fields exported as
record fields are represented. This broke the `zeek` debug log which
hardcoded a different representation.
This patch brings the generation of the debug log in line with the
actual AST structure. We also enable debug logging in one test to
validate that it works, at least for the case we hit here.
Primarily for the following fixes:
4dfc0dd Nodejs: Do not use UV_RUN_DEFAULT during InitPostScript()
ade764e Nodejs: Unset kNoInitializeCppgc for Node 20.6 or higher
a38df85 Fix type check of parameters array with zeek.event
deddadd Fix type check of parameters array with zeek.invoke
Setting this option to false does not count missing bytes in files towards the
extraction limits, and allows to extract data up to the desired limit,
even when partial files are written.
When missing bytes are encountered, files are now written as sparse
files.
Using this option requires the underlying storage and utilities to support
sparse files.
In the past, we allocated a buffer with zeroes and wrote that with
fwrite. Now, instead we just fseek to the correct offset.
This changes the way in which the file extract limit is counted a bit;
skipped bytes do no longer count against the file size limit.
(cherry picked from commit 5071592e9b7105090a1d9de19689c499070749d4)
If an index expression in a table constructor isn't a list, don't abort with an
assertion failure (on debug builds) or a bad tag check. Instead, mark the
constructor expression erroneous and return gracefully. The following...
global a_table: table[subnet] of string = {
1.2.3.4/24 = "unspecified", # should have been [1.2.3.4/24] = "unspecified",
};
...now yields:
$ zeek ./test.zeek
error in ././test.zeek, line 2: table constructor index is not a list (1.2.3.0/24 = unspecified)
error in ././test.zeek, line 2: type clash in assignment (a_table = table(1.2.3.0/24 = unspecified))
When checking exported Spicy types for collisions with existing Zeek
types we previously would also check whether they collide with names in
global scope, i.e., we didn't provide a `no_global` arg to
`detail::lookup_ID` which defaulted to false (since we also provided a
module name I'd argue that the behavior of that function is confusing
and probably error-prone -- like seen here).
This meant that e.g., a Spicy enum `foo::Direction` (automatically in
implicit Spicy module scope) would be detected to collide with the
existing Zeek `Direction` enum.
With this patch we use the `lookup_ID` API correctly and do not check
against potential collisions with globals anymore since it is not
needed.
Closes#3279.
Setting this option to false does not count missing bytes in files towards the
extraction limits, and allows to extract data up to the desired limit,
even when partial files are written.
When missing bytes are encountered, files are now written as sparse
files.
Using this option requires the underlying storage and utilities to support
sparse files.
(cherry picked from commit afa6f3a0d3b8db1ec5b5e82d26225504c2891089)
OSS Fuzz generated a CWD request and reply followed by very many EPRT
requests. This caused Zeek to re-log the CWD request and invoke `build_url_ftp()`
over and over again resulting in long processing times.
Avoid this scenario by not logging commands that aren't pending anymore.
(cherry picked from commit b05dd31667ff634ec7d017f09d122f05878fdf65)
A call to `extract_filename_from_content_disposition()` is only
efficient if the string is guaranteed to contain the pattern that
is removed by `sub()`. Due to missing brackets around the `[:blank:]`
character class, an overly long string (756kb) ending in
"Type:dtanameaa=" matched the wrong pattern causing `sub()` to
exhibit quadratic runtime. Besides that, we may have potentially
extracted wrong information from a crafted header value.
(cherry picked from commit 6d385b1ca724a10444865e4ad38a58b31a2e2288)
* origin/topic/awelzel/deferred-default-non-const-v4:
CreationInitsOptimizer: Use PreTypedef() instead of PreType()
Fix deferred record initialization
testing/btest: Un-deferred record initalization tests
PreTypedef() does not require the PreID() trampoline and ensures
it's only called for IDs that are types. Also allows dropping
the const_cast<> due to id->GetType() returning a const TypePtr which
is different from a `const Type*`...
Put RecordFieldInit instances into creation_inits during parsing and
determine their deferrability in an InitPostScript step. Any
RecordFieldInits can be deferred are moved into deferred_inits.
Closes#3260
* origin/topic/vern/script-opt-maint.Sep23:
fix for ZAM statement-level profiling (broken by GH-3199)
ZAM fixes for compatibility with GH-3249 changes
-O gen-C++ fixes for compatibility with GH-3249 changes minor -O gen-C++ BTest updates
minor BTest reordering to diminish differences with script optimization
* origin/topic/awelzel/3267-avoid-unnecessary-inits:
ID/Stmt: Introduce INIT_SKIP and use in ForStmt
CompositeHash: Skip record initialization when recovering vals
Currently, loop vars are added to a function scope's inits and
initialized upon entering a function with default values. This
applies to vector, record and table types.
This is unnecessary for variables used in for loops as they are
guaranteed to be initialized while iterating.
Initializing fields of recovered records caused running &default expression
of fields just so that they are re-assigned in the next step with the
recovered fields. The second test case still shows that the loop var
is initialized as well even though that's not needed.
Add tests for iterating over records with &default attributes for both,
tables and vectors.
Fixes#3267
* origin/topic/awelzel/3275-fix-with-python:
ci: update-alternative to have python3 be python3.9
Add OpenSUSE Leap 15.5 to ci
configure/CMakelists: Use Python_EXECUTABLE as hint for interpreter
