* origin/topic/awelzel/2572-event-handler-stats:
Bump external baselines
external/testing-setup: Less telemetry.log volume
EventHandler: Use telemetry framework for EventHandler.call_count
For traces spanning a long time period, don't produce immense
amounts of logs. Also filter down the event handler invocation
entries to connection related events in telemetry.log.
Put the IntCounter into a std::optional rather than initializing
it at EventHandler construction time as that will currently expose
a time series per event handler through the Prometheus API.
The user and password fields are replicated to each of the ftp.log
entries. Using a very large username (100s of KBs) allows to bloat
the log without actually sending much traffic. Further, limit the
arg and reply_msg columns to large, but not unbounded values.
It's happening regularly to me that I forget the type specifier when redef'ing
records or enums and usually it takes me a while to figure out what's going
on as the errors are not descriptive. Improve the error reporting and just
bail as there's no sensible way to continue.
Closes#2777
* ekoyle/add-protocol-pbb:
Update seemingly-unrelated btests
Use a default analyzer
Simplify PBB analyzer by using Ethernet analyzer
Add btest for PBB and update baselines
Use constexpr instead of #define
Cleanup and add customer MAC addresses
Add PBB (802.1ah) support
We previously would include any and all output from stderr during
compilation in the test baseline. Depending on the used compiler this
output may contain C++ compilation warnings which are uninteresting for
the behavior under test.
(cherry picked from commit 5221edf474)
* origin/topic/awelzel/propagate-on-change-through-copy:
TableVal: Propagate &on_change attribute through copy()
testing/btest: Add test showing &expire_func/&create_expire is copied
Copying an &ordered table or set would result in a copy that is not ordered.
This seems rather surprising behavior, so propagate the &ordered attribute.
Closes#2793
Mostly for consistency with &default, &expire_func and other attributes
being propagated through a copy(). Seems this was just missed during
the implementation and/or was never tested for.
After the first 4 bytes, this traffic actually just looks like Ethernet.
Rather than try to re-implement the ethernet analyzer, just check the
length, skip 4 bytes, and pass it on.
This adds a new utility called ci/collect-repo-info.py to produce a JSON
document that is then baked into the Zeek executable file. Further, when
creating a tarball via `make dist`, put a top-level repo-info.json file
in place that is picked when no .git directory exists.
Closes#1405
We previously used the Spicy plugin's `Spicy::available` to test for
Spicy support. However, having Spicy support does not necessarily mean that we
have built Zeek with its in-tree Spicy analyzers: the Spicy plugin
could have been pulled in from external. The new BIF now reliably
tells us whether the Spicy analyzers are available; its result
corresponds to what `zeek-config --have-spicy-analyzers` returns as
well.
We also move the two current checks over to use this BIF.
(Note: I refrained from renaming the CMake-side `USE_SPICY_ANALYERS`
to `HAVE_SPICY_ANALYZERS`. We should do this eventually for
consistency, but I didn't want to make more changes than necessary
right now.)
* origin/topic/awelzel/arm64-container-follow-ups-2:
Bump private testsuite for char handling fixes
Base64: report byte as positive integer
netbios_decode: use unsigned char for result
analyzer/http: Do not assume char is signed
cirrus: Run tests in a Debian 11 container, too
A baseline difference between arm64 and x86 showed up. We would
print a wrong character as negative value on x86 due to chars
being signed by default. Force an unsigned interpretation which
is also more reasonable because we'd have never indexed the
base64 table with -112
-XXXXXXXXXX.XXXXXX XXXXXXXXXXX 131.243.99.154 3288 193.159.183.138 80 base64_illegal_encoding character -112 ignored by Base64 decoding F zeek -
+XXXXXXXXXX.XXXXXX XXXXXXXXXXX 131.243.99.154 3288 193.159.183.138 80 base64_illegal_encoding character 144 ignored by Base64 decoding F zeek -
Fixes more of #2742
The buf[i] < 3 condition in use previously allowed all chars (signed on x86)
through that had the 0x80 high-bit set after reconstructing from the two
bytes of the netbios name, resulting in escaped non-ascii content in
the logs.
Fixes more of #2742
This avoids interference from other log streams in the policy hook test cases,
which could cause deviations in output vs baselines depending on build
configuration.
* security/topic/awelzel/125-ftp-timeout-three:
testing/ftp: Add tests and pcaps with invalid reply lines
ftp: Harden reply handing a bit and don't raise bad replies to script-land
ftp: ignore invalid commands
As initial examples, this branch ports the Syslog and Finger analyzers
over. We leave the old analyzers in place for now and activate them
iff we compile without any Spicy.
Needs `zeek-spicy-infra` branches in `spicy/`, `spicy-plugin/`,
`CMake/`, and `zeek/zeek-testing-private`.
Note that the analyzer events remain associated with the Spicy plugin
for now: that's where they will show up with `-NN`, and also inside
the Zeekygen documentation.
We switch CMake over to linking the runtime library into the plugin,
vs. at the top-level through object libraries.
Add configurability of synchronous and journal_mode for SQLite backed
Broker data stores. Setting these to synchronous=normal and journal_mode=wal
can significantly improve throughput at the cost of some durability in
the presence of power loss or OS crash. In the context of Zeek, this is
likely more than acceptable.
Additionally, add integrity_check and failure_mode options to support deleting
and re-opening a corrupted SQLite database at store creation.
Closes#2698
b41a4bf06d removed a field from this record
because it had a duplicate name as another field. The field does need to
exist, but it needs the correct name.
The added test cases around function/event invocations report the
following flurry of errors when only the first one is relevant and
actionable. There's little use in reporting a mismatch with "error".
Squelch them.
error in <...>/function-invoke-mismatch-error.zeek, line 8: identifier not defined: MyEnumTypo
error in <...>/function-invoke-mismatch-error.zeek, line 12 and error: type mismatch (M::MY_ENUM_A and error)
error in <...>/function-invoke-mismatch-error.zeek, line 12: argument type mismatch in function call (M::to_string(M::MY_ENUM_A))
error in <...>/function-invoke-mismatch-error.zeek, line 16 and error: type mismatch (M::MY_ENUM_B and error)
error in <...>/function-invoke-mismatch-error.zeek, line 16: argument type mismatch in function call (M::to_string(M::MY_ENUM_B))
error in <...>/function-invoke-mismatch-error.zeek, line 20 and error: type mismatch (M::e and error)
error in <...>/function-invoke-mismatch-error.zeek, line 20: argument type mismatch in function call (M::to_string(M::e))
Record coercion also reports noisy errors when coercing to a type that
has errors for individual fields, type clashing with "error":
$ zeek language/record-field-error.zeek
error in ./language/record-coerce-error.zeek, line 8: identifier not defined: MyEnumTypo
error in ./language/record-coerce-error.zeek, line 19 and ./language/record-coerce-error.zeek, line 5: type clash for field "e" ((coerce [$e=MY_ENUM_B, $s=test] to MyRecord) and MyEnum)
