This admittedly is a quite esoteric combination of protocols. But - as
we do correctly support them, it seems nice to have a slightly more
complete testcase that covers this.
Modified ubuntu24_clang_libcpp_task to be skipped on PRs.
* origin/topic/etyp/ubuntu-clang-ci:
CI: Upgrade FreeBSD 14 to 14.1
CI: Add Ubuntu Clang task
Fix up minor warnings in touched files
Fix Clang 19 deprecation failure
* origin/topic/awelzel/data-end-offset-v1:
signatures: Add data_end_offset to signature_match() and custom events
Add pattern_end_offset to signature_state
This feels more like a band-aid solution - it still requires very manual
intervention to get the next builds, but at least this way we don't rely
on whatever Clang/libc++ happens to ship with FreeBSD for the only
libc++ testing. It seems like sanitizer builds don't use libc++, but I'd
rather a dedicated Linux/Clang/libc++ task at least.
Fixes#3994
Clang 19 with libc++ started failing to compile because the default
implementation of `std::char_traits` was removed, making uses of
`std::char_traits<unsigned char>` invalid. This was more of used for
convenience before, but it should be roughly the same behavior with
`char`.
See relevant LLVM commits:
aeecef08c308a0faf4cd
This change tracks the current offset (number of bytes fed into matchers)
on the top-level RuleEndpointState such that we can compute the relative ending
for matched texts individually.
Additionally, it adds the data_end_offset as a new optional parameter to
signature_match().
This commit fixes three issues with Zeek's Modbus message logging:
1 - Some exception responses (e.g., READ_COILS_EXCEPTION) are logged
twice: once without and once with the exception message.
2 - Some exception responses (e.g., PROGRAM_484_EXCEPTION) are not
logged.
3 - Some known but reserved function codes (e.g., PROGRAM_UNITY) are
logged as unk-xxx (e.g., unk-90), while it would be possible to
log their known name.
To address these inconsistencies, the modbus parser has been updated
to parse all exception responses (i.e., all responses where the MSB
of the function code is set) using the already defined Exception
message.
Also, the Modbus main.zeek script has been updated to consistently
demand logging exception responses to the specialized
modbus_exception event, rather than logging some exception responses
in the modbus_message event and others in the modbus_exception event.
Finally, the main.zeek script has been updated to make sure that
for every known function code, the corresponding exception code was
also present, and the enumeration of known function codes in
consts.zeek has been expanded.
Closes#3984
Add pattern_end_offset to signature_state
Update init-bare.zeek
Update RuleMatcher.cc
Update RuleMatcher.h
Update init-bare.zeek
clang format
clang format
clang format
Using Match Offsets List
Temp commit
* origin/topic/vern/CPP-when-capture-naming:
fixed "-O gen-C++" naming of "when" captures to avoid ambiguities due to inlining
Simplified & made more robust maintenance helper script for "-O gen-C++" testing
"-a cpp" baseline updates to reflect recent BTest changes
remove instance of plus sign to account for real plus in sql
account for spaces encoding to plus signs in sqli regex detection
add test cases for sqli space to plus
account for spaces encoding to plus signs in sqli regex detection
forgot semicolon
account for spaces encoding to plus signs in sqli regex detection
* topic/christian/telemetry-make-bifs-primary:
Telemetry framework: move BIFs to the primary-bif stage
Minor comment tweaks for init-frameworks-and-bifs.zeek
Adding a metric for the network time value itself should make it
possible to observe it stopping or growing slowly as compared to
realtime when Zeek isn't able to keep up.
Also, modify the telemetry/log.zeek test to include misc/stats and
log at a higher frequency with a more interesting pcap.
This stops invoking Telemetry::sync() via a scheduled event and instead
only invokes it on-demand. This makes metric collection network time
independent and lazier, too.
With Prometheus scrape requests being processed on Zeek's main thread
now, we can safely invoke the script layer Telemetry::sync() hook.
Closes#3947
This commit prevents most non-Modbus TCP traffic on port 502 to be
reported as Modbus in conn.log as well as in modbus.log.
To do so, we have introduced two &enforce checks in the Modbus
protocol definition that checks that some specific fields of the
(supposedly) Modbus header are compatible with values specified in
the specs.
To ensure non-regression, with this commit we also introduce a
new btest.
Closes#3962
* topic/vern/script-opt-headers-factoring:
factored CPP source's main header into collection of per-source-file headers
renamed script optimization Attrs.h header to prepare for factoring large Compile.h
factored ZAM source's main header into collection of per-source-file headers
This moves the Telemetry framework's BIF-defined functionalit from the
secondary-BIFs stage to the primary one. That is, this functionality is now
available from the end of init-bare.zeek, not only after the end of
init-frameworks-and-bifs.zeek.
This allows us to use script-layer telemetry in our Zeek's own code that get
pulled in during init-frameworks-and-bifs.
This change splits up the BIF features into functions, constants, and types,
because that's the granularity most workable in Func.cc and NetVar. It also now
defines the Telemetry::MetricsType enum once, not redundantly in BIFs and script
layer.
Due to subtle load ordering issues between the telemetry and cluster frameworks
this pushes the redef stage of Telemetry::metrics_port and address into
base/frameworks/telemetry/options.zeek, which is loaded sufficiently late in
init-frameworks-and-bifs.zeek to sidestep those issues. (When not doing this,
the effect is that the redef in telemetry/main.zeek doesn't yet find the
cluster-provided values, and Zeek does not end up listening on these ports.)
The need to add basic Zeek headers in script_opt/ZAM/ZBody.cc as a side-effect
of this is curious, but looks harmless.
Also includes baseline updates for the usual btests and adds a few doc strings.
This pins the generate-docs GH workflow to run on a fixed version of
ubuntu so we get defined semantics of `pip install` wrt system packages.
We were previously using the `ubuntu-latest` image and GH has been
switching this from ubuntu-22.04 to 24.04 and back again, also see
https://github.com/actions/runner-images/issues/10636.
