Adding a metric for the network time value itself should make it
possible to observe it stopping or growing slowly as compared to
realtime when Zeek isn't able to keep up.
Also, modify the telemetry/log.zeek test to include misc/stats and
log at a higher frequency with a more interesting pcap.
This stops invoking Telemetry::sync() via a scheduled event and instead
only invokes it on-demand. This makes metric collection network time
independent and lazier, too.
With Prometheus scrape requests being processed on Zeek's main thread
now, we can safely invoke the script layer Telemetry::sync() hook.
Closes#3947
This commit prevents most non-Modbus TCP traffic on port 502 to be
reported as Modbus in conn.log as well as in modbus.log.
To do so, we have introduced two &enforce checks in the Modbus
protocol definition that checks that some specific fields of the
(supposedly) Modbus header are compatible with values specified in
the specs.
To ensure non-regression, with this commit we also introduce a
new btest.
Closes#3962
* topic/vern/script-opt-headers-factoring:
factored CPP source's main header into collection of per-source-file headers
renamed script optimization Attrs.h header to prepare for factoring large Compile.h
factored ZAM source's main header into collection of per-source-file headers
This pins the generate-docs GH workflow to run on a fixed version of
ubuntu so we get defined semantics of `pip install` wrt system packages.
We were previously using the `ubuntu-latest` image and GH has been
switching this from ubuntu-22.04 to 24.04 and back again, also see
https://github.com/actions/runner-images/issues/10636.
This job uses `ubuntu-latest`, i.e., it will automatically use the
newest version of Ubuntu available. On current versions `pip install`
refuses to install packages into the system since this might break
system packages. Users are expected to explicitly opt into this
potential breakage by passing the `--break-system-packages` flag. This
is an acceptable workaround here since we do not distribute this setup,
and currently things appear to not break.
The GH action `pre-commit/action` installs `pre-commit` into the system
with `pip`.[^1] On newer distributions this is discouraged as it might break
system packages and would require a dedicated flag
`--break-system-packages` in order to pass.
Since the action does not work in such environments yet, instead pin the
distribution we use for the workflow to a fixed, older and working version.
[^1]: 576ff52938/action.yml (L11)
This isn't a straightforward fix, unfortunately. The existing GetLine()
implementation didn't deal well with input that's incrementally produced
where individually read chunks wouldn't end with the separator.
The prior implementation increased the buffer each time it failed to find
a separator in the current buffer, but then also ended up not searching the
full new buffer size for the terminator, doing that endlessly.
This change reworks the Raw reader to rely only on bufpos for reading
and searching purposes and skip reallocation if the buffer size if it
wasn't actually exhausted.
Closes#3957
Processing out-of-order commands or finishing commands based on invalid
server responses resulted in inconsistent analyzer state, potentially
triggering null pointer references for crafted traffic.
This commit reworks cf9fe91705 such that
too many pending commands are simply discarded, rather than any attempt
being made to process them. Further, invalid server responses do not
result in command completion anymore.
Test PCAP was crafted based on traffic produced by the OSS-Fuzz reproducer.
Closes#215
That test got flaky probably from #3949 on centosstream9 CI. You can
replicate that behavior by increasing the sleep time when waiting for
the file such that the test will attempt to read the missing file again.
Since the one second wait for file is glacially slow for this, speeding
it up should mean that the file gets created sooner and so the test
won't try to open the file again. But, it's always still technically
possible, since the test will wait for 10 seconds and the heartbeat
seems to be 1 second. At least if that happens, it's probably a bug or
massive slowdown of some kind.
It seems like other similar tests get by because they have more "stuff"
before they call `terminate()` most likely. But, to be safe, just
removing the "received termination signal" line seems like the best
approach.
Invalid lines in a file was the one case that would not suppress future
warnings. Just make it suppress warnings too, but clear that suppression
if there is a field in between that doesn't error.
Fixes#3692
* topic/vern/script-opt-maint.Sep24B:
factoring of logic used by ZAM's low-level optimizer when adjusting control flow info
BTest baseline update for more complete function/lambda names
tweak to -O gen-C++ maintenance script to avoid treating plugins as BTests
fixed lambda hash collision bug due to function descriptions lacking full parameter information
fixes (to avoid collisions) for AST profiling's function hash computations
removed unused ZAM cast-to-any operation
fixes for ZAM tracking the return type associated with function calls
ZAM control-flow tracking now explicitly includes the ends of loops
fix for ZAM identification of common subexpressions
"-O dump-final-ZAM" option similar to "dump-ZAM" only prints final version of functions
fix for setting object locations to avoid use-after-free situation
extended "-O allow-cond" to apply to both gen-C++ and gen-standalone-C++
-O gen-C++ fix for run-time warnings for "when" lambdas
fix to -O gen-C++ for recent AST profiling changes for identifying function parameters
fix to -O gen-C++ for dealing with "hidden" parameters
tweak to prevent an incorrect warning for scripts compiled to C++
fixed overly narrow Spicy test for manipulating packet analyzers
fixed memory leak for recursive ZAM functions that exit via an exception
remove unnecessary header include
