* topic/awelzel/topic/awelzel/2698-sqlite-backend-options:
broker/store: Extend SQLiteOptions around data safety and performance
auxil/broker: Bump submodule
Add configurability of synchronous and journal_mode for SQLite backed
Broker data stores. Setting these to synchronous=normal and journal_mode=wal
can significantly improve throughput at the cost of some durability in
the presence of power loss or OS crash. In the context of Zeek, this is
likely more than acceptable.
Additionally, add integrity_check and failure_mode options to support deleting
and re-opening a corrupted SQLite database at store creation.
Closes#2698
b41a4bf06d removed a field from this record
because it had a duplicate name as another field. The field does need to
exist, but it needs the correct name.
* origin/topic/timw/pvs-studio-findings:
Add tooling section to README
A bunch of other minor issues reported by PVS-Studio
Simplify setup of IPv6 DNS resolvers
Add missing assignment operators to RobustDictIterator
Fix some member-variable shadowing issues
Fix/simplify some if statement comparisons
Fix a few minor int pointer value initializations
Fix a bunch of missing class member initializations
Remove unnecessary virtual specifiers from final classes
The added test cases around function/event invocations report the
following flurry of errors when only the first one is relevant and
actionable. There's little use in reporting a mismatch with "error".
Squelch them.
error in <...>/function-invoke-mismatch-error.zeek, line 8: identifier not defined: MyEnumTypo
error in <...>/function-invoke-mismatch-error.zeek, line 12 and error: type mismatch (M::MY_ENUM_A and error)
error in <...>/function-invoke-mismatch-error.zeek, line 12: argument type mismatch in function call (M::to_string(M::MY_ENUM_A))
error in <...>/function-invoke-mismatch-error.zeek, line 16 and error: type mismatch (M::MY_ENUM_B and error)
error in <...>/function-invoke-mismatch-error.zeek, line 16: argument type mismatch in function call (M::to_string(M::MY_ENUM_B))
error in <...>/function-invoke-mismatch-error.zeek, line 20 and error: type mismatch (M::e and error)
error in <...>/function-invoke-mismatch-error.zeek, line 20: argument type mismatch in function call (M::to_string(M::e))
Record coercion also reports noisy errors when coercing to a type that
has errors for individual fields, type clashing with "error":
$ zeek language/record-field-error.zeek
error in ./language/record-coerce-error.zeek, line 8: identifier not defined: MyEnumTypo
error in ./language/record-coerce-error.zeek, line 19 and ./language/record-coerce-error.zeek, line 5: type clash for field "e" ((coerce [$e=MY_ENUM_B, $s=test] to MyRecord) and MyEnum)
This instantiates the SSL analyzer when the client requests SSL
so that Zeek now has a bit more visibility into encrypted MySQL
connections.
The pattern used is the same as in the IMAP, POP or XMPP analyzer.
* topic/awelzel/2696-mysql-analyzer-issues:
testing/mysql: Add traces recorded with a free-tier MySQL instance
MySQL: Fix endianness, introduce mysql_eof() event
Not sure this adds much more coverage then there was, but minimally
more recent software versions.
The instances/passwords were ephemeral, so hostname and password hashes
etc aren't useful to anyone.
We were parsing MySQL using bigendian even though the protocol is
specified as with "least significant byte first" [1]. This is most
problematic when parsing length encoded strings with 2 byte length
fields...
Further, I think, the EOF_Packet parsing was borked, either due to
testing the CLIENT_DEPRECATE_EOF with the wrong endianness, or due to
the workaround in Resultset processing raising mysql_ok(). Introduce a
new mysql_eof() that triggers for EOF_Packet's and remove the fake
mysql_ok() Resultset invocation to fix. Adapt the mysql script and tests
to account for the new event.
This is a quite backwards incompatible change on the event level, but
due to being quite buggy in general, doubt this matters to many.
I think there is more buried, but this fixes the violation of the simple
"SHOW ENGINE INNODB STATUS" and the existing tests continue to
succeed...
[1] https://dev.mysql.com/doc/dev/mysql-server/latest/page_protocol_basic_dt_integers.html
The script added as a test case reports the following messages *and*
dumps a core file. Printing the first error and a normal failure exit
seems sufficient. IMO triggering an abort() due to user scripting issues
is not something that Zeek should do
$ zeek ./identifier-not-defined-error.zeek
error in ./identifier-not-defined-error.zeek, line 10: identifier not defined: MyEnu
error in ./identifier-not-defined-error.zeek, line 10 and error: &default value has inconsistent type (M::MY_ENUM_A and error)
internal error in ./identifier-not-defined-error.zeek, line 11: type inconsistency in ZVal constructor
Aborted (core dumped)
Change is to skip certain checks when an error type is propagated.
The #124 PR introduced special treatment when HTTP version 0.9
was set. With #127, a reproducer that set HTTP/1.0 in the first
request was created and subsequent requests wouldn't reset to
HTTP version 0.9.
This is subtle, but doesn't seem like things fall apart.
Improves runtime from 20 seconds to 2 seconds for the given
reproducer.
Fixes#127.
These have been created artificially. The tests show that for an
invalid reply line without a numeric code, with a numeric code < 100
or a numeric code not followed by a space we now raise an analyzer
violation and disable the analyzer.
This improves runtime of the oss-fuzz generated traffic in #125.
Specifically, that reproducers included a 064- reply code that was
interpreted as needing to be continued.
Also, return after AnalyzerViolations() for server replies rather than
propagating bad replies them to script-land. This trusts server's to
generally behave according to specification.
* origin/topic/timw/2658-avoid-teredo-violations:
Update external testing repo commit hashes for Teredo changes
Update scripts.base.frameworks.analyzer.logging btest to use a different trace file
Update btest baselines due to reduced calls to UID::Set
Rework TeredoAnalzyer::DetectProtocol to not catch as many packets
