* origin/topic/christian/mmdb-configurability:
Modernize various C++/Zeek-isms in the MMDB code.
Fix MMDB code to re-open explicitly opened DBs correctly
Add btest to verify behavior of re-opened MMDBs opened directly via BIFs
Simplify MMDB code by moving more lookup functionality into MMDB class
Move MMDB logic out of mmdb.bif and into MMDB.cc/h.
Fix mmdb.temporary-error testcase when MMDBs are installed on system
Adapt MMDB BiF code to new script-layer variables
Update btest baselines to reflect introduction of mmdb.bif
Move MaxMind/GeoIP BiF functionality into separate file
Provide script-level configurability of MaxMind DB placement on disk
Sort toplevel .bif list in CMakeLists
ssl-log-ext had a bug that caused data present in the SSL connection to
not be logged in some cases. Specifically, the script relied on the base
ssl script to initialize some data structures; however, this means that
protocol messages that arrive before a message is handled by the base
ssl script are not logged.
This commit changes the ssl-log-ext script to also initialize the data
structures; now messages are correctly included in the log in all cases.
* origin/topic/awelzel/3439-bump-tunnel-max-depth:
NEWS: Update news for tunnel depth changes
tunnels: Add 'X' to history when reaching Tunnel::max_depth
Session/TCP/UDP: Reserve HIST_UNKNOWN_PKT mask
Conn: Deprecated AppendAddl
Conn/Session: Lift history logic into Session
init-bare: Default Tunnel::max_depth to 4
This is meant to be used for a new 'X' code in the history in scenarios when
packets are knowingly not processed or an unexpected unknown situation
is recognized.
Usually, these situations are currently reported via weirds or analyzer violations,
but being able to include it in the history field allows them to be more visible.
Will be used for exceeding tunnel depths first.
This should allow to mangle a Session's history also from packet
analyzers without necessarily knowing the concrete connection type.
Given Connection is a subclass of Session, I don't think this
changes much.
In AWS GLB environments, the max_depth of 2 is easily reached due to packets
being encapsulated with GENEVE and VXLAN [1]. Any additional encapsulation
layer causes Zeek raise a weird and ignore the inner traffic. Bump the default
maximum depth to 4, while not common it's not unusual either to observe
this in the wild.
[1] https://docs.aws.amazon.com/vpc/latest/mirroring/traffic-mirroring-packet-formats.htmlCloses#3439
The filename from which a DB first gets opened (either via an explicitly
specified filename, or via the path sequence now configurable at the script
layer) is now "sticky", meaning re-opening won't switch to a different file.
This was easiest by moving most state into the MMDB class itself. The previous
approach of tracking the two DB instances via a smart pointer and blowing the
pointed-to objects away as needed is now instead one of two objects fixed over
the lifetime of Zeek, able to open/close/reopen their underlying Maxmind DBs.
The MMDB class now only has one Lookup() method since there was no need to break
them apart -- it saves the return of a MMDB_lookup_result_s over the stack and
there's no need for throwing an exception.
The mmdb_open_location_db() and mmdb_open_asn_db() BiFs were untested, and Zeek
has a bug that makes any DBs opened that way fall back to looking up DBs via the
existing script-level config mechanism (via mmdb_dir), which is at least
unexpected and might well be unconfigured if somebody uses the direct BiFs.
The test would previously fail in settings where the user has Maxmind DBs
installed in the hardwired system locations, because the fallback logic still
picked those up.
This lifts the list of fallback directories in which Zeek will look for Maxmind
DBs into the script layer, and makes the names of the DB files themselves
(previously hardwired) configurable as well.
This does not yet change the in-core code; that commit follows.
* topic/timw/move-bifs:
Use std::move in return values from bif methods to avoid copies
Use bool return values instead of int in a couple zeek.bif static methods
After switching the known scripts away from broker stores, the
&create_expire value of the local tables/sets of the known scripts
wasn't in effect due to Cluster::node_up() and Cluster::node_down()
re-assigning these without keeping the &create_expire attribute
intact. This broke the "log hosts every 24h" behavior.
Closes#3540
* topic/timw/werror:
CI: Remove unused openssl30_config
CPP-gen: Don't emit extra braces if only one element
Use <poll.h> instead of <sys/poll.h>
ZAM: Create ListValPtr directly instead of a stack object
Bump zeekjs to pick up dprintf warning fix
Avoid unused-result warning in Supervisor
Update src/3rdparty submodule to fix sprintf warning in modp
Fix warning with attribute string lookup
Set -Werror / /WX via target_compile_options to force warnings as errors
