With this, the basic functionality of &backend seems to be working.
It is not yet integrated with zeekctl, one has to manually specify the
storage location for the sqlite files somewhere when using sqlite.
Usage for memory stores:
global table_to_share: table[string] of count &backend=Broker::MEMORY;
Usage for sqlite stores:
redef Broker::auto_store_db_directory = "[path]";
global table_to_share: table[string] of count &backend=Broker::SQLITE;
In both cases, the cluster should automatically sync to changes done by
any node. When using sqlite, data should also be saved to disk and
re-loaded on startup.
Currently this requires using this with a normal cluster - or sending
messages by yourself.
It, in principle, should also work with SQLITE - but that is a bit
nonsensical without being able to change the storage location.
Merge adjustments:
- Added back in deprecation tag for base_type_no_ref()
- Added back the deprecated plugin::hook_name() function
* origin/topic/timw/1032-namespaced-enums:
Deprecate plugin::HookType and plugin::component::Type in a different way
Deprecate init_class and IDScope in another way.
Deprecate TypeTag and friends in a different way
Deprecate attr_tag in a different way, rename to AttrTag
The &backend attribute allows for a much more convenient way of
interacting with brokerstores. One does not need to create a broker
store anymore - instead all of this is done internally.
The current state of this partially works. This should work fine for
persistence - but clones are currently not yet correctly attached.
Minor tweaks during merge:
- Remove/default superfluous ~Attributes()
- Added in resize() calls to accompany reserve() of associated vectors
* origin/topic/timw/1034-IndexTypes:
Revert Attributes::Attrs back to return an attr_list and mark it deprecated
GH-1034: Revert TypeList::Types() back to return a type_list* and mark it deprecated
When a clone attaches to a master, it just gets the diffs sent as
events. Which is neat because it means that we pretty much don't need
any extra code to handle this.
This currently only handles the most basic case, and is not thoroughly
tested.
When initializing a master store, we now check if there already is data
in it. If yes, we load it directly into the zeek table when the store is
created. We assume that this is happening at Zeek startup - and are
supremely evil and just load it synchronously. Which could block
execution for a bit for larger stores.
That being said - this might sidestep other issues that would arise when
doing this async (like scripts already inserting data).
Next step: check if this approach also works for clones.
* origin/master:
Fix shadowed variable that breaks lookup_hostname()
GH-1025: allow copying/cloning of `opaque of Broker::Store`
Fix "possibly-truncated" compiler warning in BuildJSON snprintf()
Update submodule(s)
Fixed some places where tabs became spaces
Convert to using permissions to check for access to cirrus variables in benchmark script
Integrate review feedback: improve command-line option redef parsing
Fix several issues with command-line option redefs
Remove last_access_time from TableEntryVal.
Minimize data published for expected FTP data channel analysis
Stricter checking if we have a dns field on the connection being processed
Modified the DNS protocol analyzer to add a new parameter to the dns_request event which includes the DNS query in its original case. Added a policy script that will add the original_case to the dns.log file as well. Created new btests to test both.
Place build file in explicit location for benchmarking to work correctly
cmake: Make musl support more distro agnostic
Update highwayhash submodule to upstream.
GH-998: Fix Reporter::conn_weird() to handle expired connections
Changes during merge
- Changed the policy script to use an event handler that behaves
for like the base script: &priority=5, msg$opcode != early-out,
no record field existence checks
- Also extended dns_query_reply event with original_query param
- Removed ExtractName overload, and just use default param
* 'dns-original-query-case' of https://github.com/rvictory/zeek:
Fixed some places where tabs became spaces
Stricter checking if we have a dns field on the connection being processed
Modified the DNS protocol analyzer to add a new parameter to the dns_request event which includes the DNS query in its original case. Added a policy script that will add the original_case to the dns.log file as well. Created new btests to test both.
Makes some attributes conflict with each other. This also needed the
test to change.
The test is a bit flaky - but I can, for the heck of it, not figure out
why. I am punting that for the future after spending a few hours on it.
