Mirror/zeek
1
0
Fork 0
mirror of https://github.com/zeek/zeek.git synced 2025-10-02 14:48:21 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 2

intel: Add indicator_inserted and indicator_removed hooks

Browse source 
This change adds two new hooks to the Intel framework that can be used
to intercept added and removed indicators and their type.

These hooks are fairly low-level. One immediate use-case is to count the
number of indicators loaded per Intel::Type and enable and disable the
corresponding event groups of the intel/seen scripts.

I attempted to gauge the overhead and while it's definitely there, loading
a file with ~500k DOMAIN entries takes somewhere around ~0.5 seconds hooks
when populated via the min_data_store store mechanism. While that
doesn't sound great, it actually takes the manager on my system 2.5
seconds to serialize and Cluster::publish() the min_data_store alone
and its doing that serially for every active worker. Mostly to say that
the bigger overhead in that area on the manager doing redundant work
per worker.

Co-authored-by: Mohan Dhawan <mohan@corelight.com>
This commit is contained in:
Arne Welzel 2025-05-14 12:49:27 +02:00
parent 224519c11a
commit 7eb849ddf4
19 changed files with 611 additions and 1 deletions
Download patch file Download diff file Expand all files Collapse all files

2
testing/btest/Baseline/scripts.base.frameworks.intel.cluster-indicator-inserted-new-min-store/manager..stderr Normal file
View file

@ -0,0 +1,2 @@
### BTest baseline data generated by btest-diff. Do not edit. Use "btest -U/-u" to update. Requires BTest >= 0.63.
received termination signal

6
testing/btest/Baseline/scripts.base.frameworks.intel.cluster-indicator-inserted-new-min-store/manager..stdout Normal file
View file

@ -0,0 +1,6 @@
### BTest baseline data generated by btest-diff. Do not edit. Use "btest -U/-u" to update. Requires BTest >= 0.63.
Intel::indicator_inserted, 1.2.3.4, Intel::ADDR
Intel::indicator_inserted, 1.2.3.5, Intel::ADDR
Intel::indicator_inserted, 192.168.0.0/16, Intel::SUBNET
Intel::indicator_inserted, putty, Intel::SOFTWARE
Intel::indicator_inserted, putty2, Intel::SOFTWARE

13
testing/btest/Baseline/scripts.base.frameworks.intel.cluster-indicator-inserted-new-min-store/worker-1-run-1..stdout Normal file
View file

@ -0,0 +1,13 @@
### BTest baseline data generated by btest-diff. Do not edit. Use "btest -U/-u" to update. Requires BTest >= 0.63.
Intel::new_min_data_store pre, [host_data={\x0a\x0a}, subnet_data={\x0a\x0a}, string_data={\x0a\x0a}]
Intel::new_min_data_store post, [host_data={\x0a\x0a}, subnet_data={\x0a\x0a}, string_data={\x0a\x0a}]
Intel::insert_indicator, 1.2.3.4, Intel::ADDR
Intel::indicator_inserted, 1.2.3.4, Intel::ADDR
Intel::insert_indicator, 1.2.3.5, Intel::ADDR
Intel::indicator_inserted, 1.2.3.5, Intel::ADDR
Intel::insert_indicator, 192.168.0.0/16, Intel::SUBNET
Intel::indicator_inserted, 192.168.0.0/16, Intel::SUBNET
Intel::insert_indicator, putty, Intel::SOFTWARE
Intel::indicator_inserted, putty, Intel::SOFTWARE
Intel::insert_indicator, putty2, Intel::SOFTWARE
Intel::indicator_inserted, putty2, Intel::SOFTWARE

8
testing/btest/Baseline/scripts.base.frameworks.intel.cluster-indicator-inserted-new-min-store/worker-1-run-2..stdout Normal file
View file

@ -0,0 +1,8 @@
### BTest baseline data generated by btest-diff. Do not edit. Use "btest -U/-u" to update. Requires BTest >= 0.63.
Intel::new_min_data_store pre, [host_data={\x0a\x0a}, subnet_data={\x0a\x0a}, string_data={\x0a\x0a}]
Intel::indicator_inserted, 1.2.3.5, Intel::ADDR
Intel::indicator_inserted, 1.2.3.4, Intel::ADDR
Intel::indicator_inserted, 192.168.0.0/16, Intel::SUBNET
Intel::indicator_inserted, putty, Intel::SOFTWARE
Intel::indicator_inserted, putty2, Intel::SOFTWARE
Intel::new_min_data_store post, [host_data={\x0a\x091.2.3.5,\x0a\x091.2.3.4\x0a}, subnet_data={\x0a\x09192.168.0.0/16\x0a}, string_data={\x0a\x09[putty, Intel::SOFTWARE] ,\x0a\x09[putty2, Intel::SOFTWARE] \x0a}]

13
testing/btest/Baseline/scripts.base.frameworks.intel.cluster-indicator-inserted-new-min-store/worker-2..stdout Normal file
View file

@ -0,0 +1,13 @@
### BTest baseline data generated by btest-diff. Do not edit. Use "btest -U/-u" to update. Requires BTest >= 0.63.
Intel::new_min_data_store pre, [host_data={\x0a\x0a}, subnet_data={\x0a\x0a}, string_data={\x0a\x0a}]
Intel::new_min_data_store post, [host_data={\x0a\x0a}, subnet_data={\x0a\x0a}, string_data={\x0a\x0a}]
Intel::insert_indicator, 1.2.3.4, Intel::ADDR
Intel::indicator_inserted, 1.2.3.4, Intel::ADDR
Intel::insert_indicator, 1.2.3.5, Intel::ADDR
Intel::indicator_inserted, 1.2.3.5, Intel::ADDR
Intel::insert_indicator, 192.168.0.0/16, Intel::SUBNET
Intel::indicator_inserted, 192.168.0.0/16, Intel::SUBNET
Intel::insert_indicator, putty, Intel::SOFTWARE
Intel::indicator_inserted, putty, Intel::SOFTWARE
Intel::insert_indicator, putty2, Intel::SOFTWARE
Intel::indicator_inserted, putty2, Intel::SOFTWARE