Fix races in scripts.base.frameworks.intel.cluster-transparency-with-proxy test

This test was unstable for two reasons:

- Nothing verified whether the two workers had checked in with the proxy,
meaning that messages between the workers and proxies could get lost. This adds
an extra node_up event that the proxy generates synthetically, with values
recognizable to the manager, once the proxy sees both workers connected. This is
a test-level workaround for what should really be a cluster-is-ready event in
the cluster framework proper.

- More subtle: the Intel framework makes the manager send its current
min_data_store to newly connected workers, which in the case of this tests
introduces a race: since the data store, arriving at the worker, replaces the
existing value, it could actually remove already established items if timing was
right. This would lead to the count in the test reaching 3, assuming that 3
intel items are available, when in reality it was less, causing the
Intel::seen() call to do nothing. We now disable the sending of the data store
upon connect, via the global added in the previous commit.

This also expands the test slightly so that both workers call Intel::seen() for
the items inserted by the other worker. This is added validation for the second
point above, because in the presence of that race one occasionally sees one log
entry make it, and the other fail.
This commit is contained in:
Christian Kreibich 2022-06-01 22:18:51 -07:00
parent 892a3a8452
commit ed5d60f758
4 changed files with 58 additions and 19 deletions

View file

@ -1,4 +1,5 @@
### BTest baseline data generated by btest-diff. Do not edit. Use "btest -U/-u" to update. Requires BTest >= 0.63.
### NOTE: This file has been sorted with diff-sort.
#separator \x09
#set_separator ,
#empty_field (empty)
@ -7,5 +8,6 @@
#open XXXX-XX-XX-XX-XX-XX
#fields ts uid id.orig_h id.orig_p id.resp_h id.resp_p seen.indicator seen.indicator_type seen.where seen.node matched sources fuid file_mime_type file_desc
#types time string addr port addr port string enum enum string set[enum] set[string] string string string
XXXXXXXXXX.XXXXXX - - - - - 123.123.123.123 Intel::ADDR Intel::IN_ANYWHERE worker-2 Intel::ADDR worker-1 - - -
#close XXXX-XX-XX-XX-XX-XX
XXXXXXXXXX.XXXXXX - - - - - 123.123.123.123 Intel::ADDR Intel::IN_ANYWHERE worker-2 Intel::ADDR worker-1 - - -
XXXXXXXXXX.XXXXXX - - - - - 4.3.2.1 Intel::ADDR Intel::IN_ANYWHERE worker-1 Intel::ADDR worker-2 - - -

View file

@ -4,3 +4,4 @@ new_indicator: 1.2.3.4 inserted by manager
new_indicator: 123.123.123.123 inserted by worker-1
new_indicator: 4.3.2.1 inserted by worker-2
new_item triggered for 123.123.123.123 by worker-1 on worker-1
seeing 4.3.2.1

View file

@ -1,7 +1,7 @@
### BTest baseline data generated by btest-diff. Do not edit. Use "btest -U/-u" to update. Requires BTest >= 0.63.
### NOTE: This file has been sorted with diff-sort.
Doing a lookup
new_indicator: 1.2.3.4 inserted by manager
new_indicator: 123.123.123.123 inserted by worker-1
new_indicator: 4.3.2.1 inserted by worker-2
new_item triggered for 4.3.2.1 by worker-2 on worker-2
seeing 123.123.123.123