Commit graph

16055 commits

Author SHA1 Message Date
Tim Wojtulewicz
8d97d5f56a Merge remote-tracking branch 'origin/topic/vern/zam-tests-and-NEWS.Jun22'
* origin/topic/vern/zam-tests-and-NEWS.Jun22:
  NEWS updates for 5.0
  NEWS updates for 5.0
  ZAM test suite updates due to recent changes
2022-06-06 09:36:47 -07:00
Tim Wojtulewicz
733b8c4e0f Updating CHANGES and VERSION. 2022-06-06 09:36:16 -07:00
Craig Leres
eb772d0d52 Tweak some new comments 2022-06-05 13:56:28 -07:00
Craig Leres
53ab44c098 Remove child program check, it's probably wrong given the test
failures it causes
2022-06-04 10:20:38 -07:00
Craig Leres
ca8a8d5c8e
Merge branch 'zeek:master' into update 2022-06-03 16:35:36 -08:00
naveen
8c2e23ecf6 chore: Set permissions for GitHub actions
Restrict the GitHub token permissions only to the required ones; this way, even if the attackers will succeed in compromising your workflow, they won’t be able to do much.

- Included permissions for the action. https://github.com/ossf/scorecard/blob/main/docs/checks.md#token-permissions

https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#permissions

https://docs.github.com/en/actions/using-jobs/assigning-permissions-to-jobs

[Keeping your GitHub Actions and workflows secure Part 1: Preventing pwn requests](https://securitylab.github.com/research/github-actions-preventing-pwn-requests/)

Signed-off-by: naveen <172697+naveensrinivasan@users.noreply.github.com>
2022-06-04 00:31:28 +00:00
Tim Wojtulewicz
d2cecbecb2 Keep make dist from deleting all paths containing 'build' [skip ci] 2022-06-03 22:36:19 +00:00
Vern Paxson
24bc6c1f51 Merge branch 'topic/vern/zam-tests-and-NEWS.Jun22' of github.com:zeek/zeek into topic/vern/zam-tests-and-NEWS.Jun22 2022-06-03 15:18:14 -07:00
Vern Paxson
786d06ff7e NEWS updates for 5.0 2022-06-03 15:17:58 -07:00
Vern Paxson
f3dc740036 NEWS updates for 5.0 2022-06-03 15:15:33 -07:00
Vern Paxson
1724ddfebf ZAM test suite updates due to recent changes 2022-06-03 14:58:22 -07:00
Tim Wojtulewicz
d8df7e9b42 Dockerfile changes for consistency with Broker files 2022-06-03 14:57:21 -07:00
Tim Wojtulewicz
2977af93b5 Make some of the CI Dockerfiles more consistent 2022-06-03 14:28:48 -07:00
Tim Wojtulewicz
48a858f5db Update local-compat test for 5.1 2022-06-03 12:38:20 -07:00
Tim Wojtulewicz
811caec77a Start of 5.1.0 development 2022-06-03 12:36:24 -07:00
Tim Wojtulewicz
4a569b6c68 Update submodules [nomail] 2022-06-03 12:09:34 -07:00
Tim Wojtulewicz
0952340447 Updating submodule(s) [nomail] 2022-06-03 11:22:05 -07:00
Christian Kreibich
a9c10e20a2 Merge branch 'topic/christian/management-update-refs'
* topic/christian/management-update-refs:
  Management framework: bump external cluster testsuite
  Management framework: bump zeek-client to bring back to master
2022-06-03 10:56:05 -07:00
Christian Kreibich
ec4015e28c Management framework: bump external cluster testsuite 2022-06-03 10:38:31 -07:00
Christian Kreibich
43f9c27b75 Management framework: bump zeek-client to bring back to master 2022-06-03 10:37:54 -07:00
Tim Wojtulewicz
9964a46402 Fix hang in DNS analyzer with malformed packet
(cherry picked from commit dfc9f77b68f6d02d059ade322b9c132e32c26872)
2022-06-03 10:32:03 -07:00
Tim Wojtulewicz
41aa8b2349 Merge remote-tracking branch 'origin/topic/christian/is_used_in_netcontrol_sumstats'
* origin/topic/christian/is_used_in_netcontrol_sumstats:
  Additional &is_used tags in the Netcontrol and Sumstats frameworks
2022-06-03 09:50:54 -07:00
Tim Wojtulewicz
febdc97f09 Merge remote-tracking branch 'origin/topic/christian/management-instance-handling'
* origin/topic/christian/management-instance-handling:
  Management framework: bump zeek-client to pull in rendering tweaks
  Management framework: bump external cluster testsuite
  Management framework: improve address and port handling
  Management framework: broaden get_instances response data to connected instances
  Management framework: expand notify_agent_hello event arguments
  Management framework: comment-only tweaks and typo fixes
2022-06-03 09:50:21 -07:00
Christian Kreibich
47f4342821 Management framework: bump zeek-client to pull in rendering tweaks 2022-06-03 02:18:35 -07:00
Christian Kreibich
08d1f93292 Management framework: bump external cluster testsuite 2022-06-03 02:18:35 -07:00
Christian Kreibich
c53044981a Management framework: improve address and port handling
The get-nodes command also benefits from showing the state on connected agents
more broadly (as opposed to just the one for the current configuration).

Also a bugfix: ensure we use an agent's IP address as seen by the
controller. This avoids reporting "0.0.0.0" in some cases.
2022-06-03 02:14:07 -07:00
Christian Kreibich
0c47d45bb9 Management framework: broaden get_instances response data to connected instances
This response so far contained only the connected instances that are relevant to
the current configuration, but this isn't very helpful when troubleshooting
instance connectivity. It now reports all currently connected instances, with
network addresses & ports as known to Broker.
2022-06-03 02:13:30 -07:00
Christian Kreibich
72acf24f52 Management framework: expand notify_agent_hello event arguments
This swaps the host event argument for the Broker ID. The latter is more useful,
since the sending agent doesn't necessarily know its IP address as visible to
the controller, and the controller can pull up the full Broker context via the
ID.

It also adds an explicit argument to the event to indicate whether the agent
connected to the controller or vice versa. This simplifies the controller's
internal logic.

Also minor tweaks to logging to show Broker IDs.
2022-06-03 02:12:19 -07:00
Christian Kreibich
aa689807fa Management framework: comment-only tweaks and typo fixes 2022-06-03 02:12:12 -07:00
Christian Kreibich
edef3736fb Additional &is_used tags in the Netcontrol and Sumstats frameworks
When running a cluster, these functions only get called in select node types and
could trigger no-caller warnings on stderr.
2022-06-02 22:57:07 -07:00
Tim Wojtulewicz
3ce1db1b5f Fix doc issue in zeek.bif, update docs submodule 2022-06-02 17:54:17 -07:00
Tim Wojtulewicz
facdf0b0d3 Merge remote-tracking branch 'origin/topic/timw/update-submodules'
* origin/topic/timw/update-submodules:
  Updating submodule(s) [nomail]
2022-06-02 17:01:34 -07:00
Tim Wojtulewicz
535a6013aa Merge remote-tracking branch 'zeek-as-org/as-org'
* zeek-as-org/as-org:
  Mark lookup_asn() BIF as deprecated in v6.1
  Define geo_autonomous_system record type
  Add lookup_autonomous_system() BIF that returns AS number and org
2022-06-02 16:59:29 -07:00
Phil Rzewski
320fbf49fd Mark lookup_asn() BIF as deprecated in v6.1 2022-06-02 15:27:39 -07:00
Tim Wojtulewicz
8b14539c5a Add NEWS entries for broker changes for 5.0.0 2022-06-02 13:27:32 -07:00
Tim Wojtulewicz
6196a959f1 Reformat NEWS file for proper line lengths in the 5.0.0 notes 2022-06-02 13:25:26 -07:00
Christian Kreibich
bef0597ac9 Merge branch 'topic/christian/gh-2152-check-for-websockets-module'
* topic/christian/gh-2152-check-for-websockets-module:
  Make broker.web-socket-events test require the Python websockets package
2022-06-02 13:16:56 -07:00
Christian Kreibich
26692c1249 Make broker.web-socket-events test require the Python websockets package 2022-06-02 13:11:06 -07:00
Christian Kreibich
1cebdd569d Merge branch 'topic/christian/gh-2134-fix-intel-test-races'
* topic/christian/gh-2134-fix-intel-test-races:
  Expand scripts.base.frameworks.intel.cluster-transparency test
  Fix races in scripts.base.frameworks.intel.cluster-transparency-with-proxy test
  Add Intel::send_store_on_node_up boolean to control min_data_store delivery
2022-06-02 12:20:06 -07:00
Christian Kreibich
74c21618bf Merge branch 'topic/christian/gcc12-memset-warning-fix'
* topic/christian/gcc12-memset-warning-fix:
  Fix a -Wstringop-overflow memset size warning with GCC 12
2022-06-02 12:18:46 -07:00
Tim Wojtulewicz
d3cba26974 Updating submodule(s) [nomail] 2022-06-02 11:49:55 -07:00
Robin Sommer
24c2090ffb
Merge remote-tracking branch 'origin/topic/robin/websocket'
* origin/topic/robin/websocket:
  Add WebSocket support for exchanging events with external clients.
2022-06-02 12:08:31 +02:00
Robin Sommer
d99f041ac5
Add WebSocket support for exchanging events with external clients.
This exposes Broker's new WebSocket support in Zeek. To enable it,
call `Broker::listen_websocket()`. Zeek will then start listening on
port 9997 for incoming WebSocket connections.

See the Broker documentation for a description of the message format
expected over these WebSocket connections.
2022-06-02 10:31:52 +02:00
Christian Kreibich
984d6c0136 Expand scripts.base.frameworks.intel.cluster-transparency test
This simply expands this test to match the behavior of
cluster-transparency-with-proxy, since the two are so similar. This test does
not seem to need disabling the worker's initial send of the data store.
2022-06-01 22:26:25 -07:00
Christian Kreibich
ed5d60f758 Fix races in scripts.base.frameworks.intel.cluster-transparency-with-proxy test
This test was unstable for two reasons:

- Nothing verified whether the two workers had checked in with the proxy,
meaning that messages between the workers and proxies could get lost. This adds
an extra node_up event that the proxy generates synthetically, with values
recognizable to the manager, once the proxy sees both workers connected. This is
a test-level workaround for what should really be a cluster-is-ready event in
the cluster framework proper.

- More subtle: the Intel framework makes the manager send its current
min_data_store to newly connected workers, which in the case of this tests
introduces a race: since the data store, arriving at the worker, replaces the
existing value, it could actually remove already established items if timing was
right. This would lead to the count in the test reaching 3, assuming that 3
intel items are available, when in reality it was less, causing the
Intel::seen() call to do nothing. We now disable the sending of the data store
upon connect, via the global added in the previous commit.

This also expands the test slightly so that both workers call Intel::seen() for
the items inserted by the other worker. This is added validation for the second
point above, because in the presence of that race one occasionally sees one log
entry make it, and the other fail.
2022-06-01 22:23:07 -07:00
Phil Rzewski
c08fe7c237 Define geo_autonomous_system record type 2022-06-01 18:39:07 -07:00
Christian Kreibich
80f6b1ee5e Fix a -Wstringop-overflow memset size warning with GCC 12 2022-06-01 18:30:24 -07:00
Christian Kreibich
892a3a8452 Add Intel::send_store_on_node_up boolean to control min_data_store delivery
This adds a redefinable const to the internals of the Intel framework, to allow
suppression of the manager sending its current min_data_store when a worker
connects. This feature is desirable for nodes that check in "late" to bring them
up to speed, but during testing it introduces nondeterminism.
2022-06-01 17:45:19 -07:00
Phil Rzewski
7fe5a9cfa2 Add lookup_autonomous_system() BIF that returns AS number and org 2022-06-01 17:44:26 -07:00
Tim Wojtulewicz
4b0e1063ed Merge remote-tracking branch 'origin/topic/timw/update-submodules'
* origin/topic/timw/update-submodules:
  Update auxil/package-manager submodule [nomail]
  Update submodule(s) [nomail]
2022-06-01 11:10:15 -07:00