Commit graph

1120 commits

Author SHA1 Message Date
Robin Sommer
d99f041ac5
Add WebSocket support for exchanging events with external clients.
This exposes Broker's new WebSocket support in Zeek. To enable it,
call `Broker::listen_websocket()`. Zeek will then start listening on
port 9997 for incoming WebSocket connections.

See the Broker documentation for a description of the message format
expected over these WebSocket connections.
2022-06-02 10:31:52 +02:00
Christian Kreibich
93bed5a261 Merge branch 'topic/christian/node-status-notification'
* topic/christian/node-status-notification:
  Add Supervisor::node_status notification event
2022-05-31 12:53:18 -07:00
Christian Kreibich
14188fc7a7 Add Supervisor::node_status notification event
The Supervisor generates this event every time it receives a status update from
the stem, meaning a node got created or re-created. A corresponding
SupervisorControl::node_status event relays the same information for users
interacting with the Supervisor over Broker.
2022-05-30 21:36:35 -07:00
Vern Paxson
07cf5cb089 deprecation messages for unused base script functions 2022-05-27 14:36:30 -07:00
Vern Paxson
6dc711c39e annotate orphan base script components with &deprecated 2022-05-26 17:39:17 -07:00
Vern Paxson
9b8ac44169 annotate base scripts with &is_used as needed 2022-05-26 17:39:17 -07:00
Tim Wojtulewicz
f8bc23d3e1 Propagate BPF_Program error message to script land 2022-05-25 09:41:35 -07:00
Tim Wojtulewicz
8b0263cb39 Merge remote-tracking branch 'origin/topic/vern/script-profiling'
* origin/topic/vern/script-profiling:
  tidy up after generating profile
  test suite updates for refined script coverage, use of new BiF to speed startup
  fix for coverage reporting for functions that use "when" statements
  new global_options() BiF to speed up startup, plus a micro-preen
  hooks for new --profile-scripts option
  classes for managing script profiles
  address some holes in script coverage
  fix for script coverage missing on-exit activity
  memory management fixes for loggers
  make curr_CPU_time() broadly available rather than just isolated to ZAM
2022-05-11 12:56:41 -07:00
Johanna Amann
3a7b127a71 Merge remote-tracking branch 'origin/topic/johanna/topk-cite'
* origin/topic/johanna/topk-cite:
  Add exact name of the Top-k algorithm.
2022-05-11 18:45:48 +01:00
Johanna Amann
25c33d2a29 Add exact name of the Top-k algorithm.
I needed to figure out which exact algorithm we use for our
probabilistic top-k measurements. It turns out that we do not mention
this in our source tree at all so far.
2022-05-11 13:21:26 +01:00
Vern Paxson
fab4905fc2 new global_options() BiF to speed up startup, plus a micro-preen 2022-05-03 11:13:15 -07:00
Vern Paxson
f62c04c9eb tweaks to base scripts revealed by switch to expression-based initialization 2022-03-23 15:36:35 -07:00
Christian Kreibich
6a46d1584c Add capture to a Sumstats when-statement
This resolves a deprecation warning that currently triggers when running
Sumstats in clusterized Zeek.
2022-02-04 12:54:34 -08:00
Vern Paxson
98cd3f2213 update uses of "when" in base scripts to include captures 2022-01-07 14:53:33 -08:00
Vern Paxson
72a59bf828 removed unused script variable 2021-12-14 12:49:27 -08:00
Tim Wojtulewicz
8816547964 Fix types for Analyzer::register_for_port(s) to be the same 2021-12-10 17:48:19 +00:00
Christian Kreibich
1aaed1cc2e Add LogAscii::json_include_unset_fields flag to control unset field rendering
The flag controls whether JSON rendering includes unset &optional log fields
(F, the default), or includes them with a null value (T).
2021-12-08 17:29:07 -08:00
Tim Wojtulewicz
e82a78616b Update NEWS and some minor fixes for docs/zeekygen 2021-11-23 19:39:36 -07:00
Tim Wojtulewicz
2044fbe53b Add GTPv1 packet analyzer, disable old analyzer 2021-11-23 19:36:50 -07:00
Tim Wojtulewicz
dc0ecf9811 Add Teredo packet analyzer, disable old analyzer 2021-11-23 19:36:50 -07:00
Tim Wojtulewicz
05574ecce1 Add VXLAN packet analyzer, disable old analyzer 2021-11-23 19:36:50 -07:00
Tim Wojtulewicz
cbb0bcd49c Add Geneve packet analyzer, disable old analyzer 2021-11-23 19:36:50 -07:00
Tim Wojtulewicz
7e40094f2c Add AYIYA packet analyzer, disable old analyzer 2021-11-23 19:36:50 -07:00
Tim Wojtulewicz
44e0760e96 Add PacketAnalyzer::register_for_port(s) functions
These allow packet analyzers to register ports as identifiers to forward from
parent analyzers, while also adding those ports to the now-global
Analyzer::ports table at the same time.
2021-11-23 19:36:50 -07:00
Tim Wojtulewicz
612212568a Add analyzer_confirmation and analyzer_violation events 2021-11-23 19:36:50 -07:00
Tim Wojtulewicz
a7d3cb48ef Add concept of "parent" tag namespaces
This allows us to create an EnumType that groups all of the analyzer
tag values into a single type, while still having the existing types
that split them up. We can then use this for certain events that benefit
from taking all of the tag types at once.
2021-11-23 19:36:49 -07:00
Tim Wojtulewicz
a6378531db Remove trailing whitespace from script files 2021-10-20 09:57:09 -07:00
Tim Wojtulewicz
0a0ed65306 Merge remote-tracking branch 'origin/topic/robin/gh-54-sanitize'
* origin/topic/robin/gh-54-sanitize:
  Sanitize log files names before they go into system().
2021-09-22 12:17:05 -07:00
Robin Sommer
74680bf4e6 Merge remote-tracking branch 'origin/topic/justin/software-framework-parse-cache'
* origin/topic/justin/software-framework-parse-cache:
  Restore behavior of Software::register event
  Optimzie software framework version parsing
2021-09-21 18:00:46 +02:00
Justin Azoff
ef5fb790ef Restore behavior of Software::register event
Use an intermediary event to ensure that software versions are parsed
before calling Software::register.
2021-09-20 14:38:47 -04:00
Seth Hall
a4ceb98bf8 Switch the TSV Zeek logs to be UTF8 by default.
There is a paired zeek-testing branch for some updates there.
2021-09-07 09:16:53 -07:00
Justin Azoff
3bf8c8ceb6 Optimzie software framework version parsing
Add a small cache in front of the parse method.  This cache should
reduce most of the calls to parse, and ultimately save memory because
redundant versions of the parsed strings will not be created in memory.

Move the parsing itself to the proxies where the caching can be more
efficient.
2021-09-05 13:18:51 -04:00
Christian Kreibich
90b7c6961e Simplify the supervisor's listen() on default address/port 2021-08-18 12:35:49 -07:00
Christian Kreibich
281eef398f Merge branch 'topic/vlad/doc_tweaks'
* topic/vlad/doc_tweaks:
  docs: escape * in documentation
  Update the mail_dest documentation with the new email_dest field
2021-07-09 12:02:32 -07:00
Christian Kreibich
8db985ea78 Merge branch 'topic/christian/cluster-controller'
* topic/christian/cluster-controller:
  Add a cluster controller testcase for agent-controller checkin
  Add zeek-client via new submodule
  Update baselines affected by cluster controller changes
  Introduce cluster controller and cluster agent scripting
  Establish a separate init script when using the supervisor
  Add optional bare-mode boolean flag to Supervisor's node configuration
  Add support for making the supervisor listen for requests
  Add support for setting environment variables via supervisor
2021-07-08 16:51:11 -07:00
Christian Kreibich
6eddaed62d Fix reference in the logging framework docs re. Log::StreamPolicyHook 2021-07-08 15:39:46 -07:00
Vlad Grigorescu
10159a3fdf Update the mail_dest documentation with the new email_dest field 2021-07-08 16:01:21 -05:00
Christian Kreibich
7bee79b400 Add optional bare-mode boolean flag to Supervisor's node configuration
When omitted, the node inherits the Supervisor's bare-mode
status. When true/false, the new Zeek node will enable/disable bare
mode, respectively. It continues to load any scripts passed at the
command line and in the additional scripts list already provided in
the node configuration.

Includes testcase.
2021-07-08 13:12:53 -07:00
Christian Kreibich
efaa9ec3be Add support for making the supervisor listen for requests
The supervisor now starts listening on the configured Broker default address and
port when the new boolean SupervisorControl::enable_listen is T. Listening
remains disabled by default. Listening allows nodes to communicate with the
supervisor via the events laid out in control.zeek, to conduct further node
management.
2021-07-08 13:12:53 -07:00
Christian Kreibich
36051dc9a1 Add support for setting environment variables via supervisor
The NodeConfig record now has a table for specifying environment variable names
and values, which the supervisor sets in the created node.

This also repositions the cpu_affinity member to keep the order the same in
the corresponding script-layer and in-core types.

Includes testcase.
2021-07-08 13:12:53 -07:00
Christian Kreibich
795a7ea98e Add a global log policy hook to the logging framework
This addresses the need for a central hook on any log write, which
wasn't previously doable without a lot of effort. The log manager
invokes the new Log::log_stream_policy hook prior to any filter-specific
hooks. Like filter-level hooks, it may veto a log write. Even when
it does, filter-level hooks still get invoked, but cannot "un-veto".

Includes test cases.
2021-07-02 12:42:45 -07:00
Tim Wojtulewicz
30973af94a Merge remote-tracking branch 'origin/topic/johanna/remove-stepping-stone'
* origin/topic/johanna/remove-stepping-stone:
  Remove the Stepping Stone analyzer
2021-06-21 16:14:25 -07:00
Vlad Grigorescu
9d1e9a20e4 Expand the tests, and have email_admin email src contacts in one e-mail, and dst contacts in another. 2021-06-17 14:49:24 -05:00
Vlad Grigorescu
9d0bd96e1f Simplify the logic a big by making Notice$email_dest have a default rather than be optional 2021-06-17 10:31:04 -05:00
Vlad Grigorescu
e579497247 Merge branch 'master' into topic/vlad/page_hostnames 2021-06-11 10:18:15 -05:00
Johanna Amann
e0d284ec9f Merge branch 'logging/script-logdir' of https://github.com/kramse/zeek
* 'logging/script-logdir' of https://github.com/kramse/zeek:
  Copy of ascii-empty test, just changed path in the beginning
  Logdir: Change requested by 0xxon, no problem
  Introduce script-land variable that can be used to set logdir.

Closes GH-772
2021-06-10 12:19:15 +01:00
Henrik Kramselund Jereminsen
74561e922f Logdir: Change requested by 0xxon, no problem
Suggestion from 0xxon to look at Input Framework inspired this change
2021-06-10 10:33:26 +01:00
Henrik Kramselund Jereminsen
6bde33aca7 Introduce script-land variable that can be used to set logdir.
Addresses GH-772
2021-06-10 10:32:11 +01:00
Johanna Amann
011ac442a1 Remove the Stepping Stone analyzer
This commit removes the stepping stone analyzer. It has been deactivated
by default since at least Zeek 2.0, is dysfunctional in cluster settings
and has a bunch of other issued.

Relates to GH-1573
2021-06-03 14:28:12 +01:00
Dominik Charousset
7767c3d36c Sync new broker options, fix name inconsistencies 2021-05-25 17:22:45 +02:00