Commit graph

810 commits

Author SHA1 Message Date
Tim Wojtulewicz
46ff48c29a Change all instruments to only handle doubles 2024-05-31 13:36:37 -07:00
Tim Wojtulewicz
635198793d Fix header comments in scripts/policy/frameworks/telemetry/prometheus.zeek 2024-05-31 13:36:37 -07:00
Tim Wojtulewicz
017ee4509c Update telemetry log policy due to the fact that unit will not be filled in anymore 2024-05-31 13:30:31 -07:00
Tim Wojtulewicz
84aa308527 Rework everything to access the prometheus-cpp objects more directly 2024-05-31 13:30:31 -07:00
Tim Wojtulewicz
e93e4cc26d Add a services.json endpoint for Prometheus service discovery 2024-05-31 13:30:31 -07:00
Tim Wojtulewicz
4718e5cf00 Remove everything related to aggregation 2024-05-31 13:30:31 -07:00
Tim Wojtulewicz
97a35011a7 Add necessary script-land changes 2024-05-31 13:30:31 -07:00
Tim Wojtulewicz
6821a41c4e Move the options from policy/tuning/defaults to actual Zeek defaults, deprecate that package 2024-05-06 11:13:04 -07:00
Arne Welzel
f4ed1e05fc signatures/iso-9660: Add \x01 suffix to CD001
As discussed with Tim, that should make it a bit more robust against
false positives.
2024-02-26 21:00:01 +01:00
Arne Welzel
1a5ce65e3d signatures: Move ISO 9660 signature to policy
The previous "fix" caused significant performance degradation without
the signature ever having a chance to trigger. Moving it to policy
seems the best compromise, the alternative being outright removing it.
2024-02-26 13:35:23 +01:00
Johanna Amann
737f828bcb Netcontrol: add rule_added_policy
rule_added_policy allows the modification of rules just after they have
been added. This allows the implementation of some more complex features
- like changing rule states depending on insertion in other plugins.
2024-02-05 18:52:27 +00:00
Johanna Amann
979d43eac0 Netcontrol: more logging in catch-and-release
Catch-and-release logs now include the plugin that is responsible for an
action. Furthermore, the catch-and-release log also includes instances
where a rule already existed, and where an error occurred during an
operation.
2024-02-05 14:38:21 +00:00
Johanna Amann
273731e1ce Merge branch 'topic/johanna/fix-logging-of-ssl-log-ext-in-some-cases'
* topic/johanna/fix-logging-of-ssl-log-ext-in-some-cases:
  Fix ssl-log-ext omitting data in some cases
2024-01-16 13:10:35 +00:00
Johanna Amann
46e54bab76 Fix ssl-log-ext omitting data in some cases
ssl-log-ext had a bug that caused data present in the SSL connection to
not be logged in some cases. Specifically, the script relied on the base
ssl script to initialize some data structures; however, this means that
protocol messages that arrive before a message is handled by the base
ssl script are not logged.

This commit changes the ssl-log-ext script to also initialize the data
structures; now messages are correctly included in the log in all cases.
2024-01-11 17:12:33 +00:00
Arne Welzel
60d8872969 Known: Keep &create_expire on local tables/sets valid
After switching the known scripts away from broker stores, the
&create_expire value of the local tables/sets of the known scripts
wasn't in effect due to Cluster::node_up() and Cluster::node_down()
re-assigning these without keeping the &create_expire attribute
intact. This broke the "log hosts every 24h" behavior.

Closes #3540
2024-01-10 09:07:43 +01:00
Vern Paxson
a927d61e59 minor fixes for script optimization, exporting of attr_name, script layout tweak 2023-12-04 16:48:19 -08:00
Arne Welzel
384453346d policy/misc/load-balancing: Deprecate script 2023-11-07 16:06:16 +01:00
Justin Azoff
047232df37 Fix check for emailed notices
This bug was introduced in 9d0bd96
2023-09-01 13:23:45 -04:00
Johanna Amann
4156c4764a Community-id: load main script in notice script, fix notice script
This change makes the community-id script that adds the community id to
notice.log automatically load the main script if this was not already
loaded.

In the past, the script just did not perform any action if the main
script was not loaded.

This change also makes the notice script respect the seed/base64
settings that were set in the main script.

Fixes GH-3242
2023-08-22 15:09:47 +01:00
Arne Welzel
0b317aced3 telemetry: Disable metrics centralization by default
Move the telemetry/cluster.zeek file over into policy/frameworks/telemetry/prometheus.zeek.

Mention it in local.zeek.

Relates to zeek/broker#366.
2023-06-21 20:13:55 +02:00
Tim Wojtulewicz
a55e5e3724 Remove full scripts marked as 6.1 deprecations 2023-06-14 10:07:22 -07:00
Arne Welzel
7a043e5e8f all: Fix typos identified by typos pre-commit hook 2023-06-13 17:57:32 +02:00
Arne Welzel
737ac43942 catch-and-release: Mark cr_check_rule as is_used 2023-05-31 00:37:38 -07:00
Tim Wojtulewicz
5a3abbe364 Revert "Merge remote-tracking branch 'origin/topic/vern/at-if-analyze'"
This reverts commit 4e797ddbbc, reversing
changes made to 3ac28ba5a2.
2023-05-31 09:20:33 +02:00
Tim Wojtulewicz
4e797ddbbc Merge remote-tracking branch 'origin/topic/vern/at-if-analyze'
* origin/topic/vern/at-if-analyze:
  updates reflecting review comments
  change base scripts to use run-time if's or @if ... &analyze
  a number of BTests updated with @if ... &analyze
  update for scripting coverage BTest demonstrating utility of @if ... &analyze
  BTests for new @if ... &analyze functionality
  "if ( ... ) &analyze" language feature
  classes for tracking "@if (...) &analyze" notion of code being/not being "activated"
  RemoveGlobal() method for Scope class + simplifying interfaces
2023-05-26 14:56:01 -07:00
Vern Paxson
e441ba394a updates reflecting review comments 2023-05-25 18:00:13 -07:00
Arne Welzel
f396c2b16e stats: Add zeek-net-packet-lag-seconds metric
While writing documentation about troubleshooting and looking a bit
at the older stats.log, realized we don't have the packet lag metric
exposed as metric/telemetry. Add it.

This is a Zeek instance lagging behind in network time ~6second because
it's very overloaded:

    zeek_net_packet_lag_seconds{endpoint=""} 6.169406 1684848998092
2023-05-24 09:12:01 +02:00
Vern Paxson
890010915a change base scripts to use run-time if's or @if ... &analyze 2023-05-19 13:26:27 -07:00
Robin Sommer
ecf00295c2
Move spicy/misc scripts to policy and clarify purpose. 2023-05-16 10:21:21 +02:00
Arne Welzel
e82a7455d7 policy/community-id: Do not use new_connection()
Issue #3028 tracks how a flipped connections reset a connection's value
including any state set during new_connection(). For the time being,
update community-id functionality back to the original connection_state_remove()
approach to avoid missing community_ids on flipped connections.
2023-05-10 17:48:33 +02:00
Arne Welzel
ffb73e4de9 Merge remote-tracking branch 'origin/topic/awelzel/add-community-id'
* origin/topic/awelzel/add-community-id:
  testing/external: Bump hashes for community_id addition
  NEWS: Add entry for Community ID
  policy: Import zeek-community-id scripts into protocols/conn frameworks/notice
  Add community_id_v1() based on corelight/zeek-community-id
2023-04-24 10:12:56 +02:00
Christian Kreibich
b90351b7e6 policy: Import zeek-community-id scripts into protocols/conn frameworks/notice
Slightly adapted for indentation.
2023-04-24 09:43:19 +02:00
Jan Grashoefer
0cd32ba07c Add cluster_started and node_fully_connected events. 2023-04-21 19:04:52 +02:00
Tim Wojtulewicz
67802e711a Report packet statistics via the telemetry framework 2023-04-06 13:41:09 -07:00
Tim Wojtulewicz
ae3d6a4df0 Add optional packet filtered statistics for packet sources 2023-04-06 09:47:04 -07:00
Vern Paxson
9560e38784 fixed type mismatch for ssl_certificate_request event 2023-04-01 12:56:54 -07:00
Johanna Amann
b8d658ac77 SSL/TLS CertificateRequest message: Address review feedback
Minor stylistic changes; see https://github.com/zeek/zeek/pull/2855 for
details
2023-03-15 09:19:26 +01:00
Johanna Amann
b56b856da9 SSL/TLS: Parse CertificateRequest message
This commit introduces parsing of the CertificateRequest message in the
TLS handshake. It introduces a new event ssl_certificate_request, as
well as a new function parse_distinguished_name, which can be used to
parse part of the ssl_certificate_request event parameters.

This commit also introduces a new policy script, which appends
information about the CAs a TLS server requests in the
CertificateRequest message, if it sends it.
2023-03-09 09:12:29 +01:00
Tim Wojtulewicz
f6d3ed2395 Default known stores to false to avoid Broker stores 2023-03-02 12:48:53 -07:00
Arne Welzel
6d19c49efe intel/seen/file-names: Use file_over_new_connection()
The seen/file-names script relies on f$info$filename to be populated.
For HTTP and other network protocols, however, this field is only
populated during file_over_new_connection() that's running after
file_new().

Use the file_new() event only for files without connections and
file_over_new_connection() implies that f$conns is populated, anyway.

Special case SMB to avoid finding files twice, because there's a
custom implementation in seen/smb-filenames.zeek.

Fixes #2647
2023-01-10 10:10:28 +01:00
Arne Welzel
eb3bea4e4a mqtt: Move from policy/ into base/
Register dpd signatures and the analyzer when running in default mode.

Closes #2583
2022-11-30 10:14:20 +01:00
Josh Soref
21e0d777b3 Spelling fixes: scripts
* accessing
* across
* adding
* additional
* addresses
* afterwards
* analyzer
* ancillary
* answer
* associated
* attempts
* because
* belonging
* buffer
* cleanup
* committed
* connects
* database
* destination
* destroy
* distinguished
* encoded
* entries
* entry
* hopefully
* image
* include
* incorrect
* information
* initial
* initiate
* interval
* into
* java
* negotiation
* nodes
* nonexistent
* ntlm
* occasional
* omitted
* otherwise
* ourselves
* paragraphs
* particular
* perform
* received
* receiver
* referring
* release
* repetitions
* request
* responded
* retrieval
* running
* search
* separate
* separator
* should
* synchronization
* target
* that
* the
* threshold
* timeout
* transaction
* transferred
* transmission
* triggered
* vetoes
* virtual

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>
2022-11-02 17:36:39 -04:00
Robin Sommer
9f3a234f40
Merge remote-tracking branch 'origin/topic/awelzel/blank-identifer'
* origin/topic/awelzel/blank-identifer:
  scripts: Migrate table iteration to blank identifiers
  Introduce special treatment for the blank identifier _
2022-10-25 12:36:23 +02:00
Christian Kreibich
147283c8f5 Management framework: add websocket support to controller
The controller now listens on an additional port, defaulting to 2149, for Broker
connections via websockets. Configuration works as for the existing traditional
Broker port (2150), via ZEEK_CONTROLLER_WEBSOCKET_ADDR and
ZEEK_CONTROLLER_WEBSOCKET_PORT environment variables, as well as corresponding
redef'able constants.

To disable the websockets feature, leave ZEEK_CONTROLLER_WEBSOCKET_PORT unset
and redefine Management::Controller::default_port_websocket to 0/unknown.
2022-10-24 15:59:26 -07:00
Arne Welzel
8c5896a74d scripts: Migrate table iteration to blank identifiers
No obvious hot-cases. Maybe the describe_file() ones or the intel ones
if/when there are hot intel hits.
2022-10-24 10:36:09 +02:00
Arne Welzel
bc8fd5a4c6 Introduce generic analyzer_confirmation_info and analyzer_violation_info
Introduce two new events for analyzer confirmation and analyzer violation
reporting. The current analyzer_confirmation and analyzer_violation
events assume connection objects and analyzer ids are available which
is not always the case. We're already passing aid=0 for packet analyzers
and there's not currently a way to report violations from file analyzers
using analyzer_violation, for example.

These new events use an extensible Info record approach so that additional
(optional) information can be added later without changing the signature.
It would allow for per analyzer extensions to the info records to pass
analyzer specific info to script land. It's not clear that this would be
a good idea, however.

The previous analyzer_confirmation and analyzer_violation events
continue to exist, but are deprecated and will be removed with Zeek 6.1.
2022-09-27 17:49:51 +02:00
Robin Sommer
a60d569f7b Merge remote-tracking branch 'origin/topic/awelzel/dpd-analyzer-merger'
* origin/topic/awelzel/dpd-analyzer-merger:
  analyzer/dpd: Address review comments
  Remove @load base/frameworks/dpd from tests
  frameworks/dpd: Move to frameworks/analyzer/dpd, load by default
  scripts/dce-rpc,ntlm: Do not load base/frameworks/dpd
  btest: Remove unnecessary loading of frameworks/dpd
2022-09-07 14:30:28 +02:00
Arne Welzel
4aaebf613e analyzer/dpd: Address review comments
* word-smith deprecation message
* do not load scripts/base/analyzer in segment logging policy script
2022-09-06 14:32:10 +02:00
Christian Kreibich
e73b561dca Update Management framework to new Supervisor::NodeConfig script fields 2022-09-02 12:12:19 -07:00
Arne Welzel
9e7f2a04c1 frameworks/dpd: Move to frameworks/analyzer/dpd, load by default
* Because frameworks/analyzer is loaded via init-frameworks-and-bifs the
  dpd functionality (really just dpd.log and disabling of analyzers) is
  now enabled even in bare mode.
* Not sure we need to keep frameworks/base/dpd/__load__.zeek around
  or can just remove it right away.
2022-08-31 16:50:47 +02:00