Commit graph

2794 commits

Author SHA1 Message Date
Tim Wojtulewicz
d549e3d56a Add Telemetry::metrics_address option 2024-06-07 09:28:27 -07:00
Tim Wojtulewicz
99e64aa113 Restore label_names field in MetricOpts record 2024-06-04 14:14:58 -07:00
Tim Wojtulewicz
433c257886 Move telmetry label names out of opts records, into main metric records 2024-06-04 14:14:58 -07:00
Tim Wojtulewicz
93717ca8f8 Remove is_sum arguments from counters and gauges 2024-05-31 13:36:37 -07:00
Tim Wojtulewicz
46ff48c29a Change all instruments to only handle doubles 2024-05-31 13:36:37 -07:00
Tim Wojtulewicz
e3e806ca23 Remove all of the ZEEK_METRICS_ environment variables 2024-05-31 13:36:37 -07:00
Tim Wojtulewicz
9fb952a5f3 Regenerate docs [nomail] 2024-05-31 13:30:32 -07:00
Tim Wojtulewicz
53c3d2032a Remove the is_sum argument from BIF histogram creation methods 2024-05-31 13:30:31 -07:00
Tim Wojtulewicz
4361880e09 Remove Telemetry::metrics_export_prefixes option 2024-05-31 13:30:31 -07:00
Tim Wojtulewicz
e195d3d778 Fix some determinism issues with btests 2024-05-31 13:30:31 -07:00
Tim Wojtulewicz
84aa308527 Rework everything to access the prometheus-cpp objects more directly 2024-05-31 13:30:31 -07:00
Tim Wojtulewicz
128bf3fe9f Remove Broker metrics configuration values and methods 2024-05-31 13:30:31 -07:00
Tim Wojtulewicz
e93e4cc26d Add a services.json endpoint for Prometheus service discovery 2024-05-31 13:30:31 -07:00
Tim Wojtulewicz
4718e5cf00 Remove everything related to aggregation 2024-05-31 13:30:31 -07:00
Tim Wojtulewicz
643bb38419 Make all options const-redefs, remove all of the change handler code 2024-05-31 13:30:31 -07:00
Tim Wojtulewicz
17d09c657b Move base types from telemetry framework to init-bare 2024-05-31 13:30:31 -07:00
Tim Wojtulewicz
97a35011a7 Add necessary script-land changes 2024-05-31 13:30:31 -07:00
Johanna Amann
03b358f6d1 Merge branch 'files_pe_timestamp_sync' of https://github.com/mvhensbergen/zeek
* 'files_pe_timestamp_sync' of https://github.com/mvhensbergen/zeek:
  Don't hardcode values
  Add btest for timestamp check
  Copy timestamp from file object
2024-05-29 14:16:31 +01:00
Christian Kreibich
3d88918071 Fix Zeekygen warning for QUIC::unrecognized_version event
Docs updates have been warning about being unable to locate that identifier, and
it's because we've not defined it alongside the other QUIC events.
2024-05-24 14:30:42 -07:00
Johanna Amann
34225e83ba Update TLS consts, mainly new named curves.
Add test for X25519Kyber768Draft00 (post-quantum key agreement)
2024-05-23 14:50:36 +01:00
Vern Paxson
74bf453d6d Fix for suppressing SMB logging of previously-logged files 2024-05-18 14:13:52 -07:00
mvhensbergen
a4f73ee45f
Copy timestamp from file object
In some cases, e.g. running zeek on short pcaps as opposed to continuous packet streams, network_time() may not equal the time that was used when generating the file object.

This results in the pe.log entry having a different timestamp than its corresponding files.log entry which is strange as they refer to the exact same file.
2024-05-17 15:03:06 +02:00
Tim Wojtulewicz
29f5a49baf Merge remote-tracking branch 'origin/topic/christian/private_address_clarification'
* origin/topic/christian/private_address_clarification:
  Clarify membership in the Site::private_address_space prefix list. [skip ci]
2024-05-07 10:26:50 -07:00
Tim Wojtulewicz
6821a41c4e Move the options from policy/tuning/defaults to actual Zeek defaults, deprecate that package 2024-05-06 11:13:04 -07:00
Christian Kreibich
0554951cb6 Clarify membership in the Site::private_address_space prefix list. [skip ci]
IANA's IPv6 special-purpose address registry now has members that technically
meet the definition of not being globally reachable, but don't imply operating
locally. An example: https://datatracker.ietf.org/doc/draft-ietf-6man-sids/06/

This change just explains that distinction.
2024-05-04 11:21:26 -07:00
Johanna Amann
2217eab38a Fix cid propagation into files.log
Changes to the connection id were not propagated to files.log in all
cases.

Fixes GH-3700
2024-04-29 14:13:19 +01:00
Vern Paxson
263093af78 flag base script as being known to ZAM optimization 2024-04-25 09:15:13 -07:00
Vern Paxson
c11c2830b1 performance speed-up for SMB base scripts 2024-04-25 09:15:12 -07:00
Tim Wojtulewicz
d566961da5 Merge remote-tracking branch 'pbcullen/topic/pbcullen/remove_exclude'
* pbcullen/topic/pbcullen/remove_exclude:
  Update core.pcap.filter-warning baseline
  Add PacketFilter::remove_exclude function
2024-04-18 09:03:57 -07:00
Peter Cullen
81856cc391 Add PacketFilter::remove_exclude function
This adds a public function to the PacketFilter framework that
allows the caller to remove an exclude filter by filter id.
2024-04-17 21:25:35 +00:00
Peter Cullen
187f93cb96 Parse and store localversion string
Add localversion to the VersionDescription record and populate it
during version string parsing.
This change also modifies the version string syntax, removing the
deprecated dash (-) between beta|dev|rc and the commmit count; those
must now be separated by a period.
The test version strings were updated accordingly along with the
baseline.
2024-04-17 14:17:22 -07:00
Peter Cullen
e4942d45b9 Modify version parsing for localversion
The regex used for validating version strings was updated to allow
for an optional localversion component. The piece of the version string
also needs to be removed before parsing a possible commit number.
2024-04-17 14:17:05 -07:00
Arne Welzel
d728981b77 Merge remote-tracking branch 'origin/topic/awelzel/move-iso-9660-sig-to-policy'
* origin/topic/awelzel/move-iso-9660-sig-to-policy:
  signatures/iso-9660: Add \x01 suffix to CD001
  test-all-policy: Do not load iso-9660.zeek
  signatures: Move ISO 9660 signature to policy
2024-02-26 21:16:48 +01:00
Arne Welzel
77d57fccfd Merge remote-tracking branch 'origin/topic/awelzel/find-in-zeekpath'
* origin/topic/awelzel/find-in-zeekpath:
  NEWS: Mention can_load() and find_in_zeekpath()
  utils: Introduce packages.zeek with can_load() helper
  zeek.bif: Add find_in_zeekpath() helper
2024-02-26 21:14:59 +01:00
Arne Welzel
1a5ce65e3d signatures: Move ISO 9660 signature to policy
The previous "fix" caused significant performance degradation without
the signature ever having a chance to trigger. Moving it to policy
seems the best compromise, the alternative being outright removing it.
2024-02-26 13:35:23 +01:00
Johanna Amann
c81274b549 Update Mozilla CA and Google CT list and related tests.
Test updates are necessary due to removed CT logs.

Fixes GH-3620
2024-02-23 11:50:31 +00:00
Arne Welzel
d2409dd432 signatures: Fix ISO 9960 signature
This signature only really works when default_file_bof_buffer_size is bumped
to a sufficient value (40k).
2024-02-22 12:37:40 +01:00
Arne Welzel
036bcfe919 Merge remote-tracking branch 'origin/topic/awelzel/ftp-clear-fuid-after-logging'
* origin/topic/awelzel/ftp-clear-fuid-after-logging:
  testing: Update zeek-testing-private commit hash
  ftp: Reset fuid after logging
2024-02-22 12:23:21 +01:00
Arne Welzel
947294efab utils: Introduce packages.zeek with can_load() helper 2024-02-22 11:41:50 +01:00
Arne Welzel
31b548babc ftp: Reset fuid after logging
A user reported being confused about the fuid association of subsequent
FTP commands when a data transfer has completed. It seems reasonable to
unset fuid upon logging a FTP command which had a fuid.

The current behavior results in the PORT or PASV commands after a RETR or STOR
to have the fuid of the prior file transfer. Similarly, any CWD or DEL commands
following a file transfer will unnecessarily be logged with the fuid of the
prior file transfer.

This tickles the baselines for the private testing PCAP a lot, primarily
because there data connections in that pcap are never established properly.
E.g, the fuids FzDzid1Dxm9srVKHXf and FEfYX73q5C6GEQZXX9 have been re-used
for multiple commands.

This may look like we're losing information, but the fuids vanishing
in the normal btests belong to a LIST command that isn't logged by
default into ftp.log. If it was, the fuid would be attached to it.
2024-02-21 12:41:32 +01:00
Arne Welzel
c1a685a05d websocket: Add Spicy parser version, too.
The Spicy analyzer is added as a child analyzer when enabled and the
WebSocket.cc logic dispatches between the BinPac and Spicy version.

It substantially slower when tested against a somewhat artificial
2.4GB PCAP. The first flamegraph indicates that the unmask() function
stands out with 35% of all samples, and above it shared_ptr samples.
2024-02-06 17:29:55 +01:00
Johanna Amann
160ccda45f Merge remote-tracking branch 'origin/topic/johanna/netcontrol-updates'
* origin/topic/johanna/netcontrol-updates:
  Netcontrol: add rule_added_policy
  Netcontrol: more logging in catch-and-release
  Netcontrol: allow supplying explicit name to Debug plugin
2024-02-06 11:54:26 +00:00
Christian Kreibich
873d734c79 Do not default PoolSpec topics to the empty string.
Similar to `node_topic`, we already spell out a topic in the existing use and
there's no obviously meaningful default value.
2024-02-05 18:03:08 -08:00
Christian Kreibich
8437012346 Do not default to proxy nodes in Broker::PoolSpec
This requires pool creation to spell out a spec explicitly, which the only code
using these types already does. There's no reason for pools to automatically
refer to proxies.
2024-02-05 17:51:11 -08:00
Johanna Amann
737f828bcb Netcontrol: add rule_added_policy
rule_added_policy allows the modification of rules just after they have
been added. This allows the implementation of some more complex features
- like changing rule states depending on insertion in other plugins.
2024-02-05 18:52:27 +00:00
Johanna Amann
979d43eac0 Netcontrol: more logging in catch-and-release
Catch-and-release logs now include the plugin that is responsible for an
action. Furthermore, the catch-and-release log also includes instances
where a rule already existed, and where an error occurred during an
operation.
2024-02-05 14:38:21 +00:00
Johanna Amann
2df520414e Netcontrol: allow supplying explicit name to Debug plugin
This change extends the arguments of NetControl::create_debug, and
allows the specification of an optional name argument, which can be used
instead of the default-generated name.

This is helpful when one wants to attach several plugins to verify
behavior in those cases.
2024-02-03 18:14:29 +00:00
Christian Kreibich
2cbc41a70c Fix markup typo in the Intel::seen_policy hook docstring [skip ci] 2024-02-02 12:30:39 -08:00
Arne Welzel
1c516f738c quic: Log client's source connection id, too.
Seem reasonable give we log the server SCID. Interestingly, the Chromium
examples actually have zero length (empty) source connection IDs. I wonder
if that's part of their "protocol ossification avoidance" effort.
2024-01-30 21:46:38 +01:00
Arne Welzel
ac524b4a33 quic: Rename set_conn() to set_session()
set_conn() should be about the c$conn record. Most other base scripts
for protocols use set_session(), so do the same.
2024-01-30 21:46:38 +01:00