Commit graph

16055 commits

Author SHA1 Message Date
Arne Welzel
32ee6960f2 telemetry: Switch to histogram family bucket and bound accessors
Instead of determining the bucket bounds from a concrete metric instance,
broker now provides accessors via the concrete histogram family handles.
2022-08-09 11:00:07 +02:00
Tim Wojtulewicz
514df9f179 Fix module-scoped type definitions that conflict with existing global ones 2022-08-08 15:30:34 -07:00
Tim Wojtulewicz
cb3abccfb1 Merge remote-tracking branch 'ynadji/topic/yacin/2319-add-change-handler-to-site'
* ynadji/topic/yacin/2319-add-change-handler-to-site:
  update plugins.hooks baseline
  lower priority for change handlers
  split update_zones_regex into two functions
  GH-2319: Add change handlers to Site
2022-08-08 11:09:16 -07:00
Tim Wojtulewicz
3da1848279 Merge remote-tracking branch 'AmazingPP/topic/amazingpp/command-line-script-warning'
* AmazingPP/topic/amazingpp/command-line-script-warning:
  Squelch the zeekygen warnings for command line
2022-08-08 11:03:32 -07:00
Arne Welzel
c2ca92d772 Try adding Broker::metrics_import_topics, stuck 2022-08-08 17:20:13 +02:00
Yacin Nadji
84610ed832 update plugins.hooks baseline 2022-08-08 11:52:06 +02:00
Yacin Nadji
825fb1c24a lower priority for change handlers 2022-08-08 11:47:52 +02:00
Yacin Nadji
dc1102e9dd split update_zones_regex into two functions 2022-08-08 11:40:18 +02:00
AmazingPP
9869226e1e Squelch the zeekygen warnings for command line 2022-08-06 11:35:05 +08:00
zeek-bot
857b21ae73 Update doc submodule [nomail] [skip ci] 2022-08-06 00:46:41 +00:00
TheAvgJojo
1e37e91eda UPDATED: improving email address splitting for common comma case 2022-08-05 17:38:54 -04:00
Arne Welzel
acacc6b6c2 telemetry: Remove singleton BIFs and the C++ pieces
The low-level singleton Telemetry BIFs have been removed with the that there
haven't been any users. Singleton metrics can be instantiated by providing
an empty label vector instead and aren't in any way a special concept.

Closes #2262.
2022-08-05 19:33:49 +02:00
Tim Wojtulewicz
9524963da6 Merge remote-tracking branch 'anniebryan/notice-suppression-bug-fix'
* anniebryan/notice-suppression-bug-fix:
  Added test case with back-to-back notices
  Fix notice suppression atomicity bug
2022-08-05 16:21:29 +00:00
Tim Wojtulewicz
66c4e57442 Merge remote-tracking branch 'origin/topic/timw/2034-ifdef-namespace-name'
* origin/topic/timw/2034-ifdef-namespace-name:
  GH-2034: Store module names and use them in lookups for ifdef
2022-08-05 16:20:12 +00:00
Tim Wojtulewicz
1870d26684 GH-2034: Store module names and use them in lookups for ifdef 2022-08-05 15:36:21 +00:00
Robin Sommer
686e740bbe
Merge remote-tracking branch 'origin/topic/awelzel/telemetry-script-land-v0'
* origin/topic/awelzel/telemetry-script-land-v0:
  Introduce telemetry framework
  Bump broker submodule to master.
2022-08-05 17:14:23 +02:00
Yacin Nadji
4a04b563f5 GH-2319: Add change handlers to Site 2022-08-05 16:17:50 +02:00
Arne Welzel
3fe930dbf2 Introduce telemetry framework
Adds base/frameworks/telemetry with wrappers around telemetry.bif
and updates telemetry/Manager to support collecting metrics from
script land.

Add policy/frameworks/telemetry/log for logging of metrics data
into a new telemetry.log and telemetry_histogram.log and add into
local.zeek by default.
2022-08-05 11:43:44 +02:00
zeek-bot
901bc1e983 Update doc submodule [nomail] [skip ci] 2022-08-05 00:35:30 +00:00
Annie Bryan
d916469510 Added test case with back-to-back notices 2022-08-04 11:25:20 +02:00
Annie Bryan
e96d3743b4
Merge branch 'zeek:master' into notice-suppression-bug-fix 2022-08-04 10:03:51 +02:00
Arne Welzel
95fba8fd29 Bump broker submodule to master.
Includes changes to run a collector in the broker::telemetry layer,
as well as pybind upgrade, no-discard removals and formatting changes.
2022-08-04 09:16:10 +02:00
Tim Wojtulewicz
9b546cbb2b Merge remote-tracking branch 'origin/topic/awelzel/config-util-fmt-thread-fmt'
* origin/topic/awelzel/config-util-fmt-thread-fmt:
  input/config: Use thread-safe Fmt() rather than util::fmt() for regex formatting
2022-08-03 10:27:38 -07:00
Tim Wojtulewicz
930fd75ed8 Merge remote-tracking branch 'origin/topic/awelzel/2010-zeek-done-change-handlers'
* origin/topic/awelzel/2010-zeek-done-change-handlers:
  testing/missing-enum-value: redef exit_only_after_terminate=T
  option.bif: Short-circuit option changes when terminating
2022-08-03 10:27:06 -07:00
Tim Wojtulewicz
3ffffe33bc Merge remote-tracking branch 'origin/topic/vern/bit-shift-fixes'
* origin/topic/vern/bit-shift-fixes:
  btest portability fix address review comment about shifting corner-case
  canonicalize filenames for new vector deprecation btest
  updates for gen-C++ maintenance, including skipping some inappropriate tests
  fix for profiling "when" statements
  gen-C++ support for vector bit-shift operations
  corrected wording in some btest comments
  make gen-C++ maintenance scripts directly executable
  ZAM support for bit-shifting
  don't allow deprecated-style mixing of vectors and scaling for shifting leverage restrictions placed on shifting (RHS is always unsigned) split deprecated vector operations into separate test, with separate ZAM baseline
  ZAM fix for vector "in" operator
  ensure that language tests pay attention to .stderr
  fix vector tests, including checking for errors
2022-08-03 09:56:55 -07:00
Annie Bryan
82ee17be8d Fix notice suppression atomicity bug 2022-08-03 16:30:57 +02:00
Vern Paxson
c10885e5f9 btest portability fix
address review comment about shifting corner-case
2022-08-02 19:36:25 -07:00
zeek-bot
85e8d17887 Update doc submodule [nomail] [skip ci] 2022-08-03 00:42:36 +00:00
Arne Welzel
866f8269ab input/config: Use thread-safe Fmt() rather than util::fmt() for regex formatting
Calling util::fmt() from DoUpdate() of a thread is not safe as it is
using a statically allocated buffer and other threads or the main
thread may concurrently modify this buffer.

This was found by observing the scripts.base.frameworks.config.several-files
failing once in a blue moon (1/250 sometimes 1/1000 runs) with messages like
"Failed to compile regex: Parenthesis is not closed.":

    scripts.base.frameworks.config.several-files ...
      > btest-bg-run zeek zeek -b %INPUT
      > btest-bg-wait 10
    ... scripts.base.frameworks.config.several-files failed
      % 'btest-bg-wait 10' failed unexpectedly (exit code 1)
      % cat .stderr
      The following processes did not terminate:
      zeek -b /home/awelzel/corelight-oss/zeek/testing/btest/.tmp/scripts.base.frameworks.config.several-files/several-files.zeek
      -----------
      <<< [3667265] zeek -b /home/awelzel/corelight-oss/zeek/testing/btest/.tmp/scripts.base.frameworks.config.several-files/several-files.zeek
      error: ../configfile1/Input::READER_CONFIG: Failed to compile regex: Parenthesis is not closed.
      received termination signal
      >>>
2022-08-02 21:43:03 +02:00
Tim Wojtulewicz
c82eb13cc8 Merge remote-tracking branch 'pbcullen/topic/pbcullen/prevent-large-dhcp-logs'
* pbcullen/topic/pbcullen/prevent-large-dhcp-logs:
  Update plugins.hooks baseline with new DHCP options
  Prevent large dhcp log entries
2022-08-02 12:38:03 -07:00
Vern Paxson
ea3d25b380 canonicalize filenames for new vector deprecation btest 2022-08-02 12:37:33 -07:00
Tim Wojtulewicz
2cba2415fd Merge remote-tracking branch 'origin/topic/timw/1129-slash-s-patterns'
* origin/topic/timw/1129-slash-s-patterns:
  Add support for /s modifier to RE matcher and parser
  Code cleanup in RE_Matcher code
  Add basic unit tests for RE_Matcher
  Add /s modifier to parser for patterns
2022-08-02 11:33:35 -07:00
Tim Wojtulewicz
18126c2d50 Add support for /s modifier to RE matcher and parser 2022-08-02 11:31:57 -07:00
Tim Wojtulewicz
f67f6e4507 Code cleanup in RE_Matcher code
- Use std::string in Specific_RE_Matcher instead of char*
- Change a couple of ints-as-bools to bools
2022-08-02 11:31:57 -07:00
Tim Wojtulewicz
abf2da781d Add basic unit tests for RE_Matcher 2022-08-02 11:31:57 -07:00
Tim Wojtulewicz
36e31e28ac Add /s modifier to parser for patterns 2022-08-02 11:31:57 -07:00
Tim Wojtulewicz
14e48733ac Update gen-zam submodule [nomail] 2022-08-02 10:43:52 -07:00
Fatema BW
660278be1f
Update src/packet_analysis/protocol/tcp/TCPSessionAdapter.cc
Co-authored-by: Robin Sommer <robin@icir.org>
2022-08-02 10:02:32 -07:00
Fatema BW
1348b739ab
Updating the weird names to use all lower case 2022-08-02 10:01:31 -07:00
Fatema BW
5ffeb657a5
Fixing whitespaces.. 2022-08-02 09:59:34 -07:00
AmazingPP
2c4449e19a Use json_escape_utf8 in broker debug output 2022-08-02 21:11:40 +08:00
peter.cullen
d93eb0b002 Update plugins.hooks baseline with new DHCP options 2022-08-02 11:42:04 +00:00
Vern Paxson
7a41170a59 updates for gen-C++ maintenance, including skipping some inappropriate tests 2022-08-01 16:47:17 -07:00
Vern Paxson
c034dc68ed fix for profiling "when" statements 2022-08-01 13:36:24 -07:00
Vern Paxson
1bbf345e46 gen-C++ support for vector bit-shift operations 2022-08-01 10:26:26 -07:00
Tim Wojtulewicz
6fd0645aef Merge branch 'topic/timw/1344-illegal-base-type'
* topic/timw/1344-illegal-base-type:
  GH-1344: Give better warning when using a type that doesn't exist
2022-08-01 09:57:05 -07:00
Vern Paxson
8640430920 corrected wording in some btest comments 2022-07-31 15:06:02 -07:00
Vern Paxson
f912ac06ff make gen-C++ maintenance scripts directly executable 2022-07-31 11:31:01 -07:00
Vern Paxson
f535a7c3a7 ZAM support for bit-shifting 2022-07-30 18:33:57 -07:00
Vern Paxson
99e265bf87 don't allow deprecated-style mixing of vectors and scaling for shifting
leverage restrictions placed on shifting (RHS is always unsigned)
split deprecated vector operations into separate test, with separate ZAM baseline
2022-07-30 18:17:22 -07:00