Commit graph

3980 commits

Author SHA1 Message Date
Arne Welzel
abb7f9a509 Introduce global disabling_analyzer() hook to veto disable_analyzer()
This hook can be used to coordinate disabling an analyzer for a given
connection. The contract is simple: Any script can veto a disable_analyzer()
call by breaking from this hook. The decision is local to the script taking
into account any state attached to the connection object or script specific
state stored elsewhere.
A script breaking from the hook takes over the responsibility to call
disable_analyzer() at a later point when it finds the condition due to which
it vetoed fulfilled (which may be never).

Signature:

    disabling_analyzer: hook(c: connection, atype: AllAnalyzers::Tag, aid: count);

Example use-cases are keeping the SSL analyzer enabled for finger-printing
until a certain amount of bytes or packets have been transferred or
similarly the connection duration exceed a certain threshold.

Other example use-cases might be keeping analyzers for SSH, RDP or SSL
enabled for connections from specific subnets.

It's a bit quirky as it makes disable_analyzer() a maybe operation. While log
policy hooks and/or the notice hook have similar semantics, they are not as
stateful. It still seems like a quite powerful primitive.

The disable_analyzer() call in dpd/main.zeek may motivate the addition of a
force flag as a follow-up for situations where the caller "knows better" or
absolutely wants to override.

Closes #1678 #1593.
2022-08-11 09:40:36 +02:00
Christian Kreibich
3a0a702723 Merge branch 'topic/christian/management-bugfixes'
* topic/christian/management-bugfixes:
  Management framework: bump cluster testsuite
  Management framework: log node set in dispatch requests cleanly
  Management framework: log additional node events
  Management framework: upon deployment, make agent log multiple node results
  Management framework: fix early return condition for get-id-value
2022-08-10 14:18:38 -07:00
Christian Kreibich
20e08f0d38 Management framework: bump cluster testsuite 2022-08-10 11:24:10 -07:00
Tim Wojtulewicz
514df9f179 Fix module-scoped type definitions that conflict with existing global ones 2022-08-08 15:30:34 -07:00
Tim Wojtulewicz
cb3abccfb1 Merge remote-tracking branch 'ynadji/topic/yacin/2319-add-change-handler-to-site'
* ynadji/topic/yacin/2319-add-change-handler-to-site:
  update plugins.hooks baseline
  lower priority for change handlers
  split update_zones_regex into two functions
  GH-2319: Add change handlers to Site
2022-08-08 11:09:16 -07:00
Tim Wojtulewicz
3da1848279 Merge remote-tracking branch 'AmazingPP/topic/amazingpp/command-line-script-warning'
* AmazingPP/topic/amazingpp/command-line-script-warning:
  Squelch the zeekygen warnings for command line
2022-08-08 11:03:32 -07:00
Yacin Nadji
84610ed832 update plugins.hooks baseline 2022-08-08 11:52:06 +02:00
AmazingPP
9869226e1e Squelch the zeekygen warnings for command line 2022-08-06 11:35:05 +08:00
Tim Wojtulewicz
9524963da6 Merge remote-tracking branch 'anniebryan/notice-suppression-bug-fix'
* anniebryan/notice-suppression-bug-fix:
  Added test case with back-to-back notices
  Fix notice suppression atomicity bug
2022-08-05 16:21:29 +00:00
Tim Wojtulewicz
1870d26684 GH-2034: Store module names and use them in lookups for ifdef 2022-08-05 15:36:21 +00:00
Robin Sommer
686e740bbe
Merge remote-tracking branch 'origin/topic/awelzel/telemetry-script-land-v0'
* origin/topic/awelzel/telemetry-script-land-v0:
  Introduce telemetry framework
  Bump broker submodule to master.
2022-08-05 17:14:23 +02:00
Yacin Nadji
4a04b563f5 GH-2319: Add change handlers to Site 2022-08-05 16:17:50 +02:00
Arne Welzel
3fe930dbf2 Introduce telemetry framework
Adds base/frameworks/telemetry with wrappers around telemetry.bif
and updates telemetry/Manager to support collecting metrics from
script land.

Add policy/frameworks/telemetry/log for logging of metrics data
into a new telemetry.log and telemetry_histogram.log and add into
local.zeek by default.
2022-08-05 11:43:44 +02:00
Annie Bryan
d916469510 Added test case with back-to-back notices 2022-08-04 11:25:20 +02:00
Tim Wojtulewicz
930fd75ed8 Merge remote-tracking branch 'origin/topic/awelzel/2010-zeek-done-change-handlers'
* origin/topic/awelzel/2010-zeek-done-change-handlers:
  testing/missing-enum-value: redef exit_only_after_terminate=T
  option.bif: Short-circuit option changes when terminating
2022-08-03 10:27:06 -07:00
Tim Wojtulewicz
3ffffe33bc Merge remote-tracking branch 'origin/topic/vern/bit-shift-fixes'
* origin/topic/vern/bit-shift-fixes:
  btest portability fix address review comment about shifting corner-case
  canonicalize filenames for new vector deprecation btest
  updates for gen-C++ maintenance, including skipping some inappropriate tests
  fix for profiling "when" statements
  gen-C++ support for vector bit-shift operations
  corrected wording in some btest comments
  make gen-C++ maintenance scripts directly executable
  ZAM support for bit-shifting
  don't allow deprecated-style mixing of vectors and scaling for shifting leverage restrictions placed on shifting (RHS is always unsigned) split deprecated vector operations into separate test, with separate ZAM baseline
  ZAM fix for vector "in" operator
  ensure that language tests pay attention to .stderr
  fix vector tests, including checking for errors
2022-08-03 09:56:55 -07:00
Vern Paxson
c10885e5f9 btest portability fix
address review comment about shifting corner-case
2022-08-02 19:36:25 -07:00
Tim Wojtulewicz
c82eb13cc8 Merge remote-tracking branch 'pbcullen/topic/pbcullen/prevent-large-dhcp-logs'
* pbcullen/topic/pbcullen/prevent-large-dhcp-logs:
  Update plugins.hooks baseline with new DHCP options
  Prevent large dhcp log entries
2022-08-02 12:38:03 -07:00
Vern Paxson
ea3d25b380 canonicalize filenames for new vector deprecation btest 2022-08-02 12:37:33 -07:00
Tim Wojtulewicz
18126c2d50 Add support for /s modifier to RE matcher and parser 2022-08-02 11:31:57 -07:00
peter.cullen
d93eb0b002 Update plugins.hooks baseline with new DHCP options 2022-08-02 11:42:04 +00:00
Vern Paxson
7a41170a59 updates for gen-C++ maintenance, including skipping some inappropriate tests 2022-08-01 16:47:17 -07:00
Vern Paxson
8640430920 corrected wording in some btest comments 2022-07-31 15:06:02 -07:00
Vern Paxson
99e265bf87 don't allow deprecated-style mixing of vectors and scaling for shifting
leverage restrictions placed on shifting (RHS is always unsigned)
split deprecated vector operations into separate test, with separate ZAM baseline
2022-07-30 18:17:22 -07:00
Vern Paxson
5569d81953 ensure that language tests pay attention to .stderr 2022-07-29 21:30:40 -07:00
Vern Paxson
8d3265d540 fix vector tests, including checking for errors 2022-07-29 21:26:02 -07:00
Tim Wojtulewicz
8e4017f781 GH-1344: Give better warning when using a type that doesn't exist 2022-07-29 12:38:25 -07:00
Arne Welzel
07f1895925 testing/missing-enum-value: redef exit_only_after_terminate=T
Seems this was the intention all along as the corresponding terminate()
call is there.
2022-07-29 18:22:03 +02:00
Arne Welzel
11cde53373 option.bif: Short-circuit option changes when terminating
Due to the asynchronous behavior of the input framework and broker
communication, change handlers were previously called even after
zeek_done() event processing completed and also broker shutdown.

Accessing broker store handles within change handlers this late
triggered invalid Broker store handle messages:

    error in ././my_option_store.zeek, line 13: invalid Broker store handle (Broker::put(Test::store, to_any_coercemy_option, to_any_coerceTest::new_value, 0 secs) and broker::store::{})

Fixes #2010
2022-07-29 18:22:03 +02:00
Peter Cullen
fb4858d42b Prevent large dhcp log entries
A flood of DHCP traffic can result if very large log entries consisting
of many uids and/or msg_types. Such large log entries can disrupt a SIEM
ingestion pipeline. This change forcing a log entry to be written when
the number of uids or the number of msg_Types exceed a certain value.
The values are treated as options for easy configuration.
2022-07-28 11:34:18 -07:00
Tim Wojtulewicz
25f88bc394 Add btest for vector bit-shift operators 2022-07-25 11:32:42 -07:00
Tim Wojtulewicz
8fff3c76b9 Handle error cases for bit-shift operators more cleanly 2022-07-25 11:32:42 -07:00
Tim Wojtulewicz
00e111135b Return an error if redef'ing a variable without redef attribute 2022-07-25 09:43:19 -07:00
Tim Wojtulewicz
b41a4bf06d GH-1628: Return an error for duplicate record field names 2022-07-21 11:53:47 -07:00
Tim Wojtulewicz
d57ad3e405 Fix a bug with splitting strings in Ascii input reader
The previous way of splitting strings would break if the last string in
the line was an empty string, and it would return one fewer fields than
it should have. This was breaking the last line in the
scripts.base.framework.input.ascii.setspecialcases once the bug fixed in
GH #1628 was fixed.
2022-07-21 11:53:47 -07:00
Tim Wojtulewicz
5665696a05 Merge remote-tracking branch 'amazingpp/topic/amazingpp/bitshift-operators'
* amazingpp/topic/amazingpp/bitshift-operators:
  Add bitshift operators
2022-07-21 09:06:41 -07:00
AmazingPP
f2bfa61fc6 Add bitshift operators 2022-07-21 11:12:30 +08:00
Robin Sommer
ad2557755d Merge remote-tracking branch 'origin/topic/robin/gh-2267-doctest'
* origin/topic/robin/gh-2267-doctest:
  Add btest that runs the doctest-based unit tests.
2022-07-19 15:21:39 +02:00
Robin Sommer
d5db2623dc Merge remote-tracking branch 'origin/topic/robin/gh-2280-modifiable-const-via-loops-merge'
* origin/topic/robin/gh-2280-modifiable-const-via-loops-merge:
  Add help function to check loop variable
  Fix local const variables can be modified via loops
2022-07-19 15:20:34 +02:00
Robin Sommer
f40ca42590 Merge branch 'topic/amazingpp/modifiable-const-via-loops' of ssh://github.com/AmazingPP/zeek
Edits: Slight tweaking, plus a simple test.

* 'topic/amazingpp/modifiable-const-via-loops' of ssh://github.com/AmazingPP/zeek:
  Add help function to check loop variable
  Fix local const variables can be modified via loops
2022-07-19 14:26:59 +02:00
Robin Sommer
265fa83a24 Merge remote-tracking branch 'origin/topic/awelzel/2276-smb2-delete-on-close'
* origin/topic/awelzel/2276-smb2-delete-on-close:
  smb2: Raise smb2_file_delete for CREATE with FILE_DELETE_ON_CLOSE
2022-07-19 12:02:16 +02:00
Robin Sommer
202037181c Add btest that runs the doctest-based unit tests.
Closes #2267.
2022-07-19 11:35:09 +02:00
Tim Wojtulewicz
05bb509789 GH-636: Fix regex to handle commas at the end of config parser lines 2022-07-18 08:54:06 -07:00
Arne Welzel
3dae8ab086 smb2: Raise smb2_file_delete for CREATE with FILE_DELETE_ON_CLOSE
When a CREATE request contains the FILE_DELETE_ON_CLOSE option and
the subsequent CREATE response indicates success, we now raise the
smb2_file_delete event to log a delete action in smb_files.log and
also give users a way to handle this scenario.

The provided pcap was generated locally by recording a smbtorture run
of the smb2.delete-on-close-perms test case.

Placed the create_options into the CmdInfo record for potential
exposure in smb_cmd.log (wasn't sure how that would look so left it
for the future).

Fixes #2276.
2022-07-16 17:14:13 +02:00
Christian Kreibich
6543c21d23 Merge branch 'topic/christian/management-agent-boot-fix'
* topic/christian/management-agent-boot-fix:
  Management framework: bump cluster testsuite
  Management framework: undo the testsuite result mask
  Management framework: additional logging tweaks
  Management framework: additional context in a few log messages
  Management framework: await Supervisor peering before sending agent's hello
  Management framework: remove outdated comment
2022-07-14 15:50:14 -07:00
AmazingPP
a76dc6a389 Add pow function 2022-07-14 22:13:34 +08:00
Christian Kreibich
8d10cbfb36 Fix requirement check in a logging framework / sqlite btest 2022-07-13 17:20:03 -07:00
Christian Kreibich
48486b4156 Merge branch 'topic/christian/gh-2239-stdin-ctrl-c'
* topic/christian/gh-2239-stdin-ctrl-c:
  Stop signal-masking upon running unit tests
  Pause signal-masking during script parsing
  Add btests to verify Zeek's handling of SIGTERM and reading stdin
  Add procps/procps-ng to several CI Docker images
2022-07-13 11:57:52 -07:00
Tim Wojtulewicz
ed2934d345 Merge remote-tracking branch 'origin/topic/bbannier/format-btest_testing_plugins'
* origin/topic/bbannier/format-btest_testing_plugins:
  Use clang-format for all files in `testing/btest/plugins`.
2022-07-13 11:23:45 -07:00
Tim Wojtulewicz
7e56605d83 GH-696: Add bytestring_to_float BIF 2022-07-13 10:44:24 -07:00