Commit graph

16055 commits

Author SHA1 Message Date
Arne Welzel
93f3c2b612 testing/external: Bump cluster testing commit 2022-12-09 16:59:36 +01:00
Arne Welzel
a0aa00fa81 logging: Add event_groups to Stream
This commit adds an optional event_groups field to the Logging::Stream record
to associated event groups with logging streams.

This can be used to disable all event groups of a logging stream when it is
disabled. It does require making an explicit connection between the
logging stream and the involved groups, however.
2022-12-09 16:59:36 +01:00
Arne Welzel
ba4b8faea2 zeek.bif: Add has_event_group() / has_module_events()
Introduce helpers to determine if a given attribute or module event
group exists given a string.
2022-12-09 16:59:07 +01:00
Arne Welzel
33244ae75f Merge remote-tracking branch 'origin/topic/vern/script-opt-maint.Dec22'
* origin/topic/vern/script-opt-maint.Dec22:
  maintenance updates for -O C++
  maintenance updates for -O ZAM
  update test suite to avoid GH-2385 problems, including incorrect typing
2022-12-09 16:08:26 +01:00
Arne Welzel
7ca78ea195 Func: Use std::stable_sort() for sorting bodies by priority
Avoids platform/stdlib dependent sorting of bodies with the same
priority and the same handler. They should be guaranteed to be
executed in load order now.
2022-12-09 15:34:54 +01:00
Arne Welzel
63aaf91bf7 btest/dump-events: Do not skip everywhere and update baselines 2022-12-09 15:34:54 +01:00
Arne Welzel
ded011ef02 Merge remote-tracking branch 'origin/topic/awelzel/dfa-transition-counter'
* origin/topic/awelzel/dfa-transition-counter:
  dfa: Remove transition_counter
2022-12-08 10:28:34 +01:00
zeek-bot
b9684faf5b Update doc submodule [nomail] [skip ci] 2022-12-07 00:36:15 +00:00
Tim Wojtulewicz
93166d0ac4 Remove .lgtm.yml 2022-12-06 11:06:32 -07:00
Arne Welzel
5d64105353 dfa: Remove transition_counter
I believe that's unused since 2011 (8eb241fde7).
2022-12-06 17:27:31 +01:00
Arne Welzel
a349679a4c Merge remote-tracking branch 'origin/topic/awelzel/2629-notice-file-info'
* origin/topic/awelzel/2629-notice-file-info:
  analyzer/files: handle non-analyzer names in describe_file()
  frameworks/notice: Handle fa_file with no or more than a single connection better
2022-12-06 12:23:47 +01:00
Arne Welzel
7bbdce61e8 Merge remote-tracking branch 'origin/topic/awelzel/2547-fix-supress-weird-spelling'
* origin/topic/awelzel/2547-fix-supress-weird-spelling:
  ContentLine: Fix spelling of "suppress", deprecate SupressWeirds()
2022-12-06 12:18:36 +01:00
Arne Welzel
2246a3344d Merge remote-tracking branch 'origin/topic/awelzel/2577-zeekygen-special-case-plugin-dir'
* origin/topic/awelzel/2577-zeekygen-special-case-plugin-dir:
  zeekygen/normalize_script_path: Special case plugin dirnames without _
2022-12-06 12:14:33 +01:00
Arne Welzel
85ce48eb1e analyzer/files: handle non-analyzer names in describe_file()
When a fa_file object is created through the use of Input::add_analysis(),
the fa_file's source is likely not valid representation of an analyzer's
tag and a Files::describe() should not error and instead return an empty
description.

Add a new Analyzer::is_tag() helper that can be used to pre-check `f$source`.
2022-12-06 11:17:30 +01:00
Arne Welzel
1e06c8bfda frameworks/notice: Handle fa_file with no or more than a single connection better
* When a file is transferred over multiple connection, have
  create_file_info() just pick the first one instead of none.

* Do not unconditionally assume cid and cuid as set on a
  Notice::FileInfo object.
2022-12-06 11:17:30 +01:00
Arne Welzel
d9b7da334d Merge branch 'topic/fox-ds/ssh-key-init-events' of github.com:fox-ds/zeek
* 'topic/fox-ds/ssh-key-init-events' of github.com:fox-ds/zeek:
  Added several events for detailed info on the SSH2 key init directions

* Straightened out the zeek:see lines in events.bif to be the same across all events.
2022-12-06 10:04:53 +01:00
Joost Jansen
bcdbca4bb9 Added several events for detailed info on the SSH2 key init directions 2022-12-05 12:35:05 +01:00
Arne Welzel
cd20766ed0 Updating CHANGES and VERSION. 2022-12-05 11:32:53 +01:00
Arne Welzel
56b23fc0b4 Merge branch 'topic/nadavk/krb_pa_ts' of github.com:nadavkluger/zeek
* 'topic/nadavk/krb_pa_ts' of github.com:nadavkluger/zeek:
  Expose PA_ENC_TIMESTAMP to script land
2022-12-05 10:33:53 +01:00
Christian Kreibich
01930b0b07 Update btest submodule [nomail] [skip ci] 2022-12-04 18:45:00 -08:00
Christian Kreibich
dbd5f4e1b7 Merge remote-tracking branch 'origin/topic/awelzel/2628-is-used-multiple'
* origin/topic/awelzel/2628-is-used-multiple:
  ID: Add AddAttr() helper
  Attr: Duplicated &is_used is allowed
2022-12-04 18:37:11 -08:00
Vern Paxson
52775bea89 maintenance updates for -O C++ 2022-12-04 17:59:14 -08:00
Vern Paxson
ab7cccf9b6 maintenance updates for -O ZAM 2022-12-04 17:58:33 -08:00
Vern Paxson
66c5bdbeac update test suite to avoid GH-2385 problems, including incorrect typing 2022-12-04 17:56:30 -08:00
Christian Kreibich
3c33728a2a Merge branch 'topic/awelzel/trailing-commas'
* topic/awelzel/trailing-commas:
  parse.y: Allow trailing commas for table, set, vector and record construction
2022-12-02 18:39:22 -08:00
Arne Welzel
da5fdb2072 ID: Add AddAttr() helper
This is just a follow-up as code cleanup, but not required
to be backported to 5.1.
2022-12-02 18:06:39 +01:00
Arne Welzel
ef920ef3f5 Attr: Duplicated &is_used is allowed
When using the same function for eval conditions in signatures, we
previously attempted to add &is_used multiple times to the function
triggering an ambiguous attribute error.

Turns out there's already a list of attributes that are accepted
to be used multiple times, so just add ATTR_IS_USED there.

Fixes #2628
2022-12-02 17:15:05 +01:00
nadavkluger
433b535e10 Expose PA_ENC_TIMESTAMP to script land 2022-12-02 15:43:19 +02:00
Arne Welzel
76ba9d4698 ContentLine: Fix spelling of "suppress", deprecate SupressWeirds()
Closes #2547
2022-12-02 12:40:47 +01:00
Johanna Amann
dbbb6cd6f0 Merge remote-tracking branch 'origin/topic/awelzel/alpine-317-musl-fts-dev'
* origin/topic/awelzel/alpine-317-musl-fts-dev:
  ci/alpine: Use musl-fts-dev rather than fts-dev
2022-12-01 13:30:39 +00:00
Arne Welzel
203a309612 parse.y: Allow trailing commas for table, set, vector and record construction
Python, Ruby, Javascript, Go, ..., allow use of trailing commas and is even
recommended in some style-guides as it keeps diffs smaller. The black formatter
for Python even goes as far to take a trailing comma as an indication to
format a list one-item on a line.

It has been a bit unusual to not be able to put trailing commas in Zeek
scripts, so this change allows for it.

It explicitly prevents trailing commas in list expressions on the left
hand side. Concretely, this disallows trailing commas in the key list
expression during table initializations.

It probably allows for commas in more places that I haven't fully grasped.
Maybe we should tighten those down again if we find them surprising.
2022-12-01 12:05:29 +01:00
Arne Welzel
c61fdc7866 ci/alpine: Use musl-fts-dev rather than fts-dev
With the release of alpine 3.17 (2022-11-22), the fts-dev package
was renamed to musl-fts-dev. Change it accordingly.

This is currently breaking CI in a fork of zeek/zeek. Suspect some
caching prevents it from breaking here currently.

Alpine 3.16 is still supported till 2024-05-23, but we're essentially
not doing CI testing for it anymore. Appears to be in line with the
comment in .cirrus.yml
2022-12-01 09:17:09 +01:00
zeek-bot
017525af71 Update doc submodule [nomail] [skip ci] 2022-12-01 00:35:52 +00:00
Christian Kreibich
1c381b5531 Merge branch 'topic/christian/gh-846-tcp-duration-docs'
* topic/christian/gh-846-tcp-duration-docs:
  Expand Conn::Info$duration comment to clarify TCP end-of-connection handling
2022-11-30 09:42:18 -08:00
Christian Kreibich
b0f96fa22c Expand Conn::Info$duration comment to clarify TCP end-of-connection handling
From Vern in GH-846: This is a conscious decision in the TCP analysis to
consider a connection's "duration" to run up through the end of its
productive (= data can be delivered) lifetime, not extending beyond that. So
once it's closed, packets seen subsequently (until the state-holding for the
connection times out) get processed in terms of updating the associated
history, but not the duration. This can include (unnecessarily) retransmitted
data packets, like in one of the examples above. An advantage of this definition
of "duration" is it allows more accurate computation of connection data rates.
2022-11-30 09:39:57 -08:00
Johanna Amann
d38e227a04 Merge remote-tracking branch 'origin/topic/awelzel/2583-mqtt-cluster-testing-follow-up'
* origin/topic/awelzel/2583-mqtt-cluster-testing-follow-up:
  testing/external: Bump cluster testsuite
2022-11-30 15:41:50 +00:00
Arne Welzel
6a98386820 testing/external: Bump cluster testsuite
...also add/propose aliases for sync-commits / sync-repos, because I could not
remember for the past 5 times or so in which direction they would sync and
would find myself reading the README, Makefile, or comments in the scripts.
2022-11-30 15:10:14 +01:00
Johanna Amann
12774ae757 Merge remote-tracking branch 'origin/topic/vern/script-opt-Nov22-maint2'
* origin/topic/vern/script-opt-Nov22-maint2:
  script optimization fixes:  new initialization model for standalone C++ scripts  type coercion fix  ZAM fix for compiling using C++ optimizer  disambiguate empty constructors
2022-11-30 13:51:56 +00:00
Johanna Amann
f4527ee4db Merge remote-tracking branch 'origin/topic/awelzel/2514-expire-all-timers-special-case'
* origin/topic/awelzel/2514-expire-all-timers-special-case:
  TimerMgr: Add back max_timer_expires=0 special case
  Add btest for expiration of all pending timers.
2022-11-30 13:45:09 +00:00
Johanna Amann
a7695bc784 Merge remote-tracking branch 'origin/topic/awelzel/2613-report-missing-intel-enum'
* origin/topic/awelzel/2613-report-missing-intel-enum:
  input: Add location info for invalid enums
2022-11-30 13:44:50 +00:00
Johanna Amann
3253168a53 Merge remote-tracking branch 'origin/topic/awelzel/2583-mqtt-to-base'
* origin/topic/awelzel/2583-mqtt-to-base:
  mqtt: Move from policy/ into base/
2022-11-30 13:44:27 +00:00
Johanna Amann
cb365d0ec5 Merge remote-tracking branch 'origin/topic/awelzel/2486-count-underflow'
* origin/topic/awelzel/2486-count-underflow:
  Expr: Warn on count underflow for c -= 1 and c = c - 1
  Reporter: Add ExprRuntimeWarning()
2022-11-30 13:43:37 +00:00
Arne Welzel
eb3bea4e4a mqtt: Move from policy/ into base/
Register dpd signatures and the analyzer when running in default mode.

Closes #2583
2022-11-30 10:14:20 +01:00
Arne Welzel
e48618e244 Expr: Warn on count underflow for c -= 1 and c = c - 1
I've skipped treating overflows as warnings, as ++ wrapping around at 0
doesn't currently trigger a runtime error and might be expected to be
quiet and silently wrap.

Closes #2486
2022-11-30 10:08:52 +01:00
zeek-bot
5c139ca786 Update doc submodule [nomail] [skip ci] 2022-11-30 00:31:02 +00:00
Christian Kreibich
f5ce597125 Merge remote-tracking branch 'origin/topic/awelzel/smb-follow-up-cleanup-six-years-later'
* origin/topic/awelzel/smb-follow-up-cleanup-six-years-later:
  smb: Drop references to uid_map in state.
  smb: Drop AUTH_LOG
2022-11-29 14:59:59 -08:00
Christian Kreibich
b4aaf2c1ef Merge branch 'topic/christian/ci-add-fedora-37'
* topic/christian/ci-add-fedora-37:
  CI: add Fedora 37
2022-11-29 12:12:47 -08:00
Arne Welzel
a07b0c333f Reporter: Add ExprRuntimeWarning()
...and update Expr.cc invalidation messages to use it. This aligns the
warning format to the one used by runtime errors.
2022-11-29 15:28:58 +01:00
Arne Welzel
e5d9a715ce input: Add location info for invalid enums
Fixes #2613
2022-11-29 12:36:11 +01:00
Christian Kreibich
17039c1298 Merge remote-tracking branch 'origin/topic/vern/vec-construct-compat'
* origin/topic/vern/vec-construct-compat:
  provide deprecated version of merge_type_list()
  keep merge_types() externally available; address reviewing suggestion
  change vector constructors to require direct type equivalence for non-arithmetics
2022-11-28 17:43:19 -08:00