Commit graph

15378 commits

Author SHA1 Message Date
nadavkluger
433b535e10 Expose PA_ENC_TIMESTAMP to script land 2022-12-02 15:43:19 +02:00
Arne Welzel
76ba9d4698 ContentLine: Fix spelling of "suppress", deprecate SupressWeirds()
Closes #2547
2022-12-02 12:40:47 +01:00
Johanna Amann
dbbb6cd6f0 Merge remote-tracking branch 'origin/topic/awelzel/alpine-317-musl-fts-dev'
* origin/topic/awelzel/alpine-317-musl-fts-dev:
  ci/alpine: Use musl-fts-dev rather than fts-dev
2022-12-01 13:30:39 +00:00
Arne Welzel
203a309612 parse.y: Allow trailing commas for table, set, vector and record construction
Python, Ruby, Javascript, Go, ..., allow use of trailing commas and is even
recommended in some style-guides as it keeps diffs smaller. The black formatter
for Python even goes as far to take a trailing comma as an indication to
format a list one-item on a line.

It has been a bit unusual to not be able to put trailing commas in Zeek
scripts, so this change allows for it.

It explicitly prevents trailing commas in list expressions on the left
hand side. Concretely, this disallows trailing commas in the key list
expression during table initializations.

It probably allows for commas in more places that I haven't fully grasped.
Maybe we should tighten those down again if we find them surprising.
2022-12-01 12:05:29 +01:00
Arne Welzel
c61fdc7866 ci/alpine: Use musl-fts-dev rather than fts-dev
With the release of alpine 3.17 (2022-11-22), the fts-dev package
was renamed to musl-fts-dev. Change it accordingly.

This is currently breaking CI in a fork of zeek/zeek. Suspect some
caching prevents it from breaking here currently.

Alpine 3.16 is still supported till 2024-05-23, but we're essentially
not doing CI testing for it anymore. Appears to be in line with the
comment in .cirrus.yml
2022-12-01 09:17:09 +01:00
zeek-bot
017525af71 Update doc submodule [nomail] [skip ci] 2022-12-01 00:35:52 +00:00
Christian Kreibich
1c381b5531 Merge branch 'topic/christian/gh-846-tcp-duration-docs'
* topic/christian/gh-846-tcp-duration-docs:
  Expand Conn::Info$duration comment to clarify TCP end-of-connection handling
2022-11-30 09:42:18 -08:00
Christian Kreibich
b0f96fa22c Expand Conn::Info$duration comment to clarify TCP end-of-connection handling
From Vern in GH-846: This is a conscious decision in the TCP analysis to
consider a connection's "duration" to run up through the end of its
productive (= data can be delivered) lifetime, not extending beyond that. So
once it's closed, packets seen subsequently (until the state-holding for the
connection times out) get processed in terms of updating the associated
history, but not the duration. This can include (unnecessarily) retransmitted
data packets, like in one of the examples above. An advantage of this definition
of "duration" is it allows more accurate computation of connection data rates.
2022-11-30 09:39:57 -08:00
Johanna Amann
d38e227a04 Merge remote-tracking branch 'origin/topic/awelzel/2583-mqtt-cluster-testing-follow-up'
* origin/topic/awelzel/2583-mqtt-cluster-testing-follow-up:
  testing/external: Bump cluster testsuite
2022-11-30 15:41:50 +00:00
Arne Welzel
6a98386820 testing/external: Bump cluster testsuite
...also add/propose aliases for sync-commits / sync-repos, because I could not
remember for the past 5 times or so in which direction they would sync and
would find myself reading the README, Makefile, or comments in the scripts.
2022-11-30 15:10:14 +01:00
Johanna Amann
12774ae757 Merge remote-tracking branch 'origin/topic/vern/script-opt-Nov22-maint2'
* origin/topic/vern/script-opt-Nov22-maint2:
  script optimization fixes:  new initialization model for standalone C++ scripts  type coercion fix  ZAM fix for compiling using C++ optimizer  disambiguate empty constructors
2022-11-30 13:51:56 +00:00
Johanna Amann
f4527ee4db Merge remote-tracking branch 'origin/topic/awelzel/2514-expire-all-timers-special-case'
* origin/topic/awelzel/2514-expire-all-timers-special-case:
  TimerMgr: Add back max_timer_expires=0 special case
  Add btest for expiration of all pending timers.
2022-11-30 13:45:09 +00:00
Johanna Amann
a7695bc784 Merge remote-tracking branch 'origin/topic/awelzel/2613-report-missing-intel-enum'
* origin/topic/awelzel/2613-report-missing-intel-enum:
  input: Add location info for invalid enums
2022-11-30 13:44:50 +00:00
Johanna Amann
3253168a53 Merge remote-tracking branch 'origin/topic/awelzel/2583-mqtt-to-base'
* origin/topic/awelzel/2583-mqtt-to-base:
  mqtt: Move from policy/ into base/
2022-11-30 13:44:27 +00:00
Johanna Amann
cb365d0ec5 Merge remote-tracking branch 'origin/topic/awelzel/2486-count-underflow'
* origin/topic/awelzel/2486-count-underflow:
  Expr: Warn on count underflow for c -= 1 and c = c - 1
  Reporter: Add ExprRuntimeWarning()
2022-11-30 13:43:37 +00:00
Arne Welzel
eb3bea4e4a mqtt: Move from policy/ into base/
Register dpd signatures and the analyzer when running in default mode.

Closes #2583
2022-11-30 10:14:20 +01:00
Arne Welzel
e48618e244 Expr: Warn on count underflow for c -= 1 and c = c - 1
I've skipped treating overflows as warnings, as ++ wrapping around at 0
doesn't currently trigger a runtime error and might be expected to be
quiet and silently wrap.

Closes #2486
2022-11-30 10:08:52 +01:00
zeek-bot
5c139ca786 Update doc submodule [nomail] [skip ci] 2022-11-30 00:31:02 +00:00
Christian Kreibich
f5ce597125 Merge remote-tracking branch 'origin/topic/awelzel/smb-follow-up-cleanup-six-years-later'
* origin/topic/awelzel/smb-follow-up-cleanup-six-years-later:
  smb: Drop references to uid_map in state.
  smb: Drop AUTH_LOG
2022-11-29 14:59:59 -08:00
Christian Kreibich
b4aaf2c1ef Merge branch 'topic/christian/ci-add-fedora-37'
* topic/christian/ci-add-fedora-37:
  CI: add Fedora 37
2022-11-29 12:12:47 -08:00
Arne Welzel
a07b0c333f Reporter: Add ExprRuntimeWarning()
...and update Expr.cc invalidation messages to use it. This aligns the
warning format to the one used by runtime errors.
2022-11-29 15:28:58 +01:00
Arne Welzel
e5d9a715ce input: Add location info for invalid enums
Fixes #2613
2022-11-29 12:36:11 +01:00
Christian Kreibich
17039c1298 Merge remote-tracking branch 'origin/topic/vern/vec-construct-compat'
* origin/topic/vern/vec-construct-compat:
  provide deprecated version of merge_type_list()
  keep merge_types() externally available; address reviewing suggestion
  change vector constructors to require direct type equivalence for non-arithmetics
2022-11-28 17:43:19 -08:00
zeek-bot
f6297effa7 Update doc submodule [nomail] [skip ci] 2022-11-29 00:15:49 +00:00
Vern Paxson
732faa998f provide deprecated version of merge_type_list() 2022-11-28 10:00:11 -08:00
Christian Kreibich
417ff10a0a CI: add Fedora 37 2022-11-27 19:33:52 -08:00
Arne Welzel
2becb1337f TimerMgr: Add back max_timer_expires=0 special case
Commit 58fae22708 removed the max_expire==0
handling from DoAdvance() due to not being obvious what use it is. Jan
later reported that it broke the `redef max_timer_expires=0` (#2514).

This commit adds back the special case re-introducing the `max_timer_expires=0` ,
trying to make it fairly explicit that it exists.

This is an adaption of #2516 not adding a new option and trying a bit
to avoid global variable accesses down in DoAdvance(), though that
just moved to InitPostScript().

Fixes #2514.
2022-11-27 15:02:14 +01:00
Jan Grashoefer
8cdc3e4374 Add btest for expiration of all pending timers. 2022-11-27 15:02:09 +01:00
zeek-bot
0e97c29eb8 Update doc submodule [nomail] [skip ci] 2022-11-24 00:38:34 +00:00
Vern Paxson
c4973706e0 keep merge_types() externally available; address reviewing suggestion 2022-11-23 11:32:27 -08:00
Arne Welzel
8698a00f03 smb: Drop references to uid_map in state.
This isn't ever written to and probably was meant to be removed during
the following commit: 5b5589e167
2022-11-23 18:19:53 +01:00
Arne Welzel
b04f378f0f smb: Drop AUTH_LOG
This is never used and probably should've been removed
with 143eee5d8d
2022-11-23 18:18:20 +01:00
Arne Welzel
4718010388 zeekygen/normalize_script_path: Special case plugin dirnames without _
In normal installations, a plugin's basename has an underscore in it
to separate the namespace from the plugin name. E.g Zeek_Spicy. When
there is no underscore, this is most likely due to ./build being
picked up when using ZEEK_PLUGIN_PATH. The basename ends-up "build"
and is susceptible to collisions.

Prepend one parent directory as a heuristic to make this scenario less
likely, assuming ./build is usually below a repository checkout that
uniquely identifies the plugin.

Fixes #2577
2022-11-23 11:29:10 +01:00
Christian Kreibich
0a760c50b7 Merge remote-tracking branch 'origin/topic/vern/add-to-remove-from-coercions'
* origin/topic/vern/add-to-remove-from-coercions:
  Type coercion fix: transform +=/-= operators with arithmetic targets to explict assignments
2022-11-22 21:16:03 -08:00
Christian Kreibich
7c5f3c94b7 Merge remote-tracking branch 'origin/topic/awelzel/zeekygen-declaring-script'
* origin/topic/awelzel/zeekygen-declaring-script:
  zeekygen.bif: Drop using namespace zeekygen
  zeekygen: Add BIFs to access declaring scripts
2022-11-22 15:49:38 -08:00
Vern Paxson
baf2a91a85 Type coercion fix: transform +=/-= operators with arithmetic targets to explict assignments 2022-11-22 13:01:10 -08:00
Tim Wojtulewicz
ee11a4d5d2 Update broker submodule [nomail] 2022-11-22 13:22:45 -07:00
Tim Wojtulewicz
743d42f211 Merge remote-tracking branch 'security/topic/timw/120-ipv6-options-negative-length'
* security/topic/timw/120-ipv6-options-negative-length:
  Add additional length checking to IPv6::ToVal
2022-11-22 12:28:01 -07:00
Tim Wojtulewicz
9e8833e2d5 Merge remote-tracking branch 'security/topic/awelzel/121-ftp-timeout-again'
* security/topic/awelzel/121-ftp-timeout-again:
  ftp: Introduce FTP::max_command_length
2022-11-22 12:27:37 -07:00
Tim Wojtulewicz
eb3fb68fcc Merge remote-tracking branch 'security/topic/awelzel/119-http-timeout-tspacerr'
* security/topic/awelzel/119-http-timeout-tspacerr:
  http: Heuristic around rejecting malformed HTTP/0.9 traffic
2022-11-22 12:27:10 -07:00
Arne Welzel
4aecbbda49 zeekygen.bif: Drop using namespace zeekygen
I might be missing something, but not sure why there are there.
2022-11-22 13:23:15 +01:00
Arne Welzel
ef166649bf zeekygen: Add BIFs to access declaring scripts
Two new BIFs get_identifier_declaring_script() and get_record_field_declaring_script()
mirroring existing functions for comments. This allows to query the declaring
script information from Zeek scripts and further determine if a redef operations
were involved for record fields or enum names by comparing the declaring script
with the one of the involved type. See the tests.

Yet another one of @stevesmoot's requests.
2022-11-22 13:23:01 +01:00
zeek-bot
e7c6d17156 Update doc submodule [nomail] [skip ci] 2022-11-22 00:33:27 +00:00
Vern Paxson
47152e38c4 change vector constructors to require direct type equivalence for non-arithmetics 2022-11-21 15:32:46 -08:00
Tim Wojtulewicz
c1e5389929 Merge remote-tracking branch 'dop/topic/dopheide/vfmt-buf_len'
* dop/topic/dopheide/vfmt-buf_len:
  Fix logic error in vfmt() when growing the buffer
2022-11-21 09:10:37 -07:00
Tim Wojtulewicz
26030f4a57 Merge remote-tracking branch 'nadav/topic/nadavk/ntlm'
* nadav/topic/nadavk/ntlm:
  Added NTLM challenge and response
2022-11-21 09:09:18 -07:00
Arne Welzel
3f5cb75a2a ftp: Introduce FTP::max_command_length
oss-fuzz produced FTP traffic with a ~550KB long FTP command. Cap FTP command
length at 100 bytes, log a weird if a command is larger than that and move
on to the next. Likely it's not actual FTP traffic, but raising an
analyzer violation would allow clients an easy way to disable the analyzer
by sending an overly long command.

The added test PCAP was generated using a fake Python socket server/client.
2022-11-21 09:36:29 +01:00
Vern Paxson
134f8f2ef5 script optimization fixes:
new initialization model for standalone C++ scripts
 type coercion fix
 ZAM fix for compiling using C++ optimizer
 disambiguate empty constructors
2022-11-20 12:16:25 -08:00
zeek-bot
dbb2aa88a6 Update doc submodule [nomail] [skip ci] 2022-11-19 00:36:18 +00:00
Michael Dopheide
e47b055769 Fix logic error in vfmt() when growing the buffer 2022-11-18 17:46:03 +00:00