Commit graph

14920 commits

Author SHA1 Message Date
Arne Welzel
630db664a8 Merge remote-tracking branch 'origin/topic/awelzel/2998-ntp-flip-roles'
* origin/topic/awelzel/2998-ntp-flip-roles:
  NTP: Detect out-of-order packets
2023-05-04 20:08:50 +02:00
Tim Wojtulewicz
44b7e91f87 Merge remote-tracking branch 'origin/topic/neverlord/gh-2711'
* origin/topic/neverlord/gh-2711:
  Integrate review feedback
  Include compiler in --show-config output
  Fix CMake option defaults on Windows
  Move build defaults from configure to CMake
2023-05-04 10:56:46 -07:00
Arne Welzel
2c8b97c522 NTP: Detect out-of-order packets
The NTP mode provides us with the identity of the endpoints. For the
simple CLIENT / SERVER modes, flip the connection if we detect
orig/resp disagreeing with what the message says. This mainly
results in the history getting a ^ and the ntp.log / conn.log
showing the corrected endpoints.

Closes #2998.
2023-05-04 19:44:02 +02:00
Arne Welzel
12252743b1 Merge remote-tracking branch 'origin/topic/awelzel/smb2-state-handling'
* origin/topic/awelzel/smb2-state-handling:
  NEWS: Add entry about SMB::max_pending_messages and state discarding
  scripts/smb2-main: Reset script-level state upon smb2_discarded_messages_state()
  smb2: Limit per-connection read/ioctl/tree state
2023-05-04 09:40:19 +02:00
zeek-bot
8e49c87912 Update doc submodule [nomail] [skip ci] 2023-05-04 00:24:39 +00:00
Tim Wojtulewicz
8eed46dfa1 Merge remote-tracking branch 'origin/topic/johanna/update-ssl-databases'
* origin/topic/johanna/update-ssl-databases:
  Update Mozilla CA and Google CT lists
2023-05-03 11:13:23 -07:00
Tim Wojtulewicz
87636947a6 Merge remote-tracking branch 'origin/topic/timw/zeek-config-quoted-strings'
* origin/topic/timw/zeek-config-quoted-strings:
  Surround string assignments in zeek-config in quotes
2023-05-03 09:31:44 -07:00
Tim Wojtulewicz
7305f2ba9b Merge remote-tracking branch 'origin/topic/timw/2997-ding-dong-bro-init-is-dead'
* origin/topic/timw/2997-ding-dong-bro-init-is-dead:
  Remove parser error message for bro_init, et al
2023-05-03 09:31:26 -07:00
Tim Wojtulewicz
bc82b657c4 Merge remote-tracking branch 'origin/topic/timw/ieee-80211-length-check'
* origin/topic/timw/ieee-80211-length-check:
  Add additional length check to IEEE 802.11 analyzer
2023-05-03 09:31:08 -07:00
Johanna Amann
bef1d51015 Update SQLite to 3.41.2
[nomail]
2023-05-03 14:22:08 +01:00
Johanna Amann
9742d9a76e Merge branch 'topic/johanna/no-error-message-durning-tls-or-dtls-protocol-violations'
* topic/johanna/no-error-message-durning-tls-or-dtls-protocol-violations:
  SSL: failing analyzer handling - address review feedback
  SSL: do not try to disable failed analyzer

Also folds in minor feedback from GH-3012
2023-05-03 14:20:44 +01:00
Johanna Amann
9a47e201f8 SSL: failing analyzer handling - address review feedback
Fold the two analyzer_violation_info events into one. See GH-3012
2023-05-03 13:41:36 +01:00
Johanna Amann
21888a145a SSL: do not try to disable failed analyzer
Currently, if a TLS/DTLS analyzer fails with a protocol violation, we
will still try to remove the analyzer later, which results in the
following error message:

error: connection does not have analyzer specified to disable

Now, instead we don't try removing the analyzer anymore, after a
violation occurred.
2023-05-03 11:16:14 +01:00
Johanna Amann
40a4da94b6 Update Mozilla CA and Google CT lists 2023-05-03 10:46:41 +01:00
Arne Welzel
042aa1383b NEWS: Add entry about SMB::max_pending_messages and state discarding 2023-05-03 11:22:01 +02:00
Arne Welzel
3ac877e20d scripts/smb2-main: Reset script-level state upon smb2_discarded_messages_state()
This is similar to what the external corelight/zeek-smb-clear-state script
does, but leverages the smb2_discarded_messages_state() event instead of
regularly checking on the state of SMB connections.

The pcap was created using the dperson/samba container image and mounting
a share with Linux's CIFS filesystem, then copying the content of a
directory with 100 files. The test uses a BPF filter to imitate mostly
"half-duplex" traffic.
2023-05-03 11:22:01 +02:00
Arne Welzel
5caab1a667 smb2: Limit per-connection read/ioctl/tree state
Users on Slack observed memory growth in an environment with a lot of
SMB traffic. jeprof memory profiling pointed at the offset and fid maps
kept per-connection for smb2 read requests.

These maps can grow unbounded if responses are seen before requests, there's
packet drops, just one side of the connection is visible, or we fail to parse
responses properly.

Forcefully wipe out these maps when they grow too large and raise
smb2_discarded_messages_state() to notify script land about this.
2023-05-03 10:58:59 +02:00
zeek-bot
cc25129b2f Update doc submodule [nomail] [skip ci] 2023-05-03 00:48:40 +00:00
Tim Wojtulewicz
7782f73251 Add additional length check to IEEE 802.11 analyzer 2023-05-02 16:12:43 -07:00
Dominik Charousset
94ec8167b2
Integrate review feedback
Co-authored-by: Tim Wojtulewicz <timwoj@gmail.com>
2023-05-02 21:24:53 +02:00
Tim Wojtulewicz
e07f3f305d Remove parser error message for bro_init, et al 2023-05-02 12:06:41 -07:00
Arne Welzel
df0788252e Merge remote-tracking branch 'origin/topic/awelzel/2994-ip-mf-df-offset'
* origin/topic/awelzel/2994-ip-mf-df-offset:
  ip4_hdr: Add DF, MF, offset and sum fields
2023-05-02 20:45:43 +02:00
Dominik Charousset
7b35d471ad Include compiler in --show-config output 2023-05-02 20:44:48 +02:00
Dominik Charousset
c2c34148ca Fix CMake option defaults on Windows 2023-05-02 20:43:33 +02:00
Arne Welzel
5570f806ca Merge remote-tracking branch 'origin/topic/awelzel/2991-unload-zeek-package'
* origin/topic/awelzel/2991-unload-zeek-package:
  unload: Fix unloading of packages
2023-05-02 20:28:07 +02:00
Tim Wojtulewicz
86839aee34 Surround string assignments in zeek-config in quotes 2023-05-02 11:24:40 -07:00
Tim Wojtulewicz
c02940e4f3 Merge remote-tracking branch 'origin/topic/timw/windows-setvbuf'
* origin/topic/timw/windows-setvbuf:
  Use workaround for setvbuf on Windows in DebugLogger/Extract file analyzer
  Add ifdef'd implementation of setvbuf to zeek::util
2023-05-02 11:13:03 -07:00
Tim Wojtulewicz
29b0380815 Use workaround for setvbuf on Windows in DebugLogger/Extract file analyzer 2023-05-02 10:28:50 -07:00
Tim Wojtulewicz
f2a3e23dfa Add ifdef'd implementation of setvbuf to zeek::util 2023-05-02 10:28:50 -07:00
Tim Wojtulewicz
71731ffcb1 Merge remote-tracking branch 'origin/topic/timw/smith-waterman-compare-semantics'
* origin/topic/timw/smith-waterman-compare-semantics:
  Fix smith-waterman sorting to follow correct Compare semantics
2023-05-02 09:42:33 -07:00
Tim Wojtulewicz
81b94423a0 Fix smith-waterman sorting to follow correct Compare semantics 2023-04-30 11:54:08 -07:00
Arne Welzel
df4f90b65c ip4_hdr: Add DF, MF, offset and sum fields
For low-level packet analysis use-cases, these fields are currently
not script-land accessible via raw_packet() or so. They are accessible
on the icmp_context record, but not on the actual ip4_hdr record, so
add them.
2023-04-29 17:09:43 +02:00
Dominik Charousset
7af3611807 Move build defaults from configure to CMake
Moving the defaults for build variables from the `configure` script to
`CMakeLists.txt` gives the same default behavior on platforms where the
`configure` script is not available (Windows) and also allows a pure
CMake-based work flow (e.g., the standard `cmake -S . -B build`) without
having to manually adjust the defaults.

The `configure` script also becomes much simpler as a result.
2023-04-29 11:04:30 +02:00
zeek-bot
3a172bde38 Update doc submodule [nomail] [skip ci] 2023-04-29 00:24:30 +00:00
Tim Wojtulewicz
a7c9454224 Merge remote-tracking branch 'origin/topic/timw/2980-cirrus-updates'
* origin/topic/timw/2980-cirrus-updates:
  CI: Automatically update opensuse-tumbleweed
  CI: Remove Fedora 36, add Fedora 38
2023-04-28 16:33:56 -07:00
Tim Wojtulewicz
15a39c230c CI: Automatically update opensuse-tumbleweed 2023-04-28 16:33:11 -07:00
Tim Wojtulewicz
64bcf50837 CI: Remove Fedora 36, add Fedora 38 2023-04-28 16:33:11 -07:00
Arne Welzel
b677f5b37b unload: Fix unloading of packages
@ynadji found that unloading packages doesn't work due to @unload not
resolving the __load__.zeek file within a directory like @load does.

Fixes #2991
2023-04-28 11:47:55 +02:00
Arne Welzel
f227b30d30 Merge remote-tracking branch 'origin/topic/awelzel/2791-pcap-stat-overflow'
* origin/topic/awelzel/2791-pcap-stat-overflow:
  pcap/Source: Allow more than 32bit for link and dropped stats
2023-04-28 10:02:30 +02:00
Arne Welzel
3c646b5bf6 Merge remote-tracking branch 'origin/topic/awelzel/record-fields-optional'
* origin/topic/awelzel/record-fields-optional:
  record_fields: Include information about optionality of fields
2023-04-28 10:00:27 +02:00
Arne Welzel
440a7e7ccf Merge remote-tracking branch 'origin/topic/timw/compiler-warnings'
* origin/topic/timw/compiler-warnings:
  Fix a few warnings from recent changes
2023-04-28 10:00:00 +02:00
Arne Welzel
5541066660 pcap/Source: Allow more than 32bit for link and dropped stats
The PktSrc::Stats object works with 64bit unsigned integers. Unfortunately,
libpcap's struct pcap_stat is using 32bit values and users have reported
the wrapping of these values being visible in their stats.log roughly every
7.5 hours (~160kpps).

This change moves tracking of link and drop counters into the PktSrc::Stats
object (like is done for received and bytes_received) and updates them
on a call to PcapSource::Statistics() with the difference to the
previous stats values to prevent the wrap from becoming visible to
script land.

This doesn't cover the case of the stats counters wrapping around multiple
times between two invocations of PktSrc::Statistics(). With the default
interval of 5 minutes for the stats script, this seems acceptable.

Closes #2791.
2023-04-28 09:59:26 +02:00
Arne Welzel
667cdd5c27 Merge remote-tracking branch 'jgras/topic/jgras/cluster-init-tests'
* jgras/topic/jgras/cluster-init-tests:
  Simplify btests using cluster_started event.
  Remove old cluster start tests.

Reverted removal of old cluster start tests during merge.
2023-04-28 09:37:25 +02:00
zeek-bot
605e749133 Update doc submodule [nomail] [skip ci] 2023-04-28 00:28:42 +00:00
Tim Wojtulewicz
eb9d851a07 Fix a few warnings from recent changes 2023-04-27 15:38:02 -07:00
Arne Welzel
f4bb8fae33 record_fields: Include information about optionality of fields
This was reported as a wish for log schema generation, so add it...
2023-04-27 21:18:35 +02:00
Tim Wojtulewicz
f3f139deab Merge remote-tracking branch 'origin/topic/timw/windows-ci-unit-testing'
* origin/topic/timw/windows-ci-unit-testing:
  Modify Windows test cmd file to actually run tests
  Add template file and cmake call for zeek-path setup on Windows
2023-04-27 11:26:00 -07:00
Arne Welzel
0374d0fe42 Update zeek-aux submodule [nomail] 2023-04-27 15:17:11 +02:00
Arne Welzel
a3ef8783b1 Merge remote-tracking branch 'origin/topic/neverlord/gh-2806'
* origin/topic/neverlord/gh-2806:
  test/builtin-plugins: Add plugin using ZEEK_VERSION_NUMBER
  Revert putting plugins into a fresh scope for now
  Propagate zeek-version.h skip via CMake properties
2023-04-27 13:23:14 +02:00
Arne Welzel
004dce2cf2 Merge remote-tracking branch 'origin/topic/awelzel/zeekctl-multiple-loggers'
* origin/topic/awelzel/zeekctl-multiple-loggers:
  NEWS: Add entry for ZeekControl and multi-loggers
  Bump zeekctl to multi-logger version
  logging: Support rotation_postprocessor_command_env
2023-04-27 12:17:02 +02:00