Commit graph

16736 commits

Author SHA1 Message Date
Benjamin Bannier
9aba561e44 Make sure Spicy symbols are available.
The Spicy/HILTI runtime libraries are modelled as object libraries and
linked into the `spicy` and `hilti` libraries. These libraries are then
linked into `zeek_objs` which is another object library linked into the
`zeek` executable (target `zeek_exe`). On some platforms this transitive
link of object libraries is broken with `--binary-package` which causes
creation of static archives, even with our minimum required CMake
version which already contains many fixes for object libraries.

With this patch we now explicitly linked the Spicy/HILTI runtime
libraries into `zeek_exe`.

Closes #3177.
2024-01-19 12:26:44 +01:00
Christian Kreibich
6d10082cc3 Merge branch 'topic/christian/a-couple-of-nits'
* topic/christian/a-couple-of-nits:
  Fix a zeek_init -> zeek_done confusion in a docstring [skip ci]
  Fix typo in docstring [skip ci]
2024-01-18 16:17:21 -08:00
Christian Kreibich
3d95bd08ed Fix a zeek_init -> zeek_done confusion in a docstring [skip ci]
Thanks to Siavash Tahmureszadeh for flagging this one.
2024-01-18 16:14:33 -08:00
Christian Kreibich
ae2fd8f171 Fix typo in docstring [skip ci] 2024-01-18 16:14:27 -08:00
Christian Kreibich
2947bcda56 Default to setting ZEEK_HAVE_JAVASCRIPT=no in CMakeLists.txt
This still overrides it as appropriate when building with JS support, but shows
"JavaScript: no" in the configure output also when using
--disable-javascript, where it previously showed no output.
2024-01-18 15:52:19 -08:00
Christian Kreibich
b97907daed Show --disable-javascript in configure --help
This is already supported and just missing from the help output.
2024-01-18 15:52:19 -08:00
Christian Kreibich
e8007f04d9 Show AF_PACKET support status in cmake output, and sort features 2024-01-18 15:52:19 -08:00
Christian Kreibich
8ae560e7c6 Add feature tests for AF_PACKET, GeoIP, and JavaScript to zeek-config 2024-01-18 15:52:15 -08:00
zeek-bot
3b7749b8de Update doc submodule [nomail] [skip ci] 2024-01-18 00:13:18 +00:00
Arne Welzel
2a858d252e MIME: Cap nested MIME analysis depth to 100
OSS-Fuzz managed to produce a MIME multipart message construction with
thousands of nested entities (or that's what Zeek makes out of it anyhow).
Prevent such deep analysis by capping at a nesting depth of 100,
preventing unnecessary resource usage. A new weird named exceeded_mime_max_depth
is reported when this limit is reached.

This change reduces the runtime of the OSS-Fuzz reproducer from ~45 seconds
to ~2.5 seconds.

The test PCAP was produced from a Python script using the email package
and sending the rendered version via POST to a HTTP server.

Closes #208
2024-01-17 10:18:13 -07:00
Tim Wojtulewicz
cea7c473ac Merge remote-tracking branch 'origin/topic/timw/security-darwin-builds'
* origin/topic/timw/security-darwin-builds:
  Enable darwin builds for zeek-security repo
2024-01-17 10:00:21 -07:00
Tim Wojtulewicz
c4edd8410e Enable darwin builds for zeek-security repo 2024-01-17 09:59:49 -07:00
zeek-bot
bf3e4c5291 Update doc submodule [nomail] [skip ci] 2024-01-17 00:12:17 +00:00
Tim Wojtulewicz
9033018509 Drop support for subscribing to bro/ topics 2024-01-16 16:16:01 -07:00
Tim Wojtulewicz
1649e3e7cc Merge remote-tracking branch 'origin/topic/timw/ends-with-rework'
* origin/topic/timw/ends-with-rework:
  Squeeze a bit more performance out of the ends_with bif
2024-01-16 12:07:25 -07:00
Tim Wojtulewicz
b962bd30ce Squeeze a bit more performance out of the ends_with bif 2024-01-16 12:06:08 -07:00
Arne Welzel
378f380b71 Merge remote-tracking branch 'origin/topic/awelzel/smtp-bdat-follow-up'
* origin/topic/awelzel/smtp-bdat-follow-up:
  SMTP: No state update for bad BDAT commands
  SMTP/BDAT: Harden BDAT argument parsing a bit
2024-01-16 18:04:51 +01:00
Johanna Amann
273731e1ce Merge branch 'topic/johanna/fix-logging-of-ssl-log-ext-in-some-cases'
* topic/johanna/fix-logging-of-ssl-log-ext-in-some-cases:
  Fix ssl-log-ext omitting data in some cases
2024-01-16 13:10:35 +00:00
Arne Welzel
ae2a5c83a4 SMTP: No state update for bad BDAT commands
OSS-Fuzz found that providing an invalid BDAT line would tickle an
assert in UpdateState(). The BDAT state was never initialized, but
within UpdateState() that was expected.

This also removes the AnalyzerViolation() call for bad BDAT commands
and instead raises a weird. The SMTP analyzer is very lax and not triggering
the violation allows to parse the server's response to such an invalid
command.

PCAP files produced by a custom Python SMTP client against Postfix.
2024-01-15 18:25:41 +01:00
Arne Welzel
5ad11e00e3 SMTP/BDAT: Harden BDAT argument parsing a bit
If the size is followed by something and it's not " LAST", treat it
as an error.
2024-01-15 18:25:38 +01:00
Arne Welzel
c375610917 Merge remote-tracking branch 'origin/topic/awelzel/conn-session-history-lift-fixup'
* origin/topic/awelzel/conn-session-history-lift-fixup:
  Session/Conn: Follow-up fix for hist_seen and history lift
2024-01-15 15:18:48 +01:00
Arne Welzel
ec7c02a695 Merge remote-tracking branch 'origin/topic/vern/script-opt-maint.Jan24'
* origin/topic/vern/script-opt-maint.Jan24:
  ZAM speedup for constructing empty vectors
  fixes for ZAM optimization of "switch" statements
  BTests to catch regressions for recent ZAM fixes
  "-a zam" BTest baseline update for recent changes
  fix for needing to always flush optimization information for identifiers
  fix for logic bug in ldap base script
  better name for key variable in script optimization
  ZAM fix for tracking variable usage
  ZAM fixes for "for" loops that are only used to choose an element from a table/set
  ZAM fixes for loops indexed with variables not used in the loop body
  fix for ZAM location tracking - more extensive changes are pending
  fixes for ZAM's special-casing of that "cat" BiF
  some fixes for ZAM memory management
  streamlining of some script optimization APIs
  fixes for initializations of "-O gen-C++" script compilations
  script optimization fixes for "concretizing" vector-of-any's
2024-01-15 15:18:16 +01:00
Vern Paxson
96f5de8df8 ZAM speedup for constructing empty vectors 2024-01-15 15:03:56 +01:00
Vern Paxson
4bd0a46c29 fixes for ZAM optimization of "switch" statements 2024-01-15 15:03:56 +01:00
Vern Paxson
0dc587d1ad BTests to catch regressions for recent ZAM fixes 2024-01-15 15:03:56 +01:00
Vern Paxson
a26f7ac81b "-a zam" BTest baseline update for recent changes 2024-01-15 15:03:56 +01:00
Vern Paxson
71a1aa0afd fix for needing to always flush optimization information for identifiers 2024-01-15 15:03:56 +01:00
Vern Paxson
501bf167c3 fix for logic bug in ldap base script 2024-01-15 15:03:56 +01:00
Vern Paxson
24e182f080 better name for key variable in script optimization 2024-01-15 15:03:56 +01:00
Vern Paxson
cb4900847d ZAM fix for tracking variable usage 2024-01-15 15:03:56 +01:00
Vern Paxson
6660738b7d ZAM fixes for "for" loops that are only used to choose an element from a table/set 2024-01-15 15:03:56 +01:00
Vern Paxson
d1dffd3e1b ZAM fixes for loops indexed with variables not used in the loop body 2024-01-15 15:03:56 +01:00
Vern Paxson
e0736afe4c fix for ZAM location tracking - more extensive changes are pending 2024-01-15 15:03:56 +01:00
Vern Paxson
dc78a94c78 fixes for ZAM's special-casing of that "cat" BiF 2024-01-15 15:03:56 +01:00
Vern Paxson
eac764595c some fixes for ZAM memory management 2024-01-15 15:03:56 +01:00
Vern Paxson
896238c173 streamlining of some script optimization APIs 2024-01-15 15:03:56 +01:00
Vern Paxson
f7b739a47d fixes for initializations of "-O gen-C++" script compilations 2024-01-15 15:03:56 +01:00
Vern Paxson
bae87fb606 script optimization fixes for "concretizing" vector-of-any's 2024-01-15 15:03:56 +01:00
zeek-bot
fc12ac2b06 Update doc submodule [nomail] [skip ci] 2024-01-13 00:13:56 +00:00
Arne Welzel
9a922d88b2 Session/Conn: Follow-up fix for hist_seen and history lift
Borked in commit b4e86f28b8.

We don't need to initialize std::string explicitly either.

Thanks coverity!
2024-01-12 12:24:27 +01:00
Arne Welzel
2182ec03b3 Merge remote-tracking branch 'origin/topic/awelzel/3264-smtp-bdat'
* origin/topic/awelzel/3264-smtp-bdat:
  btest/smtp: Test with smtp-bdat-pipeline-8bitmime.pcap
  SMTP: Add BDAT support
2024-01-12 10:49:28 +01:00
Arne Welzel
00e7977732 btest/smtp: Test with smtp-bdat-pipeline-8bitmime.pcap
Not sure about the origin of this pcap, so adding it in a separate
commit, but it seems a nice real-world test case.
2024-01-12 10:18:14 +01:00
Arne Welzel
14949941ce SMTP: Add BDAT support
Closes #3264
2024-01-12 10:18:07 +01:00
Arne Welzel
ffffd88bef Merge remote-tracking branch 'origin/topic/christian/mmdb-configurability'
* origin/topic/christian/mmdb-configurability:
  Modernize various C++/Zeek-isms in the MMDB code.
  Fix MMDB code to re-open explicitly opened DBs correctly
  Add btest to verify behavior of re-opened MMDBs opened directly via BIFs
  Simplify MMDB code by moving more lookup functionality into MMDB class
  Move MMDB logic out of mmdb.bif and into MMDB.cc/h.
  Fix mmdb.temporary-error testcase when MMDBs are installed on system
  Adapt MMDB BiF code to new script-layer variables
  Update btest baselines to reflect introduction of mmdb.bif
  Move MaxMind/GeoIP BiF functionality into separate file
  Provide script-level configurability of MaxMind DB placement on disk
  Sort toplevel .bif list in CMakeLists
2024-01-12 09:28:36 +01:00
Christian Kreibich
dbad072f06 Modernize various C++/Zeek-isms in the MMDB code. 2024-01-11 17:13:41 -08:00
zeek-bot
5c61d1e551 Update doc submodule [nomail] [skip ci] 2024-01-12 00:20:45 +00:00
Arne Welzel
2ce4823c7a Merge remote-tracking branch 'origin/topic/awelzel/3540-known-hosts-expire-time'
* origin/topic/awelzel/3540-known-hosts-expire-time:
  Known: Keep &create_expire on local tables/sets valid
2024-01-11 20:18:22 +01:00
Johanna Amann
46e54bab76 Fix ssl-log-ext omitting data in some cases
ssl-log-ext had a bug that caused data present in the SSL connection to
not be logged in some cases. Specifically, the script relied on the base
ssl script to initialize some data structures; however, this means that
protocol messages that arrive before a message is handled by the base
ssl script are not logged.

This commit changes the ssl-log-ext script to also initialize the data
structures; now messages are correctly included in the log in all cases.
2024-01-11 17:12:33 +00:00
Arne Welzel
28b33b5c0d Merge remote-tracking branch 'origin/topic/neverlord/backward-compatibility'
* origin/topic/neverlord/backward-compatibility:
  Document upcoming breaking change for OpaqueVal
  Integrate review feedback
  Fix formatting
  Backward compatibility for OpaqueVal serialization
2024-01-11 12:38:39 +01:00
Arne Welzel
1ba0d4e31c Merge remote-tracking branch 'origin/topic/awelzel/3439-bump-tunnel-max-depth'
* origin/topic/awelzel/3439-bump-tunnel-max-depth:
  NEWS: Update news for tunnel depth changes
  tunnels: Add 'X' to history when reaching Tunnel::max_depth
  Session/TCP/UDP: Reserve HIST_UNKNOWN_PKT mask
  Conn: Deprecated AppendAddl
  Conn/Session: Lift history logic into Session
  init-bare: Default Tunnel::max_depth to 4
2024-01-11 11:03:15 +01:00