Commit graph

6667 commits

Author SHA1 Message Date
Tim Wojtulewicz
9b15db25f6 Reorder fields in some classes for more compact memory layout 2021-07-14 14:59:49 -07:00
Tim Wojtulewicz
f849f024e5 Redo how reassembled flag is accessed in IP_Hdr, filling in a memory hole 2021-07-14 14:59:49 -07:00
Christian Kreibich
281eef398f Merge branch 'topic/vlad/doc_tweaks'
* topic/vlad/doc_tweaks:
  docs: escape * in documentation
  Update the mail_dest documentation with the new email_dest field
2021-07-09 12:02:32 -07:00
Tim Wojtulewicz
f8389c92e0 Merge remote-tracking branch 'origin/topic/timw/freebsd-warning'
* origin/topic/timw/freebsd-warning:
  Fix a compile warning on FreeBSD
2021-07-08 18:00:34 -07:00
Vlad Grigorescu
8d2a6f48e8 docs: escape * in documentation 2021-07-08 18:26:42 -05:00
Tim Wojtulewicz
12ac62645a Fix a compile warning on FreeBSD 2021-07-08 16:10:19 -07:00
Christian Kreibich
a3623bfb2d Establish a separate init script when using the supervisor
The supervisor does not require the full weight of scripts that
init-default.zeek brings with it. The new file, init-supervisor.zeek, contains
only what's required by the supervisor in addition to the other always-loaded
init files.
2021-07-08 13:12:53 -07:00
Christian Kreibich
7bee79b400 Add optional bare-mode boolean flag to Supervisor's node configuration
When omitted, the node inherits the Supervisor's bare-mode
status. When true/false, the new Zeek node will enable/disable bare
mode, respectively. It continues to load any scripts passed at the
command line and in the additional scripts list already provided in
the node configuration.

Includes testcase.
2021-07-08 13:12:53 -07:00
Christian Kreibich
36051dc9a1 Add support for setting environment variables via supervisor
The NodeConfig record now has a table for specifying environment variable names
and values, which the supervisor sets in the created node.

This also repositions the cpu_affinity member to keep the order the same in
the corresponding script-layer and in-core types.

Includes testcase.
2021-07-08 13:12:53 -07:00
Tim Wojtulewicz
caba3dc151 GH-1644: Pass a larger structure to be calloc'd in patricia.c 2021-07-07 08:56:25 -07:00
Tim Wojtulewicz
fcee137b1a Fix multiple spelling errors in debug output 2021-07-07 08:55:48 -07:00
Tim Wojtulewicz
cf0745da9f Merge remote-tracking branch 'origin/topic/seth/subproject-build'
* origin/topic/seth/subproject-build:
  Removing a fix from another branch
  Update binpac pointer
  Update bifcl and binpac for building Zeek as a submodule
  Updated the cmake pointer
  Fix the builtin plugin preload mechanism
  Update submodule pointer for cmake to point at the appropriate branch
  Switch to more accurate source and binary references in cmake
2021-07-06 08:59:22 -07:00
Tim Wojtulewicz
2021df7418 Merge remote-tracking branch 'origin/topic/timw/coverity-findings'
* origin/topic/timw/coverity-findings:
  Fix a number of Coverity findings
2021-07-06 08:49:58 -07:00
Johanna Amann
7ec50bf434 Merge remote-tracking branch 'origin/topic/johanna/gh-859'
* origin/topic/johanna/gh-859:
  Add X509/SSL changes to NEWS
  X509: add check if function succeeds
  GH-1634: Address feedback
  Small indentation fixes in ssl-log-ext.zeek
  Fix memory leak in x509_check_cert_hostname bif
  Small bugfix and updates for external test hashes (SSL/X509)
  Baseline updates for recent SSL changes.
  Add ability to check if hostname is valid for a specific cert
  Add ssl_history field to ssl.log
  Add policy script suppressing certificate events
  Add new ssl-log-ext policy script
  Deprecate extract-certs-pem.zeek and add log-certs-base64.zeek
  Implement X509 certificate log caching
  Deprecate ICSI SSL notary script.
  Change SSL and X.509 logging format
  Enable OCSP logging by default.
  Split the code that handles X509 event hashing into its own file

Closes GH-859
2021-07-05 10:12:46 +01:00
Johanna Amann
43ef7d43fb X509: add check if function succeeds
Addresses feedback of GH-1634
2021-07-05 09:40:04 +01:00
Christian Kreibich
795a7ea98e Add a global log policy hook to the logging framework
This addresses the need for a central hook on any log write, which
wasn't previously doable without a lot of effort. The log manager
invokes the new Log::log_stream_policy hook prior to any filter-specific
hooks. Like filter-level hooks, it may veto a log write. Even when
it does, filter-level hooks still get invoked, but cannot "un-veto".

Includes test cases.
2021-07-02 12:42:45 -07:00
Christian Kreibich
50c5968c30 Merge branch 'topic/christian/mallinfo2'
* topic/christian/mallinfo2:
  Use mallinfo2() instead of mallinfo() when available
2021-07-02 11:58:54 -07:00
Tim Wojtulewicz
5e00f78920 Fix a number of Coverity findings
- 1458048: Use-after-free in the SQLite logger
- 1457823: Missing a break statement in script-opt reduction
- 1453966: Dead code in CompHash
- 1445417: Unintialized variable in StaticHash64
- 1437716: Unintialized variables in FileInfo in scan.l
2021-07-02 11:18:19 -07:00
Dominik Charousset
fbd95973b4 Update Broker submodule 2021-07-02 13:17:56 +02:00
Christian Kreibich
63259ef9fa Use mallinfo2() instead of mallinfo() when available
glibc 2.33 deprecates mallinfo in favor of a struct that returns
its members as size_ts instead of ints.
2021-07-01 16:40:28 -07:00
Tim Wojtulewicz
0b342b7bfa Merge remote-tracking branch 'origin/topic/vern/CPP-workflow2'
* origin/topic/vern/CPP-workflow2:
  low-level coding style fixes
  support for standalone compiled scripts to export globals with module qualifiers
  updates for documentation of functionality for compiling scripts to C++
  fixes for standalone C++ scripts making types & variables/functions available
  fixed bug limiting availability of load_CPP() BiF
  updates to development helper scripts to support new workflow
  simpler workflow for -O gen-C++ ; also some hooks for -O gen-standalone-C++
  ReplaceBody now deletes a body if the replacement is nil
  removal of can't-actually-be-executed code
2021-07-01 08:46:53 -07:00
Christian Kreibich
937bdccab5 Skip input framework entries with missing but non-optional fields
The framework so far populated data structures with missing fields
even when those fields are defined without the &optional
attribute. When using the attribute, such entries continue to get
populated.

Update tests to reflect focus on unset fields.
2021-06-30 20:02:26 -07:00
Christian Kreibich
ef08605877 Fix segfault in input framework when reading unset fields
Unset fields could trigger unexpected null pointers in the input
manager. This also adds a warning such fields come up.
2021-06-30 20:02:25 -07:00
Tim Wojtulewicz
3c0bb8f1dd Merge remote-tracking branch 'origin/topic/seth/small-builtin-plugin-fixes'
* origin/topic/seth/small-builtin-plugin-fixes:
  Fix tests
  Removed a non-functional builtin plugin preload loading mechanism
  Load the builtin-plugin preload files and fix the dev path.
2021-06-30 14:11:10 -07:00
Seth Hall
c118d1c2f4 Removing a fix from another branch 2021-06-30 17:46:41 +00:00
Tim Wojtulewicz
0c8a594d6f GH-1500: Mark BasicThread::Done to be ignored by ThreadSanitizer
There's a known false positive with the atomic variables in this method
that triggers a complaint from ThreadSanitizer. Marking it as ignored
avoids the warning.
2021-06-30 08:50:14 -07:00
Johanna Amann
09229c58d7 Update submodule
This updates the bundled SQLite to 3.36.0.

 [nomail]
2021-06-30 13:19:19 +01:00
Johanna Amann
509b501e1b Merge remote-tracking branch 'origin/master' into topic/johanna/gh-859 2021-06-30 10:36:54 +01:00
Johanna Amann
4ad5d2031b Fix memory leak in x509_check_cert_hostname bif 2021-06-30 10:16:38 +01:00
Robin Sommer
40923fdd65 Merge remote-tracking branch 'origin/topic/robin/gh-1406-m1-tests'
* origin/topic/robin/gh-1406-m1-tests:
  Fix SMB tests on Apple M1.
2021-06-30 07:52:44 +02:00
Vern Paxson
30cb91fe35 low-level coding style fixes 2021-06-29 15:17:34 -07:00
Robin Sommer
369e42a6e4 Fix SMB tests on Apple M1.
Due to different double precision on M1, file IDs for SMB could end up
changing on M1 because the access time of a file goes into their
computation. The real solution for this would be changing Zeek's
internal "time" representation to uint64; that's planned, but requires
major surgery. For now, this PR changes the SMB code to also pass SMB's
original time representation (which is a uint64) into script-land, and
then use that for computing the file ID.

Closes #1406
2021-06-29 20:17:02 +02:00
Christian Kreibich
8942da72b5 Merge branch 'topic/christian/gh-1626-remove-idmef'
* topic/christian/gh-1626-remove-idmef:
  Remove remaining vestigial IDMEF support code
2021-06-29 11:15:20 -07:00
Vern Paxson
d1334b7bca made RecordVal::AppendField protected: it's low-level & requires knowledge of internals 2021-06-29 10:24:20 -07:00
Johanna Amann
e4b2fa50a9 Merge remote-tracking branch 'origin/master' into topic/johanna/gh-859 2021-06-29 15:09:56 +01:00
Johanna Amann
833168090a Add ability to check if hostname is valid for a specific cert
This commit adds two new bifs, x509_check_hostname and
x509_check_cert_hostname. These bifs can be used to check if a given
hostname which can, e.g., be sent in a SNI is valid for a specific
certificate.

This PR furthermore modifies the ssl logs again, and adds information
about this to the log-file. Furthermore we now by default remove the
server certificate information from ssl.log - I doubt that this is often
looked at, it is not present in TLS 1.3, we do still have the SNI, and
if you need it you have the information in x509.log.

This also fixes a small potential problem in X509.cc assuming there
might be SAN-entries that contain null-bytes.

Baseline update will follow in another commit.
2021-06-29 15:00:48 +01:00
Christian Kreibich
712a28a813 Remove remaining vestigial IDMEF support code 2021-06-28 15:18:16 -07:00
Tim Wojtulewicz
b14cd1ef16 GH-1216: Enable Mobile IPv6 support by default
This removes the ENABLE_MOBILE_IPV6 #define variable. It also marks the
--enable-mobile-ipv6 configure argument as deprecated.
2021-06-28 11:11:55 -07:00
Tim Wojtulewicz
a7fd34375f GH-572: Mark MemoryAllocation() and related methods deprecated 2021-06-28 11:07:58 -07:00
Tim Wojtulewicz
98a9ae9572 Check for -1 return from FieldOffset() in Val::HasField()
Fixes Coverity 1457804
2021-06-28 11:06:16 -07:00
Tim Wojtulewicz
67b45bc502 Merge remote-tracking branch 'origin/topic/vern/ZAM-prep'
* origin/topic/vern/ZAM-prep: (45 commits)
  whoops overlooked the need to canonicalize filenames
  another set of tweaks per review comments
  addressed a number of code review comments
  baseline updates for merge
  support "any" coercions for "-O gen-C++"
  better descriptions for named record constructors
  test suite baseline updates for "-a opt" optimize-AST alternative
  test suite baseline updates for "-a xform" alternative / AST transformation
  error propagation fix for AST reduction
  updates to "-a inline" test suite alternative baseline
  updates for the main test suite baseline
  updates to test suite tests for compatibility with upcoming ZAM functionality
  "-O compile-all" option to specify compilation of inlined functions
  compile inlined functions if they're also used indirectly
  provide ZAM-generated code with low-level access to record fields
  fix for cloning records with fields of type "any"
  direct access for ZAM to VectorVal internal vector
  ZVal constructors, accessors & methods in support of ZAM
  switch ZVal representation of types from Type objects to TypeVal's
  revised error-reporting interface for ZVal's, to accommodate ZAM inner loop
  ...
2021-06-28 11:03:13 -07:00
Tim Wojtulewicz
bea3a615b6 Merge remote-tracking branch 'origin/topic/christian/gh-1600'
* origin/topic/christian/gh-1600:
  Remove unnecessary -B arguments from Zeek invocations in testsuite
  Fix perftools-enabled build
  Minor tweaks to single-character command line option handling
2021-06-27 10:50:37 -07:00
Tim Wojtulewicz
c1b2989035 Merge remote-tracking branch 'htonl/weird_segfault'
* htonl/weird_segfault:
  Add btest test case
  Fix typo
  Fix segfault with incomplete connection
2021-06-27 10:46:30 -07:00
Florian Wilkens
946d74674a ssl/analyzer: potentially fix memory leaks caused by bytestrings 2021-06-25 17:23:06 +02:00
Florian Wilkens
ebea26a065 analyzer/ssl: several improvements
- use better data structures for secret and key material storage
- add documentation to the new methods in the analyzer
2021-06-25 12:34:45 +02:00
Florian Wilkens
aaaff39e12 analyzer/ssl: defensive key length check + more debug logging 2021-06-25 11:05:29 +02:00
Florian Wilkens
979bf20769 analyzer/ssl: handle missing <openssl/kdf.h> 2021-06-25 11:05:20 +02:00
Christian Kreibich
cd066f09bf Fix perftools-enabled build 2021-06-24 16:02:45 -07:00
Christian Kreibich
a2e5251796 Minor tweaks to single-character command line option handling
- Use of "-B" now triggers usage output and errors out when Zeek
  wasn't built with debugging support.

- Always keep the perftools options (-m, -M) in the optparse string,
  for consistency with other flags dependent on configuration. We
  still fall through to usage and erroring out as before.

- Minor indentation fix.
2021-06-24 16:02:45 -07:00
Seth Hall
6602b54d42 Fix the builtin plugin preload mechanism 2021-06-23 10:40:18 +00:00