Commit graph

526 commits

Author SHA1 Message Date
Tim Wojtulewicz
a8fc63e182 Merge remote-tracking branch 'microsoft/master'
* microsoft/master: (71 commits)
  Clang formatting
  Mask ports before inserting them into the map
  Fix compiler warning from applied patch
  Remove statistics plugin in favor of stats bif
  Add EventHandler version of stats plugin
  Mark a few EventHandler methods const
  Changed implementation from std::map to std::unordered_map of Val.cc
  Removed const, Windows build is now working
  Added fixes suggested in PR
  Update src/packet_analysis/protocol/ip/IP.cc
  Apply suggestions from code review
  Clang format again but now with v13.0.1
  Rewrote usages of define(_MSC_VER) to ifdef _MSC_VER
  Clang format it all
  Fixed initial CR comments
  Add NEWS entry about Windows port
  Add a couple of extra unistd.h includes to fix a build failure
  Use std::chrono instead of gettimeofday
  Update libkqueue submodule [nomail]
  Don't call tokenize_string if the input string is empty
  ...
2022-11-11 15:23:21 -07:00
Tim Wojtulewicz
2739275b88 Merge remote-tracking branch 'jsoref/spelling-src'
* jsoref/spelling-src:
  Spelling src
2022-11-11 12:49:15 -07:00
Josh Soref
cd201aa24e Spelling src
These are non-functional changes.

* accounting
* activation
* actual
* added
* addresult
* aggregable
* aligned
* alternatively
* ambiguous
* analysis
* analyzer
* anticlimactic
* apparently
* application
* appropriate
* arithmetic
* assignment
* assigns
* associated
* authentication
* authoritative
* barrier
* boundary
* broccoli
* buffering
* caching
* called
* canonicalized
* capturing
* certificates
* ciphersuite
* columns
* communication
* comparison
* comparisons
* compilation
* component
* concatenating
* concatenation
* connection
* convenience
* correctly
* corresponding
* could
* counting
* data
* declared
* decryption
* defining
* dependent
* deprecated
* detached
* dictionary
* directional
* directly
* directory
* discarding
* disconnecting
* distinguishes
* documentation
* elsewhere
* emitted
* empty
* endianness
* endpoint
* enumerator
* essentially
* evaluated
* everything
* exactly
* execute
* explicit
* expressions
* facilitates
* fiddling
* filesystem
* flag
* flagged
* for
* fragments
* guarantee
* guaranteed
* happen
* happening
* hemisphere
* identifier
* identifies
* identify
* implementation
* implemented
* implementing
* including
* inconsistency
* indeterminate
* indices
* individual
* information
* initial
* initialization
* initialize
* initialized
* initializes
* instantiate
* instantiated
* instantiates
* interface
* internal
* interpreted
* interpreter
* into
* it
* iterators
* length
* likely
* log
* longer
* mainly
* mark
* maximum
* message
* minimum
* module
* must
* name
* namespace
* necessary
* nonexistent
* not
* notifications
* notifier
* number
* objects
* occurred
* operations
* original
* otherwise
* output
* overridden
* override
* overriding
* overwriting
* ownership
* parameters
* particular
* payload
* persistent
* potential
* precision
* preexisting
* preservation
* preserved
* primarily
* probably
* procedure
* proceed
* process
* processed
* processes
* processing
* propagate
* propagated
* prototype
* provides
* publishing
* purposes
* queue
* reached
* reason
* reassem
* reassemble
* reassembler
* recommend
* record
* reduction
* reference
* regularly
* representation
* request
* reserved
* retrieve
* returning
* separate
* should
* shouldn't
* significant
* signing
* simplified
* simultaneously
* single
* somebody
* sources
* specific
* specification
* specified
* specifies
* specify
* statement
* subdirectories
* succeeded
* successful
* successfully
* supplied
* synchronization
* tag
* temporarily
* terminating
* that
* the
* transmitted
* true
* truncated
* try
* understand
* unescaped
* unforwarding
* unknown
* unknowndata
* unspecified
* update
* usually
* which
* wildcard

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>
2022-11-09 12:08:15 -05:00
Elad Solomon
3a80b79497 Compile Zeek with MSVC
Allow Zeek to be embedded in another project
2022-11-09 18:15:30 +02:00
Arne Welzel
fa956efa79 file_analysis/Analyzer: Limit maximum number of violations
Just the same as for protocol analyzers, prevent from violation event overload.
2022-11-08 16:44:39 -07:00
Arne Welzel
d5cd023dff file_analysis: Implement AnalyzerViolation() for file_analysis/Analyzer
Add a test parsing a malformed PE file showing that analyzer_violation_info
is raised with the fa_file object set.

It could be interesting to pass through an optional connection if one
exists, but access is provided through f$conns, too.
2022-09-27 17:49:58 +02:00
Arne Welzel
8d19fa23ef Remove unified2 file analyzer 2022-08-19 14:05:00 +02:00
Tim Wojtulewicz
47e7fe2cd1 Convert Dictionary types to be templated classes
This has the fortunate side-effect of also making it so we can store
the value objects as typed pointers, instead of void*.
2022-07-05 13:33:34 -07:00
Tim Wojtulewicz
392c2f3810 Remove deprecated IterCookie-based dictionary iteration 2022-06-30 19:06:01 +00:00
Tim Wojtulewicz
d73f9a78dd Remove unnecessary uses of util::fmt() 2022-05-16 13:49:04 -07:00
Tim Wojtulewicz
b850d1dc51 Store hash analyzer kind StringVals for later reuse 2022-05-10 15:15:41 -07:00
Tim Wojtulewicz
a680c2faf0 Add Component::GetCanonicalNameVal to avoid extra StringVal allocations 2022-05-10 14:17:23 -07:00
Robin Sommer
5f1f005142
Merge remote-tracking branch 'origin/topic/neverlord/hide-caf'
* origin/topic/neverlord/hide-caf:
  Fix GCC builds and string output for Broker errors
  Update to latest Broker without public CAF dep
2022-02-07 16:25:22 +01:00
Vern Paxson
d758585e42 updated Bro->Zeek in comments in the source tree 2022-01-24 14:26:20 -08:00
Dominik Charousset
56f30b500a Update to latest Broker without public CAF dep 2021-12-20 08:16:21 +01:00
Tim Wojtulewicz
acb5e5dd26 Remove separate Tag types, note breaking change in NEWS 2021-12-06 11:25:13 -07:00
Tim Wojtulewicz
328411e807 GH-1873: Deprecate the tag types differently to avoid type clashes 2021-12-01 09:43:28 -07:00
Tim Wojtulewicz
a7d3cb48ef Add concept of "parent" tag namespaces
This allows us to create an EnumType that groups all of the analyzer
tag values into a single type, while still having the existing types
that split them up. We can then use this for certain events that benefit
from taking all of the tag types at once.
2021-11-23 19:36:49 -07:00
Tim Wojtulewicz
7d66f4252f Unify plugin::Component and plugin::TaggedComponent into a single class
These two are almost always used in conjunction with each other, and
TaggedComponent is never used by itself. Combining them together into
a single class will help simplify some of the code around managing
the mapping between Tags and Components.
2021-11-23 19:36:49 -07:00
Tim Wojtulewicz
8b544d648d Remove uses of deprecated Tag types 2021-11-23 19:36:49 -07:00
Tim Wojtulewicz
331161138a Unify all of the Tag types into one type
- Remove tag types for each component type (analyzer, etc)
- Add deprecated versions of the old types
- Remove unnecessary tag element from templates for TaggedComponent and ComponentManager
- Enable TaggedComponent to pass an EnumType when initializing Tag objects
- Update some tests that are affected by the tag enum values changing order
2021-11-23 19:36:49 -07:00
Johanna Amann
ff0bc86021 OpenSSL 3: normalize self-signed-cert error message
The message changed between OpenSSL 1.1 and 3.0 - we just use the old
one to not mess up the external test baselines.
2021-11-17 17:21:21 +00:00
Johanna Amann
253d214126 OpenSSL 3: fix warnings and tests
This commit fixes the compile-time warnings that OpenSSL 3.0 raises for
our source-code. For the cases where this was necessary we now have two
implementations - one for OpenSSL 1.1 and earlier, and one for OpenSSL
3.0.

This also makes our testsuite pass with OpenSSL 3.0

Relates to GH-1379
2021-11-17 13:33:49 +00:00
Tim Wojtulewicz
64748edab1 Replace most uses of typedef with using for type aliasing 2021-10-11 14:51:10 -07:00
Tim Wojtulewicz
9af6b2f48d clang-format: Set penalty for breaking after assignment operator 2021-09-27 10:49:48 -07:00
Tim Wojtulewicz
4423574d26 clang-format: Set IndentCaseBlocks to false 2021-09-27 10:49:48 -07:00
Tim Wojtulewicz
9cb54f5d44 clang-format: Force zeek-config.h to be earlier in the config ordering 2021-09-25 11:52:55 -07:00
Tim Wojtulewicz
b2f171ec69 Reformat the world 2021-09-16 15:35:39 -07:00
Vlad Grigorescu
8d2a6f48e8 docs: escape * in documentation 2021-07-08 18:26:42 -05:00
Johanna Amann
43ef7d43fb X509: add check if function succeeds
Addresses feedback of GH-1634
2021-07-05 09:40:04 +01:00
Johanna Amann
4ad5d2031b Fix memory leak in x509_check_cert_hostname bif 2021-06-30 10:16:38 +01:00
Johanna Amann
e4b2fa50a9 Merge remote-tracking branch 'origin/master' into topic/johanna/gh-859 2021-06-29 15:09:56 +01:00
Johanna Amann
833168090a Add ability to check if hostname is valid for a specific cert
This commit adds two new bifs, x509_check_hostname and
x509_check_cert_hostname. These bifs can be used to check if a given
hostname which can, e.g., be sent in a SNI is valid for a specific
certificate.

This PR furthermore modifies the ssl logs again, and adds information
about this to the log-file. Furthermore we now by default remove the
server certificate information from ssl.log - I doubt that this is often
looked at, it is not present in TLS 1.3, we do still have the SNI, and
if you need it you have the information in x509.log.

This also fixes a small potential problem in X509.cc assuming there
might be SAN-entries that contain null-bytes.

Baseline update will follow in another commit.
2021-06-29 15:00:48 +01:00
Robin Sommer
355ecc0c43 Extend the file analyzer API to set source manually.
The API didn't offer methods for passing in data that allow setting
all of offset, source and MIME type. Adding overloads for that (no
breakage to existing APIs).
2021-05-25 13:33:25 +02:00
Tim Wojtulewicz
63fd1611ca Rename some connection-specific methods in Session and SessionManager 2021-04-29 10:59:08 -07:00
Tim Wojtulewicz
f45df63cd0 Merge remote-tracking branch 'origin/topic/vern/zval'
* origin/topic/vern/zval: (42 commits)
  whitespace tweaks
  resolved some TODO comments
  remove unnecessary casts, and change necessary ones to use static_cast<>
  explain cmp_func default
  change functions for ZVal type management to static members
  fix some unsigned/signed integer warnings
  address lint concern about uninitialized variable
  Remove use of obsolete forward-declaration macros
  fix #include's that lack zeek/ prefixes
  explicitly populate holes created in vectors
  fixes for now-incorrect assumption that GetField always returns an existing ValPtr
  memory management for assignment to vector elements
  memory management for assignment to record fields
  destructor cleanup from ZAM_vector/ZAM_record
  fix #include's that lack zeek/ prefixes
  overlooked another way in which vector holes can be created
  initialize vector holes to the correct corresponding type
  explicitly populate holes created in vectors
  fix other instances of GetField().get() assuming long-lived ValPtr's
  fix for now-incorrect assumption that GetField always returns an existing ValPtr
  ...
2021-03-23 20:44:19 -07:00
Vern Paxson
d5b523cd2c whitespace tweaks 2021-03-22 21:23:15 -07:00
Vern Paxson
245108e86e remove unnecessary casts, and change necessary ones to use static_cast<> 2021-03-18 13:24:25 -07:00
Jon Siwek
1c58a2d86b GH-1432: Use buffered IO for file extraction
This can improve performance significantly: ~3.5x faster when tested on
a large file passing data to the file analysis framework in small chunks
of 20 bytes.
2021-03-09 18:26:48 -08:00
Vern Paxson
2f47cce8a6 fixes for now-incorrect assumption that GetField always returns an existing ValPtr 2021-02-28 07:23:06 -08:00
Vern Paxson
62bab66114 migration to using new differentiated methods for setting record fields 2021-02-25 16:59:26 -08:00
Vern Paxson
0118b6ee38 migrate to differentiated vector "At" accessors to support future efficiency 2021-02-25 13:28:23 -08:00
Vern Paxson
6121e409d3 convert RecordVal's to use vector<ZVal> as internal representation 2021-02-24 17:02:47 -08:00
Jon Siwek
39efbab3a4 Merge remote-tracking branch 'origin/topic/timw/remove-declaration-macros'
- Fixed forward decl. of ScriptInfo in zeekygen/Manager.h during merge

* origin/topic/timw/remove-declaration-macros:
  Remove obsolete ZEEK_FORWARD_DECLARE_NAMESPACED macros
2021-02-24 15:53:25 -08:00
Tim Wojtulewicz
4ad08172d0 Remove obsolete ZEEK_FORWARD_DECLARE_NAMESPACED macros 2021-02-24 14:35:44 -07:00
Jon Siwek
737d2c390b Support explicit disabling of file analyzers 2021-02-23 15:50:18 -08:00
Jon Siwek
69d73f7c83 Fix major_subsys_version field in pe_optional_header event
It was incorrectly set the same as the `minor_subsys_version` field
of the `PE::OptionalHeader` record.
2021-02-11 21:14:34 -08:00
Jon Siwek
c44cbe1feb Prefix #includes of .bif.h files with zeek/
This enables locating the headers within the install-tree using the
dirs provided by `zeek-config --include_dir`.

To enable locating these headers within the build-tree, this change also
creates a 'build/src/include/zeek -> ..' symlink.
2021-02-02 19:15:05 -08:00
Jon Siwek
8a8a983c49 Add missing zeek/ to header includes
Related to https://github.com/zeek/zeek/pull/1377
2021-01-29 19:16:29 -08:00
Tim Wojtulewicz
0618be792f Remove all of the random single-file deprecations
These are the changes that don't require a ton of changes to other files outside
of the original removal.
2021-01-27 10:52:40 -07:00