Commit graph

1825 commits

Author SHA1 Message Date
Jon Siwek
02476453bc Merge remote-tracking branch 'origin/topic/jsiwek/fix-coverity-build'
* origin/topic/jsiwek/fix-coverity-build:
  Remove inline from some static KeyedHash members
  Improve Func.h inclusion
2020-06-08 11:23:24 -07:00
Jon Siwek
ef6bd3ee39 Fix use-after-move of proc_status_file breaking -U flag 2020-06-04 15:13:10 -07:00
Tim Wojtulewicz
b4aa5d9456 Merge remote-tracking branch 'origin/topic/jsiwek/gh-973-fix-topk-get-top-return-type'
* origin/topic/jsiwek/gh-973-fix-topk-get-top-return-type:
  GH-973: Fix the return type of topk_get_top()
2020-06-03 13:29:31 -07:00
Tim Wojtulewicz
640ad2d010 Merge remote-tracking branch 'origin/topic/jsiwek/double-val-subclasses'
* origin/topic/jsiwek/double-val-subclasses:
  Deprecate Val(double, TypeTag) ctor, add TimeVal/DoubleVal subclasses
2020-06-03 13:20:04 -07:00
Jon Siwek
a431f6b45d Merge remote-tracking branch 'origin/topic/jsiwek/gh-985-double-to-interval-descs'
* origin/topic/jsiwek/gh-985-double-to-interval-descs:
  GH-985: Fix descriptions of double_to_interval() return values
2020-06-02 18:14:20 -07:00
Jon Siwek
7b0931058e Decrease number of CPUs/memory for Cirrus CI tasks
New concurrency limits are 16 Linux CPUs per user rather than 8 Linux
Tasks per user (at a max of 8 CPUs per Task, they essentially allowed 64
CPUs per user before the change in policy).
2020-06-02 12:40:26 -07:00
Jon Siwek
e1f35c46f9 Change CI script to compile from build/Makefile
Helps some `make` versions properly use the `-j` option
2020-06-02 10:56:48 -07:00
Jon Siwek
1a60fb7c0d Add missing include in util.cc 2020-06-01 20:45:37 -07:00
Jon Siwek
34a1875e74 Merge remote-tracking branch 'origin/topic/timw/reduce-func-inclusion'
- Minor tweaks to some odd includes during merge

* origin/topic/timw/reduce-func-inclusion:
  Remove Analyzer.h from bro-bif.h
  Remove IPAddr.h from Reporter.h
  Remove the inclusion of Func.h from NetVar.h, which reduces the inclusion of Func.h overall.
2020-06-01 19:26:55 -07:00
Tim Wojtulewicz
503ef26a17 Merge remote-tracking branch 'origin/topic/jsiwek/gh-893-intrusive-ptr-migration'
* origin/topic/jsiwek/gh-893-intrusive-ptr-migration: (151 commits)
  Integrate review feedback
  Switch Broker Val converter visitor to return IntrusivePtr
  Change BroFunc ctor to take const-ref IntrusivePtr<ID>
  Add version of Frame::SetElement() taking IntrusivePtr<ID>
  Change Scope/Func inits from id_list* to vector<IntrusivePtr<ID>>
  Change Scope::GenerateTemporary() to return IntrusivePtr
  Deprecate Scope::ReturnType(), replace with GetReturnType()
  Deprecate Scope::ScopeID(), replace with GetID()
  Switch parsing to use vector<IntrusivePtr<Attr>> from attr_list
  Deprecate TableVal::FindAttr(), replace with GetAttr()
  Deprecate TypeDecl::FindAttr(), replace with GetAttr()
  Deprecate ID::FindAttr(), replace with GetAttr()
  Deprecate Attributes::FindAttr(), replace with Find()
  Deprecate Attributes::AddAttrs(Attributes*)
  Add Attributes ctor that takes IntrusivePtrs
  Change Attributes to store std:vector<IntrusivePtr<Attr>>
  Change Attr::SetAttrExpr() to non-template
  Deprecate Attr::AttrExpr(), replace with GetExpr()
  Deprecate ID::Attrs(), replace with GetAttrs()
  Remove weak_ref param from ID::SetVal()
  ...
2020-06-01 10:58:02 -07:00
Jon Siwek
2f918ed9b2 Merge branch 'topic/dopheide/known-services' of https://github.com/dopheide-esnet/zeek
- Updated the logic significantly: still filters out ICMP from being
  considered an active service (like before) and adds a new
  "Known::service_udp_requires_response" option (defaults to true) for
  whether to require UDP server response before being considered an
  active service.

* 'topic/dopheide/known-services' of https://github.com/dopheide-esnet/zeek:
  Log services with unknown protocols
2020-05-29 17:19:47 -07:00
Jon Siwek
138c9402c3 Update NEWS 2020-05-28 14:51:08 -07:00
Johanna Amann
433e1154da Merge branch 'add_bzar_dce_rpc_consts' of https://github.com/ct-square/zeek
* 'add_bzar_dce_rpc_consts' of https://github.com/ct-square/zeek:
  Remove dupplicate DCE-RPC endpoint
  Add DCE-RPC constants from BZAR project

Closes GH-953
2020-05-26 22:04:33 +00:00
Jon Siwek
c171af47f5 GH-979: Update libkqueue to fix use on CentOS 6 2020-05-26 11:07:40 -07:00
Tim Wojtulewicz
94ef9f4542 Merge remote-tracking branch 'origin/topic/jsiwek/gh-983-broker-opaque-missing-type'
* origin/topic/jsiwek/gh-983-broker-opaque-missing-type:
  GH-983: Fix opaque Broker types lacking a Type after (de)serialization
2020-05-21 13:05:57 -07:00
Tim Wojtulewicz
9283b1c2b2 Merge remote-tracking branch 'origin/topic/johanna/threading-allow-SendEvent-from-stuff-that-is-not-an-input-reader-because-it-turns-out-that-it-is-actually-kind-of-convenient-to-call-this-from-other-things-like-for-example-from-logging-writers'
* origin/topic/johanna/threading-allow-SendEvent-from-stuff-that-is-not-an-input-reader-because-it-turns-out-that-it-is-actually-kind-of-convenient-to-call-this-from-other-things-like-for-example-from-logging-writers:
  Make SendEvent callable from all threads
2020-05-21 13:04:44 -07:00
Jon Siwek
05f829c727 Merge remote-tracking branch 'origin/topic/jazoff/fuzz-size-limit-speedup'
- Factored check into more descriptive function: ExceedsChunkLimit()

* origin/topic/jazoff/fuzz-size-limit-speedup:
  Speed up ChunkCount validity check
2020-05-21 11:43:34 -07:00
Jon Siwek
7d98d816d8 Disable output of Reporter messages to stderr in fuzz targets 2020-05-20 11:20:59 -07:00
Jon Siwek
6d0599c537 Improve standalone fuzz driver timing output 2020-05-20 10:57:18 -07:00
Jon Siwek
c372d53d77 Merge remote-tracking branch 'origin/topic/neverlord/caf-0.18'
* origin/topic/neverlord/caf-0.18:
  Upgrade to latest Broker changes for CAF 0.18
2020-05-19 10:50:58 -07:00
Jon Siwek
2bac702a72 Merge remote-tracking branch 'origin/topic/jazoff/packet-fuzzer'
* origin/topic/jazoff/packet-fuzzer:
  Update src/fuzzers/packet-fuzzer.cc
  Update src/fuzzers/packet-fuzzer.cc
  add initial packet corpus
  add packet fuzzer
2020-05-18 10:48:22 -07:00
Johanna Amann
781c3e5914 Merge branch 'replace-bzero' of https://github.com/brimsec/zeek
* 'replace-bzero' of https://github.com/brimsec/zeek:
  Replace bzero() with memset()
2020-05-15 19:48:47 +00:00
Johanna Amann
2aeb3d8e39 Merge remote-tracking branch 'origin/topic/timw/906-find-all-urls-regex'
* origin/topic/timw/906-find-all-urls-regex:
  Restore previous url scheme capture group
  GH-906: Fix the regex in url.zeek to better match for find_all_urls
2020-05-13 15:05:54 -07:00
Tim Wojtulewicz
2b668a6c1f Merge remote-tracking branch 'origin/topic/johanna/hash-unification'
* origin/topic/johanna/hash-unification:
  Hash unification: address PR feedback
  Update architecture checks for highwayhash
  Highwayhas: unbreak 32 bit compiles
  Update external test suites for hash changes.
  Hashing: Remove unnecessary include
  Add highwayhash in a more cmake-y way
  Remove highwayhash submodule.
  Switch file UID hashing from md5 to highwayhash.
  Switch most internal md5 calls to digest calls.
  Start refactoring hashing.
  Introduce calculate_digest convenience function
  Remove the siphash->hmac-md5 switch after 36 bytes.
  Fix plugin compile problems with highwayhash
  Replace siphash with Google implementation
2020-05-13 13:49:56 -07:00
Jon Siwek
c55bbcfcda Merge remote-tracking branch 'origin/topic/timw/disable-broker-tests'
* origin/topic/timw/disable-broker-tests:
  Disable broker tests from building on the CI
  Disable broker documentation examples if disabling broker tests
2020-05-13 11:16:30 -07:00
Tim Wojtulewicz
82978a8860 Merge remote-tracking branch 'origin/topic/jsiwek/fuzzing'
* origin/topic/jsiwek/fuzzing:
  Use zeek::detail namespace for fuzzer utils
  Set terminating flag during fuzzer cleanup
  Add missing include to standalone fuzzer driver
  Improve standalone fuzzer driver error messages
  Test fuzzers against seed corpus under CI ASan build
  Update fuzzing README with OSS-Fuzz integration notes
  Link fuzzers against shared library to reduce executable sizes
  Improve FuzzBuffer chunking
  Fix compiler warning in standalone fuzzer driver
  Adjust minor fuzzing documentation
  Exit immediately after running unit tests
  Add OSS-Fuzz Zeek script search path to fuzzers
  Assume libFuzzer when LIB_FUZZING_ENGINE file doesn't exist
  Change handling of LIB_FUZZING_ENGINE
  Change --enable-fuzzing to --enable-fuzzers
  Add standalone driver for fuzz targets
  Add basic structure for fuzzing targets
2020-05-12 18:53:06 -07:00
Tim Wojtulewicz
f539723e8e Merge remote-tracking branch 'origin/topic/jsiwek/mmdb-msg-limit'
* origin/topic/jsiwek/mmdb-msg-limit:
  Limit rate at which MMDB error/status messages are emitted
2020-05-11 11:12:12 -07:00
Jon Siwek
73a57335db Merge remote-tracking branch 'origin/topic/timw/unordered-lists'
* origin/topic/timw/unordered-lists:
  Review fixups
  Change timer_list in BroList to be an unordered list.
  Add ability for List to be ordered/unordered
  Add unit testing for List
  Fix bug with List where replace() doesn't work with non-pointer types
2020-05-08 11:52:03 -07:00
Jon Siwek
b5531ecbd3 Merge branch 'set_to_regex-docs' of https://github.com/jlagermann/zeek
- Adjusted the formatting during merge

* 'set_to_regex-docs' of https://github.com/jlagermann/zeek:
  added examples to set_to_regex comments Signed-ff-by: James Lagermann <james.lagermann@corelight.com>
2020-05-08 11:48:44 -07:00
Jon Siwek
9babb7d39f Merge remote-tracking branch 'origin/topic/johanna/958'
- Extended the improved error to cover miss-typed identifiers, too

* origin/topic/johanna/958:
  Fix crash when trying to redef non-existing enum
2020-05-07 17:15:53 -07:00
Tim Wojtulewicz
8ac591c9f5 Revert addition of final modifier to JSON formatter 2020-05-06 10:40:48 -07:00
Jon Siwek
39c00c9966 Updating CHANGES and VERSION. 2020-05-06 10:04:47 -07:00
Jon Siwek
156686b237 Correct spelling of DCE/RPC operation string NetrLogonSameLogonWithFlags
Fixes GH-952
2020-05-04 18:03:14 -07:00
Jon Siwek
eff11c5ef4 Merge branch 'network-time-init' of https://github.com/J-Gras/zeek
- Minor whitespace/format adjustments

* 'network-time-init' of https://github.com/J-Gras/zeek:
  Fix documentation for network_time_init.
  Improve network_time_init() test and add baseline.
  Add network_time_init() event.
  Add test for network_time behavior.
2020-05-04 17:51:39 -07:00
Jon Siwek
62ec60b96f Merge remote-tracking branch 'origin/topic/jazoff/avoid-redundant-inactivity-timers'
- Changed the logic significantly to just guarantee there's only ever a
  single inactivity timer per connection

- Updated language.expire_subnet btest which is unduly sensitive to
  timer-related changes

* origin/topic/jazoff/avoid-redundant-inactivity-timers:
  avoid scheduling redundant inactivity timers
2020-05-04 17:12:36 -07:00
Jon Siwek
1fe5454603 Merge remote-tracking branch 'origin/topic/jsiwek/ci-libmaxminddb'
* origin/topic/jsiwek/ci-libmaxminddb:
  Extend CI config to cover building with libmaxminddb support
2020-05-01 17:46:43 -07:00
Tim Wojtulewicz
5377dd446d Merge remote-tracking branch 'origin/topic/jsiwek/gh-938-pcap-source-always-ready'
* origin/topic/jsiwek/gh-938-pcap-source-always-ready:
  GH-938: fix IO loop iterations sometimes skipping offline pcap sources
2020-05-01 09:45:00 -07:00
Jon Siwek
fda9498276 Merge remote-tracking branch 'origin/topic/seth/more-file-type-ident'
* origin/topic/seth/more-file-type-ident:
  Adds a note in NEWS about the signature addition and restructuring
  Fixes a small bug in one signature with a duplicate name.
  Organized and added to the shipped file identification signatures.
2020-04-29 16:32:41 -07:00
Jon Siwek
ccdaf5f111 Merge branch 'stats-logging-fix' of https://github.com/brittanydonowho/zeek
- Added test case and adjusted whitespace in merge

* 'stats-logging-fix' of https://github.com/brittanydonowho/zeek:
  Fixed stats.zeek to log all data before zeek terminates rather than return too soon
2020-04-29 15:58:35 -07:00
Jon Siwek
a08b1ff56f Add warning message for unknown Broker statuses
There's now a couple placeholder/unimplemented status values in Broker
related to upcoming routing features that we don't want to handle
explicitly for compatibility reasons, but also don't want the compiler
warning about unhandled values in the switch.
2020-04-29 15:28:18 -07:00
Jon Siwek
4a76229209 GH-941: Fix build when configured to use libmaxminddb 2020-04-28 17:12:51 -07:00
Jon Siwek
034b4b5ea7 Merge remote-tracking branch 'origin/topic/timw/intrusiveptr-warnings'
* origin/topic/timw/intrusiveptr-warnings:
  Fix a few more IntrusivePtr deprecation warnings
2020-04-27 13:38:45 -07:00
Johanna Amann
faa8a38578 Merge remote-tracking branch 'origin/topic/jsiwek/gh-854-preserve-header-name'
* origin/topic/jsiwek/gh-854-preserve-header-name:
  GH-854: provide access to original HTTP/MIME header names
2020-04-27 19:31:49 +00:00
Tim Wojtulewicz
13674e7c31 Merge remote-tracking branch 'origin/topic/jsiwek/bif-return-intrusive'
* origin/topic/jsiwek/bif-return-intrusive:
  Update various BIFs to return IntrusivePtr
2020-04-27 12:09:58 -07:00
Tim Wojtulewicz
cb40dbd58b Merge remote-tracking branch 'origin/topic/jsiwek/intrusive-ptr-chipping'
* origin/topic/jsiwek/intrusive-ptr-chipping:
  Deprecate returning Val* from BIFs
  Deprecate binpac::string_to_val
  Deprecate binpac::bytestring_to_val, replace with binpac::to_stringval
  Update deprecated BifEvent::generate_* usages
  Deprecate Connection::Event and Analyzer::Event methods
  Deprecate BuildConnVal() methods and update usages to ConnVal()
  Update all BIFs to return IntrusivePtr instead of Val*
  Update deprecated ValManager::GetPort usages
  Update deprecated ValManager::GetEmptyString usages
  Update deprecated ValManager::GetCount usages
  Update deprecated ValManager::GetInt usages
  Update deprecated ValManager::GetBool usages
  Update deprecated ValManager GetTrue/GetFalse usages
  Deprecate all ValManager "Get" methods
  Change BIFs to return a wrapper object
2020-04-24 16:20:08 -07:00
Jon Siwek
6e2cd3ae44 Merge branch 'ident_overflow' of https://github.com/MaxKellermann/zeek
* 'ident_overflow' of https://github.com/MaxKellermann/zeek:
  analyzer/protocol/ident: fix buffer overflow in ParsePort()
2020-04-22 10:44:43 -07:00
Jon Siwek
f849571910 Merge remote-tracking branch 'origin/topic/johanna/remove_connection_external'
* origin/topic/johanna/remove_connection_external:
  Remove connection_external
2020-04-21 10:26:07 -07:00
Jon Siwek
8843f69002 Remove ineffective &default in netcontrol cluster event handler args 2020-04-16 15:40:27 -07:00
Johanna Amann
730f78d0c2 Merge remote-tracking branch 'origin/topic/jsiwek/no-vla'
* origin/topic/jsiwek/no-vla:
  GH-895: Remove use of Variable-Length-Arrays

Fixes GH-895
2020-04-16 14:18:38 -07:00
Johanna Amann
df71d963c9 Merge remote-tracking branch 'origin/topic/jsiwek/krb-consts-defaults'
* origin/topic/jsiwek/krb-consts-defaults:
  Add default function for Kerberos constant-lookup-tables
2020-04-16 13:05:05 -07:00