Commit graph

1952 commits

Author SHA1 Message Date
Tim Wojtulewicz
2b668a6c1f Merge remote-tracking branch 'origin/topic/johanna/hash-unification'
* origin/topic/johanna/hash-unification:
  Hash unification: address PR feedback
  Update architecture checks for highwayhash
  Highwayhas: unbreak 32 bit compiles
  Update external test suites for hash changes.
  Hashing: Remove unnecessary include
  Add highwayhash in a more cmake-y way
  Remove highwayhash submodule.
  Switch file UID hashing from md5 to highwayhash.
  Switch most internal md5 calls to digest calls.
  Start refactoring hashing.
  Introduce calculate_digest convenience function
  Remove the siphash->hmac-md5 switch after 36 bytes.
  Fix plugin compile problems with highwayhash
  Replace siphash with Google implementation
2020-05-13 13:49:56 -07:00
Jon Siwek
c55bbcfcda Merge remote-tracking branch 'origin/topic/timw/disable-broker-tests'
* origin/topic/timw/disable-broker-tests:
  Disable broker tests from building on the CI
  Disable broker documentation examples if disabling broker tests
2020-05-13 11:16:30 -07:00
Tim Wojtulewicz
82978a8860 Merge remote-tracking branch 'origin/topic/jsiwek/fuzzing'
* origin/topic/jsiwek/fuzzing:
  Use zeek::detail namespace for fuzzer utils
  Set terminating flag during fuzzer cleanup
  Add missing include to standalone fuzzer driver
  Improve standalone fuzzer driver error messages
  Test fuzzers against seed corpus under CI ASan build
  Update fuzzing README with OSS-Fuzz integration notes
  Link fuzzers against shared library to reduce executable sizes
  Improve FuzzBuffer chunking
  Fix compiler warning in standalone fuzzer driver
  Adjust minor fuzzing documentation
  Exit immediately after running unit tests
  Add OSS-Fuzz Zeek script search path to fuzzers
  Assume libFuzzer when LIB_FUZZING_ENGINE file doesn't exist
  Change handling of LIB_FUZZING_ENGINE
  Change --enable-fuzzing to --enable-fuzzers
  Add standalone driver for fuzz targets
  Add basic structure for fuzzing targets
2020-05-12 18:53:06 -07:00
Tim Wojtulewicz
f539723e8e Merge remote-tracking branch 'origin/topic/jsiwek/mmdb-msg-limit'
* origin/topic/jsiwek/mmdb-msg-limit:
  Limit rate at which MMDB error/status messages are emitted
2020-05-11 11:12:12 -07:00
Jon Siwek
73a57335db Merge remote-tracking branch 'origin/topic/timw/unordered-lists'
* origin/topic/timw/unordered-lists:
  Review fixups
  Change timer_list in BroList to be an unordered list.
  Add ability for List to be ordered/unordered
  Add unit testing for List
  Fix bug with List where replace() doesn't work with non-pointer types
2020-05-08 11:52:03 -07:00
Jon Siwek
b5531ecbd3 Merge branch 'set_to_regex-docs' of https://github.com/jlagermann/zeek
- Adjusted the formatting during merge

* 'set_to_regex-docs' of https://github.com/jlagermann/zeek:
  added examples to set_to_regex comments Signed-ff-by: James Lagermann <james.lagermann@corelight.com>
2020-05-08 11:48:44 -07:00
Jon Siwek
9babb7d39f Merge remote-tracking branch 'origin/topic/johanna/958'
- Extended the improved error to cover miss-typed identifiers, too

* origin/topic/johanna/958:
  Fix crash when trying to redef non-existing enum
2020-05-07 17:15:53 -07:00
Tim Wojtulewicz
8ac591c9f5 Revert addition of final modifier to JSON formatter 2020-05-06 10:40:48 -07:00
Jon Siwek
39c00c9966 Updating CHANGES and VERSION. 2020-05-06 10:04:47 -07:00
Jon Siwek
156686b237 Correct spelling of DCE/RPC operation string NetrLogonSameLogonWithFlags
Fixes GH-952
2020-05-04 18:03:14 -07:00
Jon Siwek
eff11c5ef4 Merge branch 'network-time-init' of https://github.com/J-Gras/zeek
- Minor whitespace/format adjustments

* 'network-time-init' of https://github.com/J-Gras/zeek:
  Fix documentation for network_time_init.
  Improve network_time_init() test and add baseline.
  Add network_time_init() event.
  Add test for network_time behavior.
2020-05-04 17:51:39 -07:00
Jon Siwek
62ec60b96f Merge remote-tracking branch 'origin/topic/jazoff/avoid-redundant-inactivity-timers'
- Changed the logic significantly to just guarantee there's only ever a
  single inactivity timer per connection

- Updated language.expire_subnet btest which is unduly sensitive to
  timer-related changes

* origin/topic/jazoff/avoid-redundant-inactivity-timers:
  avoid scheduling redundant inactivity timers
2020-05-04 17:12:36 -07:00
Jon Siwek
1fe5454603 Merge remote-tracking branch 'origin/topic/jsiwek/ci-libmaxminddb'
* origin/topic/jsiwek/ci-libmaxminddb:
  Extend CI config to cover building with libmaxminddb support
2020-05-01 17:46:43 -07:00
Tim Wojtulewicz
5377dd446d Merge remote-tracking branch 'origin/topic/jsiwek/gh-938-pcap-source-always-ready'
* origin/topic/jsiwek/gh-938-pcap-source-always-ready:
  GH-938: fix IO loop iterations sometimes skipping offline pcap sources
2020-05-01 09:45:00 -07:00
Jon Siwek
fda9498276 Merge remote-tracking branch 'origin/topic/seth/more-file-type-ident'
* origin/topic/seth/more-file-type-ident:
  Adds a note in NEWS about the signature addition and restructuring
  Fixes a small bug in one signature with a duplicate name.
  Organized and added to the shipped file identification signatures.
2020-04-29 16:32:41 -07:00
Jon Siwek
ccdaf5f111 Merge branch 'stats-logging-fix' of https://github.com/brittanydonowho/zeek
- Added test case and adjusted whitespace in merge

* 'stats-logging-fix' of https://github.com/brittanydonowho/zeek:
  Fixed stats.zeek to log all data before zeek terminates rather than return too soon
2020-04-29 15:58:35 -07:00
Jon Siwek
a08b1ff56f Add warning message for unknown Broker statuses
There's now a couple placeholder/unimplemented status values in Broker
related to upcoming routing features that we don't want to handle
explicitly for compatibility reasons, but also don't want the compiler
warning about unhandled values in the switch.
2020-04-29 15:28:18 -07:00
Jon Siwek
4a76229209 GH-941: Fix build when configured to use libmaxminddb 2020-04-28 17:12:51 -07:00
Jon Siwek
034b4b5ea7 Merge remote-tracking branch 'origin/topic/timw/intrusiveptr-warnings'
* origin/topic/timw/intrusiveptr-warnings:
  Fix a few more IntrusivePtr deprecation warnings
2020-04-27 13:38:45 -07:00
Johanna Amann
faa8a38578 Merge remote-tracking branch 'origin/topic/jsiwek/gh-854-preserve-header-name'
* origin/topic/jsiwek/gh-854-preserve-header-name:
  GH-854: provide access to original HTTP/MIME header names
2020-04-27 19:31:49 +00:00
Tim Wojtulewicz
13674e7c31 Merge remote-tracking branch 'origin/topic/jsiwek/bif-return-intrusive'
* origin/topic/jsiwek/bif-return-intrusive:
  Update various BIFs to return IntrusivePtr
2020-04-27 12:09:58 -07:00
Tim Wojtulewicz
cb40dbd58b Merge remote-tracking branch 'origin/topic/jsiwek/intrusive-ptr-chipping'
* origin/topic/jsiwek/intrusive-ptr-chipping:
  Deprecate returning Val* from BIFs
  Deprecate binpac::string_to_val
  Deprecate binpac::bytestring_to_val, replace with binpac::to_stringval
  Update deprecated BifEvent::generate_* usages
  Deprecate Connection::Event and Analyzer::Event methods
  Deprecate BuildConnVal() methods and update usages to ConnVal()
  Update all BIFs to return IntrusivePtr instead of Val*
  Update deprecated ValManager::GetPort usages
  Update deprecated ValManager::GetEmptyString usages
  Update deprecated ValManager::GetCount usages
  Update deprecated ValManager::GetInt usages
  Update deprecated ValManager::GetBool usages
  Update deprecated ValManager GetTrue/GetFalse usages
  Deprecate all ValManager "Get" methods
  Change BIFs to return a wrapper object
2020-04-24 16:20:08 -07:00
Jon Siwek
6e2cd3ae44 Merge branch 'ident_overflow' of https://github.com/MaxKellermann/zeek
* 'ident_overflow' of https://github.com/MaxKellermann/zeek:
  analyzer/protocol/ident: fix buffer overflow in ParsePort()
2020-04-22 10:44:43 -07:00
Jon Siwek
f849571910 Merge remote-tracking branch 'origin/topic/johanna/remove_connection_external'
* origin/topic/johanna/remove_connection_external:
  Remove connection_external
2020-04-21 10:26:07 -07:00
Jon Siwek
8843f69002 Remove ineffective &default in netcontrol cluster event handler args 2020-04-16 15:40:27 -07:00
Johanna Amann
730f78d0c2 Merge remote-tracking branch 'origin/topic/jsiwek/no-vla'
* origin/topic/jsiwek/no-vla:
  GH-895: Remove use of Variable-Length-Arrays

Fixes GH-895
2020-04-16 14:18:38 -07:00
Johanna Amann
df71d963c9 Merge remote-tracking branch 'origin/topic/jsiwek/krb-consts-defaults'
* origin/topic/jsiwek/krb-consts-defaults:
  Add default function for Kerberos constant-lookup-tables
2020-04-16 13:05:05 -07:00
Jon Siwek
2d91f9d89f Merge remote-tracking branch 'origin/topic/timw/dict-cleanup'
* origin/topic/timw/dict-cleanup:
  A few minor cleanups in Dict
2020-04-14 15:46:03 -07:00
Jon Siwek
f70ecccc34 Fix a confusing variable name shadowing 2020-04-14 15:30:15 -07:00
Jon Siwek
ffe8a018a1 Updating CHANGES and VERSION. 2020-04-14 11:10:07 -07:00
Tim Wojtulewicz
ba1c03188f Merge remote-tracking branch 'origin/topic/jsiwek/alternate-hook-event-prototypes'
* origin/topic/jsiwek/alternate-hook-event-prototypes:
  Add warning for ineffective &default arguments in handlers
  Fix frame size allocation of alternate event/hook handlers
  Emit error for alternate event/hook prototype args with attributes
  Improve alternate event/hook prototype matching
  Allow alternate event/hook prototype declarations
2020-04-13 15:00:25 -07:00
Johanna Amann
876c803d75 Merge remote-tracking branch 'origin/topic/timw/776-using-statements'
* origin/topic/timw/776-using-statements:
  Remove 'using namespace std' from SerialTypes.h
  Remove other using statements from headers
  GH-776: Remove using statements added by PR 770

Includes small fixes in files that changed since the merge request was
made.

Also includes a few small indentation fixes.
2020-04-09 13:31:07 -07:00
Johanna Amann
a3a38f0849 Merge remote-tracking branch 'origin/topic/timw/nullptr'
* origin/topic/timw/nullptr:
  The remaining nulls
  plugin/probabilistic/zeekygen: Replace nulls with nullptr
  file_analysis: Replace nulls with nullptr
  analyzer: Replace nulls with nullptr
  iosource/threading/input/logging: Replace nulls with nullptr
2020-04-09 08:59:53 -07:00
Jon Siwek
1210257af9 Update submodule(s)
[nomail]
2020-04-08 14:06:56 -07:00
Johanna Amann
4ee84b69f2 Merge remote-tracking branch 'origin/topic/jsiwek/udp_content_delivery_ports_use_resp'
* origin/topic/jsiwek/udp_content_delivery_ports_use_resp:
  Add "udp_content_ports" option
  Add new "udp_content_delivery_ports_use_resp" option
2020-04-07 14:28:49 -07:00
Johanna Amann
8e4656bc61 Merge remote-tracking branch 'origin/topic/jsiwek/fix-duration-threshold'
* origin/topic/jsiwek/fix-duration-threshold:
  Fix connection duration thresholding
2020-04-07 13:44:56 -07:00
Tim Wojtulewicz
08fbdb1418 Merge remote-tracking branch 'origin/topic/jsiwek/fix-patterns-as-table-index'
* origin/topic/jsiwek/fix-patterns-as-table-index:
  Fix using patterns stored as table/set indices
2020-04-07 12:05:44 -07:00
Jon Siwek
7e75eb750b Merge branch 'topic/ak/rdpeudp' of https://github.com/anthonykasza/zeek
Minor whitespace adjusted during merge.

* 'topic/ak/rdpeudp' of https://github.com/anthonykasza/zeek:
  change: -C flag to btest
  add: ignore checksum added to btest script
  change: update btest baselines for rdpeudp-handshake-success
  add: rdpeudp2-handshake-success btest baseline
  Update src/analyzer/protocol/rdp/rdpeudp-protocol.pac
  Update src/analyzer/protocol/rdp/rdpeudp-analyzer.pac
  add: btest for rdpeudp2
  add: more robust handling of RDPEUDP2 negotiation
  change: RDPUDP_CORRELATION_ID_PAYLOAD definition
  change: debugging why the orig synex flags aren't parsed correctly but the resp synex flags are...
  add: handling for orig SYNEX flags
  add: test trace for when RDPEUDP negotiates RDPEUDP2
  add: more SYN PDU parsing structs
2020-04-06 17:29:36 -07:00
Jon Siwek
e66148a13a Merge branch 'topic/ak/rdpeudp'
- Squashed the original commit set
- Cleaned up formatting
- Fixed register_for_ports() for right RDPEUDP analyzer

* topic/ak/rdpeudp:
  Add RDP over UDP analyzer
2020-04-02 18:31:40 -07:00
Jon Siwek
c42ebfa1cf Merge remote-tracking branch 'origin/topic/timw/the-missing-bools'
* origin/topic/timw/the-missing-bools:
  Fix uses of GetBool in bifs to use GetTrue/GetFalse
  Change Sessions::dump_this_packet to a bool
2020-04-02 15:03:34 -07:00
Johanna Amann
3ce1c9ffd6 Merge remote-tracking branch 'origin/topic/jsiwek/misc-lambda-fixes'
* origin/topic/jsiwek/misc-lambda-fixes:
  GH-725: fix logic for finding a lambda's usage of outer IDs
  Change record field anonymous functions to use lambda expressions
2020-03-31 15:46:54 -07:00
Johanna Amann
d18f2f8f76 Merge remote-tracking branch 'origin/topic/timw/override'
* origin/topic/timw/override:
  Mark input/output message classes as final, since nothing should be inheriting from them
  Add missing override specifier to a number of methods, remove virtual from some as well
  Add override specifier to Configure() method in almost all of the internal plugins
2020-03-31 08:48:44 -07:00
Robin Sommer
f62d7dd091 Merge branch 'topic/antonio.nappa/fix_lowercase_intel_bug' of ssh://github.com/jeppojeps/zeek 2020-03-31 07:08:14 +00:00
Robin Sommer
033fdcca74 Merge branch 'topic/timw/the-great-embooleanating'
(Squashed this one before merge).

* topic/timw/the-great-embooleanating:
  The Great Embooleanating
2020-03-31 06:57:39 +00:00
Johanna Amann
3c470ffe13 Merge remote-tracking branch 'origin/topic/jsiwek/gh-887-fix-gre-erspan-dot1q'
* origin/topic/jsiwek/gh-887-fix-gre-erspan-dot1q:
  GH-887: improve GRE/ERSPAN parsing of non-IPv4/IPv6 inner payload
  Fix ERSPAN III optional sub-header parsing
2020-03-30 14:30:00 -07:00
Tim Wojtulewicz
85c6c2b9ee Merge remote-tracking branch 'origin/topic/jsiwek/runtime-exception-leak-cleanup'
* origin/topic/jsiwek/runtime-exception-leak-cleanup:
  Func::DescribeDebug: move a NumFields() call out of loop
  Use const-ref parameter for zeek::val_list_to_args()
  Fix missing IntrusivePtr.h include and ambiguous ODesc::Add call
  Remove TimerMgr arg from event queuing/scheduling methods
  Deprecate Analyzer::ConnectionEvent()
  Deprecate file_analysis::File::FileEvent methods using val_list args
  Deprecate Connection::ConnectionEvent methods
  Deprecate EventMgr::QueueEventFast() and update usages to Enqueue()
  Deprecate EventMgr::QueueEvent() and update usages to Enqueue()
  Deprecate Func::Call(val_list*, ...)
  Use vector<IntrusivePtr<Val>> for Func::Call and Event queuing args
  Fix memory leak in Zeek when-statement bodies with runtime errors
  Change TableVal::RecoverIndex() to return IntrusivePtr
  Use IntrusivePtr in TableVal::CallExpireFunc
  Fix memory leak when runtime error occurs in a Zeek for-loop
  Enable leak checks for btests that produce runtime exceptions
2020-03-27 11:49:45 -07:00
Johanna Amann
8a4ccabc3e Merge remote-tracking branch 'origin/topic/jsiwek/fix-greedy-flowbuffer-bounds-check'
* origin/topic/jsiwek/fix-greedy-flowbuffer-bounds-check:
  Add test case for binpac flowbuffer frame length parsing bug
2020-03-24 13:49:58 -07:00
Jon Siwek
427150b27a Merge remote-tracking branch 'origin/topic/jsiwek/gh-857-rebuild-tables-on-record-redef'
* origin/topic/jsiwek/gh-857-rebuild-tables-on-record-redef:
  Initialize RecordVal default fields when redef'd
  GH-857: fix redefining record types used to index tables
  Change RecordVals to get resized at time of RecordType redef
2020-03-20 11:31:11 -07:00
Tim Wojtulewicz
535c70b087 Merge remote-tracking branch 'origin/topic/jsiwek/gh-865-smb-negotiate-context-list'
* origin/topic/jsiwek/gh-865-smb-negotiate-context-list:
  GH-865: fix parsing of SMB NegotiateContextList
2020-03-20 10:05:34 -07:00
Jon Siwek
e2aeb70efc Merge branch 'master' of https://github.com/zeek/zeek 2020-03-17 22:57:23 -07:00