Commit graph

2206 commits

Author SHA1 Message Date
Jon Siwek
b268b57194 Update embedded CAF to 0.17.4 2020-02-08 09:40:48 -08:00
Jon Siwek
6c72b09bf5 Note new CMake 3.0+ and C++17 requirements in NEWS 2020-02-07 21:11:35 -08:00
Jon Siwek
e50a8848ae Improve an input framework test
A race condition could cause unstable output: if the thread reading the
file is fast, often you see both "pred" functions execute and then both
"line" events execute with both entries already in the table, but if the
thread reading the file is slow, you see pred, event, pred, event, with
only one entry available in the first event.
2020-02-07 20:03:10 -08:00
Jon Siwek
a3b1d202a5 Improve stability of a &expire_func btest
The order in which &expire_func's get called isn't well-defined, so
separate the output from either to ensure diffs against the Baseline are
always consistent.
2020-02-07 15:11:06 -08:00
Tim Wojtulewicz
9754c2c09f Merge remote-tracking branch 'max/optimize'
* max/optimize:
  plugin/Manager: migrate to std::string_view
  util: optimize the normal_path() common case
  util: pass string_view to without_bropath_component()
  module_util: make GLOBAL_MODULE_NAME constexpr
  Scope: convert Scope::Lookup() and others to template
  Scope: Vars() returns const reference
  Anon: remove unnecessary {map,vector}::clear() calls
  Dict: make the destructor non-virtual
  Obj: make the Location constructors `constexpr`
  Obj: remove unused fields Location::{timestamp,text}
  Obj: remove Location::delete_data, nobody ever sets it
  Obj: make the Location destructor non-virtual
2020-02-07 15:26:30 -07:00
Johanna Amann
febfc30c70 Merge remote-tracking branch 'origin/topic/jsiwek/krb-addrs-and-leak'
* origin/topic/jsiwek/krb-addrs-and-leak:
  Improve kerberos analyzer address and event handling
2020-02-07 14:15:44 -08:00
Johanna Amann
56dda98512 Merge remote-tracking branch 'origin/topic/jsiwek/stabilize-ci'
* origin/topic/jsiwek/stabilize-ci:
  Add environment variable to disable supervisor SIGKILL
  Fix a btest's race condition by handling Pcap::file_done
  Add Pcap::file_done event
  Improve btest timeouts
2020-02-07 09:52:57 -08:00
Johanna Amann
a0fea11247 Merge remote-tracking branch 'origin/topic/jsiwek/use-btest-retries'
* origin/topic/jsiwek/use-btest-retries:
  Use 2 btest retries for CI
2020-02-07 09:44:24 -08:00
Johanna Amann
49a21d73d0 Merge remote-tracking branch 'origin/topic/jsiwek/gh-780-indefinite-log-buffer'
* origin/topic/jsiwek/gh-780-indefinite-log-buffer:
  GH-780: Prevent log batches from indefinite buffering
2020-02-07 09:35:29 -08:00
Robin Sommer
a394d2e0d7 Update NEWS to point out Broker wireformat change. 2020-02-07 10:38:25 +00:00
Johanna Amann
391164c9bc Merge remote-tracking branch 'origin/topic/johanna/table-on-change'
* origin/topic/johanna/table-on-change:
  &on_change: Address feedback of Jon.
  &on_change: incooperate feedback of Jon.
  &on_change wrapup: documentation, tests, whitespacing
  Table expiry now raises &on_change handlers
  Prevent recursion of &on_change handlers.
  Make bro_broker::val_to_data take a const Val* instead of a Val
  &on_change: add insertion/change notifications.
  &on_change working for removals from tables.
  Change signature of &on_change handler
  Beginning implementation of &on_change for tables.
2020-02-06 15:01:03 -08:00
Jon Siwek
8e812df6c5 Switch GitHub Action for CI emails to use zeek organization 2020-02-06 13:06:56 -08:00
Jon Siwek
3ee5f5fcaf Update CAF to current 0.17.4 release branch 2020-02-05 21:47:28 -08:00
Jon Siwek
7f76381f13 Remove presumed ownership of HeartbeatTimer
threading::Manager is currently never deleted, but if that ever changes,
deleting the stored HeartbeatTimer pointer would be a double-free
since TimerMgr owns it.
2020-02-05 17:56:02 -08:00
Jon Siwek
c5340c7c3e Merge remote-tracking branch 'origin/topic/timw/773-eventmgr-iosource'
Fixes GH-773

* origin/topic/timw/773-eventmgr-iosource:
  GH-773: Make EventMgr an IOSource so that events interrupt kevent waits
2020-02-05 14:57:26 -08:00
Jon Siwek
0de6bba95e Merge remote-tracking branch 'origin/topic/johanna/692'
* origin/topic/johanna/692:
  Resolve race condition in test
  Change threading formatter errors to warnings.
2020-02-05 10:52:39 -08:00
Tim Wojtulewicz
4a7da2669c Merge remote-tracking branch 'max/include_cleanup'
* max/include_cleanup:
  UID, ..: un-inline methods to reduce header dependencies
  include cleanup
2020-02-04 14:06:05 -07:00
Robin Sommer
532c66df51 Updating submodule.
[nomail]
2020-02-04 11:58:06 +00:00
Tim Wojtulewicz
37e0130461 Merge remote-tracking branch 'origin/topic/jsiwek/consistent-string-view-args'
* origin/topic/jsiwek/consistent-string-view-args:
  Change various functions to by-value std::string_view args
2020-02-03 15:44:28 -07:00
Tim Wojtulewicz
5aa8120259 Merge remote-tracking branch 'origin/topic/jsiwek/pktsrc-options'
* origin/topic/jsiwek/pktsrc-options:
  Change packet source fields of Options to std::optional
2020-02-03 15:42:48 -07:00
Johanna Amann
2129a808f6 Updating CHANGES and VERSION after messing up last commit message. 2020-02-03 13:25:57 -08:00
Jon Siwek
2cda94e8ee Disable Travis leak test
Travis environment doesn't offer enough memory for running ASan very
well, but Cirrus has it covered anyway.
2020-02-03 13:21:32 -08:00
Jon Siwek
d33613c2a5 Merge branch 'master' of https://github.com/ffontaine/zeek
- Also removed the setting of BinPAC_ROOT_DIR in the configure
  script's --with-binpac= option as that breaks the cross-compilation
  use-case

* 'master' of https://github.com/ffontaine/zeek:
  CMakeLists.txt: fix cross-compilation with binpac
2020-02-03 13:05:50 -08:00
Jon Siwek
1e2dd4466c Change requested memory in Cirrus CI config
The ASan build does require quite a bit more, but also the default
release builds should do ok with slightly less.
2020-02-03 10:11:46 -08:00
Jon Siwek
1df5793479 Merge remote-tracking branch 'origin/topic/timw/ioloop-followup'
* origin/topic/timw/ioloop-followup:
  Increase timeout value for live interfaces without file descriptors
  Use ranged-for loops in a few places in iosource Manager
  Change order of includes in iosource Manager, fixes build on FreeBSD 11
2020-02-03 10:04:26 -08:00
Jon Siwek
fa5b3bb91e Merge branch 'no_sscanf' of https://github.com/MaxKellermann/zeek
* 'no_sscanf' of https://github.com/MaxKellermann/zeek:
  util: optimize expand_escape() by avoiding sscanf()
2020-01-31 14:19:12 -08:00
Jon Siwek
d39bb42b14 Merge branch 'optimize_normalize_path' of https://github.com/MaxKellermann/zeek
- Minor changes in merge: extended unit test, prefer emplace_back(),
  remove unused "found" count in new function

* 'optimize_normalize_path' of https://github.com/MaxKellermann/zeek:
  util: add a tokenize_string() overload which returns string_views
  util: store std::string_view in "final_components" vector
  util: use "auto" in normalize_path()
  util: reserve space in normalize_path()
  util: skip "." completely in normalize_path()
  util: pass std::string_view to normalize_path()
  util: pass std::string_view to tokenize_string()
  util: don't modify the input string in tokenize_string()
2020-01-31 13:23:39 -08:00
Jon Siwek
719011cef0 Remove extra fmt() in a reporter->Error() call
Else compiler may warn about format string not being a string literal
2020-01-31 11:25:07 -08:00
Jon Siwek
077fdac332 Merge remote-tracking branch 'origin/topic/timw/264-alternative'
* origin/topic/timw/264-alternative:
  Add better error messaging when RegisterFd/UnregisterFd fail
  Update Supervisor code for the new IOSource API
  Show percentage of packets dropped in the final process output
  Update NEWS and docs submodule
  Remove concept of multiple timer managers
  Test changes caused by minor order-of-operation changes related to the new loop architecture
  Have terminate_processing() raise SIGTERM instead of calling the signal handler directly
  PktSrc iosource changes to match the new IOSource API
  Broker manager changes to match the new IOSource API and loop architecture
  DNS_Mgr changes to match the new IOSource API and loop architecture
  Threading changes for the new loop architecture
  Add Trigger manager for managing triggers created by things like 'when' statements
  Make TimerMgr an IOSource
  Modify IOSource Manager to implement new loop architecture
  Minor amount of code cleanup in Pcap IO source
  IOSource API changes for new loop architecture
  Type and variable usage cleanup in Net.h
  Remove #include of some iosource files from Net.h
  Only allow a single trace file (-r) or interface (-i) option on the command-line
  Remove CQ_TimerMgr in favor of PQ_TimerMgr
2020-01-31 11:04:27 -08:00
Jon Siwek
70b45d1aba Merge remote-tracking branch 'origin/topic/robin/631-deprecation-v2'
During merge I split the test for bro_init/bro_done/bro_script_loaded
event errors into individual tests since the other testing of the zeek
versions of those events seemed fine to otherwise keep.

* origin/topic/robin/631-deprecation-v2:
  Update NEWS for naming changes.
  Small cleanup and updating submodules.
  Remove test for legacy plugin.
  Remove legancy symlinks in aux/.
  Add warnings when loading scripts ending in ".bro", or using legacy environment variables.
  Fix missing rename.
  No longer symlink local.zeek to local.bro.
  Update notice user agent.
  Remove old_comm_usage_is_ok.
  Remove bro-config.h.in and bro-path-dev.in.
  Change Bro wrapper script to now abort when old executable names are still used.
  Remove APIs that were explicitly deprecated to be removed in 3.1.
2020-01-30 19:19:56 -08:00
Tim Wojtulewicz
c5b2659b6b Merge remote-tracking branch 'origin/topic/jsiwek/supervisor-coverity-fixes'
* origin/topic/jsiwek/supervisor-coverity-fixes:
  Fix leaked FDs in redirecting supervised node stdout/stderr
  Add checks for failed fcntl calls
  Set Pipe file descriptor flags correctly
2020-01-30 08:44:59 -07:00
Jon Siwek
f3e5728bcb Merge branch 'leaks' of https://github.com/MaxKellermann/zeek
* 'leaks' of https://github.com/MaxKellermann/zeek:
  Scope: fix memory leak by removing duplicate copy_string() call
  util, nb_dns: fix off-by-one bugs in strncpy() calls
  Type, util: add `constexpr` to static variables
  Net: remove unused variable
2020-01-29 11:50:09 -08:00
Johanna Amann
ad18014bed Merge remote-tracking branch 'origin/topic/jsiwek/ssl-empty-files'
* origin/topic/jsiwek/ssl-empty-files:
  Skip file analysis for zero-length SSL/TLS data
2020-01-29 11:16:35 -08:00
Robin Sommer
6bcd583836 Merge remote-tracking branch 'origin/topic/jsiwek/supervisor'
* origin/topic/jsiwek/supervisor: (44 commits)
  Add note that Supervisor script APIs are unstable until 4.0
  Move command-line arg parsing functions to Options.{h,cc}
  Add btests for supervisor stem/leaf process revival
  Move supervisor control events into SupervisorControl namespace
  Fix supervisor "destroy" call on nodes not currently alive
  Move supervisor source files into supervisor/
  Address supervisor code re-factoring feedback from Robin
  Convert supervisor internals to rapidjson
  Add Supervisor documentation
  Add supervisor btests
  Improve logging of supervised node errors
  Fix supervised node inheritence of command-line script paths
  Improve normalize_path() util function
  Use a timer to check for death of supervised node's parent
  Improve supervisor checks for parent process termination
  Improve handling of premature supervisor stem exit
  Improve supervisor signal handler safety
  Remove unused supervisor config options
  Cleanup minor Supervisor TODOs
  Improve supervisor debug logging
  ...
2020-01-29 13:11:04 +00:00
Jon Siwek
84b8ee3b06 Merge remote-tracking branch 'origin/topic/jsiwek/cirrus-catalina'
* origin/topic/jsiwek/cirrus-catalina:
  Update Cirrus CI config to use macOS Catalina
2020-01-28 17:56:22 -08:00
Jon Siwek
83874fa5fa Merge branch 'getrandom' of https://github.com/MaxKellermann/zeek
- Removed the superfluous check for C++17 in the merge since that's
  a requirement enforced at the CMake-level.

* 'getrandom' of https://github.com/MaxKellermann/zeek:
  util: use getrandom() on Linux if available
2020-01-28 12:45:15 -08:00
Jon Siwek
0412cb3996 Merge branch 'topic/frerich/gh-750-uri-with-empty-portnum' of https://github.com/frerich/zeek
* 'topic/frerich/gh-750-uri-with-empty-portnum' of https://github.com/frerich/zeek:
  Document recent fix for decompose_uri in release notes
  Fixed decompose_uri choking on URIs with empty port component
  Adding test for decompose_uri parsing URI with empty port
2020-01-24 15:16:14 -08:00
Jon Siwek
f45c2cf06b Skip check for outdated docs in Cirrus CI for PRs 2020-01-23 18:10:17 -08:00
Johanna Amann
97bea7c4f1 Merge remote-tracking branch 'origin/topic/jsiwek/fix-create-stream-priorities'
* origin/topic/jsiwek/fix-create-stream-priorities:
  Use consistent zeek_init priority for Log::create_stream calls
2020-01-23 15:31:06 -08:00
Robin Sommer
fc1c95725b Update Broker submodule.
Includes a test baseline update for change in Broker error message.
2020-01-23 13:51:29 +00:00
Robin Sommer
01b7db5b46 Merge remote-tracking branch 'origin/topic/jsiwek/smb-transaction-strings'
* origin/topic/jsiwek/smb-transaction-strings:
  Improve creation of SMB transaction data strings
2020-01-23 13:19:11 +00:00
Robin Sommer
11850088ee Merge remote-tracking branch 'origin/topic/jsiwek/ftp-word'
* origin/topic/jsiwek/ftp-word:
  Improve FTP word/whitespace handling
2020-01-23 13:14:46 +00:00
Robin Sommer
270702cacb Updating CHANGES and VERSION. 2020-01-18 14:32:30 +00:00
Robin Sommer
c8c6621a0e Merge remote-tracking branch 'origin/topic/timw/bit-fields'
* origin/topic/timw/bit-fields:
  Use bools instead of single-bit bitfields in Ident and TCP protocol analyzers
  Bit of code-modernization cleanup in BroString
  Use fixed types in NetbiosSSN.h and Timer.h instead of bit fields
2020-01-17 11:55:00 +00:00
Robin Sommer
0d29a80519 Merge remote-tracking branch 'origin/topic/jsiwek/ftp-adat-fix'
* origin/topic/jsiwek/ftp-adat-fix:
  Handle invalid Base64 encodings in FTP ADAT analyzer
2020-01-16 13:27:03 +00:00
Jon Siwek
f59e3da4f7 Merge remote-tracking branch 'origin/sethhall-fix-ssl-files-warning'
* origin/sethhall-fix-ssl-files-warning:
  Fix warning when reading files from non-network sources
2020-01-14 09:46:19 -08:00
Robin Sommer
bcf9dbac94 Update submodules. 2020-01-13 12:11:08 +00:00
Jon Siwek
6218f99b96 Update timeouts for a btest 2020-01-09 19:05:23 -08:00
Tim Wojtulewicz
03eb1d43d2 Merge remote-tracking branch 'origin/topic/jsiwek/btest-asan-options'
* origin/topic/jsiwek/btest-asan-options:
  Extend ASAN_OPTIONS in btests instead of overwrite
2020-01-09 13:39:49 -08:00
Jon Siwek
da0185a60f Update external test repo commit hash 2020-01-09 12:29:02 -08:00