Commit graph

1825 commits

Author SHA1 Message Date
Jon Siwek
5c6ebb443e Merge remote-tracking branch
* Fix potential NetbiosSSN analyzer stack overflow
2020-07-27 11:40:40 -07:00
Jon Siwek
97090ea18d Update submodules 2020-07-27 11:22:55 -07:00
Jon Siwek
b62fd71181 Misc/minor cleanup after reviewing new Broker table store code
Just some naming/style + minor memory management tweaks (avoid
some copies; use move semantics).
2020-07-24 14:00:24 -07:00
Jon Siwek
823d4e6220 Merge remote-tracking branch 'origin/topic/vladg/gh-1084'
* origin/topic/vladg/gh-1084:
  Add btest for GH-1084
  Update baselines
  MySQL: Fix parsing logic bug. We were correctly NOT expecting an EOF, but because we were parsing the header and then not parsing the rest, we would get out of sync
2020-07-24 11:41:16 -07:00
Tim Wojtulewicz
3eab3e600e Merge remote-tracking branch 'ronwellman/parse_ecs'
* ronwellman/parse_ecs:
  Avoid typecast to int& in EDNS parsing.
  Validate option_len in EDNS packets.
  Adjust for zeek namespace.
  Implement EDNS Client Subnet Option
2020-07-24 09:26:06 -07:00
Tim Wojtulewicz
ebc073ba92 Merge remote-tracking branch 'origin/topic/jsiwek/gh-1076-fix-random'
* origin/topic/jsiwek/gh-1076-fix-random:
  Deprecate bro_srandom(), replace with zeek::seed_random().
  Add zeek::max_random() & fix misuse of RAND_MAX w/ zeek::random_number()
  Deprecate bro_random(), replace with zeek::random_number()
  Deprecate bro_prng(), replace with zeek::prng()
  GH-1076: Fix bro_srandom() to replace 0 seeds with 1
  GH-1076: Fix bro_prng() implementation
  GH-1076: Fix use of getrandom()
2020-07-23 13:02:46 -07:00
Tim Wojtulewicz
ac57377a94 Merge remote-tracking branch 'origin/topic/jsiwek/gh-1068-zeek-include-dir'
* origin/topic/jsiwek/gh-1068-zeek-include-dir:
  GH-1068: Add zeek symlink to allow "zeek/Foo.h" include style
2020-07-23 08:32:20 -07:00
Tim Wojtulewicz
ebf0f65f55 Merge remote-tracking branch 'origin/topic/jsiwek/add-zeek-archiver-submodule'
* origin/topic/jsiwek/add-zeek-archiver-submodule:
  Update zeek-archiver submodule
  Add zeek-archiver tool as submodule
2020-07-21 12:40:13 -07:00
Robin Sommer
c3f4971eb2 Merge remote-tracking branch 'origin/topic/johanna/table-changes'
* origin/topic/johanna/table-changes: (26 commits)
  TableSync: try to make test more robust & add debug output
  Increase timeouts to see if FreeBSD will be happy with this.
  Try to make FreeBSD test happy with larger timeout.
  TableSync: refactor common functionality into function
  TableSync: don't raise &on_change, smaller fixes
  TableSync: rename auto_store -> table_store
  SyncTables: address feedback part 1 - naming (broker and zeek)
  BrokerStore <-> Zeek Tables: cleanup and bug workaround
  Zeek Table<->Brokerstore: cleanup, documentation, small fixes
  BrokerStore<->Zeek table: adopt to recent Zeek API changes
  BrokerStore<->Zeek Tables Fix a few small test failures.
  BrokerStore<->Zeek tables: allow setting storage location & tests
  BrokerStore<->Zeek tables: &backend works for in-memory stores.
  BrokerStore<->Zeek table - introdude &backend attribute
  BrokerStore<->Zeek tables: test for clones synchronizing to a master
  BrokerStore<->Zeek tables: load persistent tables on startup.
  Brokerstore<->Tables: attribute conflicts
  Zeek/Brokerstore updates: expiration
  Zeek/Brokerstore updates: add test that includes updates from clones
  Zeek/Brokerstore updates: first working end-to-end test
  ...
2020-07-21 15:39:39 +00:00
Jon Siwek
bd5108022f Merge remote-tracking branch 'origin/topic/timw/deprecation-fixes-for-spicy'
* origin/topic/timw/deprecation-fixes-for-spicy:
  Use namespaced version of Location to silence warnings
  Various deprecation fixes, reported by failed Spicy builds
  Add deprecated version of EnumType::GetVal() to returns EnumVal*, rename IntrusivePtr version to GetEnumVal
2020-07-17 16:37:30 -07:00
Jon Siwek
54c9f4a0e1 Merge remote-tracking branch 'origin/topic/timw/deprecation-fixes-for-spicy'
Merge adjustments:

- Revert unneeded changes in mmdb_dir lookups to fix build
- Fix deprecated EnumType::GetVal() to return +1 ref-count

* origin/topic/timw/deprecation-fixes-for-spicy:
  Various deprecation fixes, reported by failed Spicy builds
  Add deprecated version of EnumType::GetVal() to returns EnumVal*, rename IntrusivePtr version to GetEnumVal
2020-07-17 16:14:49 -07:00
Johanna Amann
a0c0236fb1 Merge remote-tracking branch 'origin/topic/jsiwek/util-fixes'
* origin/topic/jsiwek/util-fixes:
  Fix race condition in ensure_dir()
  Fix tokenize_string() to work with delimiters of length > 1
2020-07-17 22:34:48 +00:00
Robin Sommer
651831ad31 Merge remote-tracking branch 'origin/topic/jsiwek/gh-734-fix-lambda-escaping'
* origin/topic/jsiwek/gh-734-fix-lambda-escaping:
  GH-734: Improve handling of lambdas that escape enclosing frame
2020-07-17 10:19:18 +00:00
Robin Sommer
edaeb2f308 Merge remote-tracking branch 'origin/topic/jsiwek/gh-837-improve-broker-mismatch-errors'
* origin/topic/jsiwek/gh-837-improve-broker-mismatch-errors:
  GH-837: Add test cases for mismatched Broker SSL configs
  GH-837: emit Reporter errors for Broker errors
  Fix incorrect/missing Broker error status code numbers
2020-07-17 10:09:05 +00:00
Jon Siwek
fd8db24daf Merge remote-tracking branch 'origin/topic/jsiwek/gh-1062-fix-tag-subtypes'
* origin/topic/jsiwek/gh-1062-fix-tag-subtypes:
  GH-1062: fix integer conversion errors related to Tag subtypes
2020-07-14 19:11:09 -07:00
Johanna Amann
da2c968ba0 Merge remote-tracking branch 'origin/topic/jsiwek/gh-1024-broker-store-handle-type-checks'
* origin/topic/jsiwek/gh-1024-broker-store-handle-type-checks:
  Improve Broker store API's handling of invalid arguments
  Add builtin_exception() functions
  GH-1024: fix crash on passing wrong types to Broker store API

This was a bit of a bigger merge since Zeek changed inbetween the time
of the PR and me actually merging it.

I put the new functions into the zeek::detail namespace -- since it
seems unlikely that those will be used by something external.

I also renamed them to fit better with the naming scheme of the new
error functions.

Fixes GH-1024
2020-07-14 00:05:09 +00:00
Tim Wojtulewicz
75cb35e2d9 Merge remote-tracking branch 'origin/topic/jsiwek/gh-1019-deprecate-icmp-conn'
* origin/topic/jsiwek/gh-1019-deprecate-icmp-conn:
  GH-1019: deprecate icmp_conn params for ICMP events
2020-07-13 12:12:44 -07:00
Tim Wojtulewicz
64af3ec67a Merge remote-tracking branch 'origin/topic/jsiwek/deprecation-improvements'
* origin/topic/jsiwek/deprecation-improvements:
  Fix wrong frame offsets for locals of alternate event/hook prototypes
  Add deprecation expression to deprecated prototype/parameter messages
  Improve "use of deprecated prototype" warning message
  Emit deprecation warning for use of &deprecated function parameters
2020-07-13 12:11:12 -07:00
Jon Siwek
ba0ba7c684 Add more error checks to shadow log parsing
i.e. Coverity warns about possible use of ftell() negative return value
2020-07-10 11:21:49 -07:00
Jon Siwek
39f549ed68 Merge remote-tracking branch 'origin/topic/jsiwek/3.2-supervisor'
* origin/topic/jsiwek/3.2-supervisor:
  Integrate Supervisor code review suggestions
  Add Supervisor::{stdout,stderr}_hook
  Add Log::rotation_format_func and Log::default_rotation_dir options
  Allow Zeek function equality comparisons
  Implement leftover log rotation/archival for supervised nodes
  Move some supervisor structures to detail namespace
  Improve default supervisor stdout/stderr handling
  Factor Supervisor initialization data to separate struct from Stem
  Move ParentProcessCheckTimer to detail namespace
2020-07-09 14:13:26 -07:00
Jon Siwek
48362cc0c0 Merge remote-tracking branch 'origin/topic/jsiwek/remove-lexer-dead-code'
* origin/topic/jsiwek/remove-lexer-dead-code:
  scan.l: Remove "constant" did_module_restore logic
2020-07-08 17:47:07 -07:00
Jon Siwek
d15dd37b84 Merge remote-tracking branch 'origin/topic/jsiwek/fix-ci-freebsd-swig'
* origin/topic/jsiwek/fix-ci-freebsd-swig:
  Fix FreeBSD CI script to install right SWIG package
2020-07-08 12:25:31 -07:00
Robin Sommer
1fe822a745 Merge remote-tracking branch 'origin/topic/jsiwek/fix-while-stmt-ctor'
* origin/topic/jsiwek/fix-while-stmt-ctor:
  Fix WhileStmt to call Stmt(Tag) ctor
2020-07-08 15:41:50 +00:00
Robin Sommer
776250fd8c Merge remote-tracking branch 'origin/topic/jsiwek/gh-928-realpath-load-dedup'
* origin/topic/jsiwek/gh-928-realpath-load-dedup:
  GH-928: use realpath() instead of inode to de-duplicate scripts
2020-07-08 15:40:24 +00:00
Jon Siwek
7b15b82009 Merge remote-tracking branch 'origin/topic/timw/1040-str-split'
* origin/topic/timw/1040-str-split:
  GH-1040: Add zero-indexed version of str_split
2020-07-06 21:06:51 -07:00
Jon Siwek
09483619ef Merge remote-tracking branch 'origin/topic/timw/1041-compress-path'
* origin/topic/timw/1041-compress-path:
  GH-1041: Move compress_path to a bif that uses normalize_path
2020-07-06 20:41:05 -07:00
Robin Sommer
3d3d5e7eb4 Merge remote-tracking branch 'origin/topic/jsiwek/add-backtrace-bif'
(Added a Doxygen comment.)

* origin/topic/jsiwek/add-backtrace-bif:
  Add backtrace() and print_backtrace()
2020-07-06 14:56:28 +00:00
Jon Siwek
61b52b54ba Fix --enable-mobile-ipv6 build 2020-07-03 14:30:13 -07:00
Jon Siwek
3a59948238 Fix namespace of GetCurrentLocation() to zeek::detail 2020-07-03 14:18:02 -07:00
Jon Siwek
76e67ff239 Merge remote-tracking branch 'origin/topic/timw/266-namespaces-part2'
* origin/topic/timw/266-namespaces-part2:
  Rename BroString files to ZeekString
  Update NEWS entry with note about class renames
  Rename BroObj to Obj
  Rename BroString to zeek::String
  Move Func up to zeek namespace, rename BroFunc to ScriptFunc
  Mark global val_mgr as deprecated and fix uses of it to use namespaced version
  Minor cleanup items from PR review
  Update binpac and bifcl submodules with review changes
  Move Location to zeek::detail and BroObj to zeek
  Move BroString to zeek namespace
  Move Dictionary/PDict, List/PList, and Queue/PQueue to zeek namespace
  Remove typedef that should have been removed in 3.1
  Move Func and associated classes into zeek::detail namespace
  Move Frame and Scope to zeek::detail namespace
  Move all Val classes to the zeek namespaces
  Use type aliases for IntrusivePtr definitions
  Move deprecation macro to zeek-config.h.in to avoid having to over-include util.h
  Move IntrusivePtr and utility methods to the zeek namespace
2020-07-02 19:25:13 -07:00
Tim Wojtulewicz
d1814ea1ff Merge remote-tracking branch 'origin/topic/jsiwek/gh-786-fix-incomplete-casts'
* origin/topic/jsiwek/gh-786-fix-incomplete-casts:
  GH-786: fix Val::As methods casting incomplete types
  GH-786: fix Stmt::As methods casting incomplete types
  GH-786: fix Expr::As methods casting incomplete types
2020-07-02 15:57:40 -07:00
Jon Siwek
0ec2ff20c6 Merge remote-tracking branch 'origin/topic/timw/786-typecasting'
* origin/topic/timw/786-typecasting:
  GH-786: Move Type::As methods to cc file so they have type info for casting safely
2020-07-02 11:38:35 -07:00
Jon Siwek
734af6b6d1 Fix a deprecation warning compiling with GCC 2020-07-01 09:58:18 -07:00
Jon Siwek
4668378d91 Merge remote-tracking branch 'origin/topic/timw/1032-namespaced-enums'
Merge adjustments:
- Added back in deprecation tag for base_type_no_ref()
- Added back the deprecated plugin::hook_name() function

* origin/topic/timw/1032-namespaced-enums:
  Deprecate plugin::HookType and plugin::component::Type in a different way
  Deprecate init_class and IDScope in another way.
  Deprecate TypeTag and friends in a different way
  Deprecate attr_tag in a different way, rename to AttrTag
2020-06-30 16:55:36 -07:00
Jon Siwek
35fa042fd6 Merge remote-tracking branch 'origin/topic/timw/1034-IndexTypes'
Minor tweaks during merge:
- Remove/default superfluous ~Attributes()
- Added in resize() calls to accompany reserve() of associated vectors

* origin/topic/timw/1034-IndexTypes:
  Revert Attributes::Attrs back to return an attr_list and mark it deprecated
  GH-1034: Revert TypeList::Types() back to return a type_list* and mark it deprecated
2020-06-30 15:56:49 -07:00
Johanna Amann
a5a51de3c4 Merge remote-tracking branch 'origin/topic/jsiwek/gh-1036-print-log-network-time'
* origin/topic/jsiwek/gh-1036-print-log-network-time:
  GH-1036: change print.log to log network time instead of current

Fixes GH-1036
2020-06-29 19:25:16 +00:00
Jon Siwek
7e28c16dfc Merge remote-tracking branch 'origin/topic/jazoff/conn-polling-memory'
* origin/topic/jazoff/conn-polling-memory:
  reduce memory usage of ConnPolling
2020-06-29 10:25:26 -07:00
Johanna Amann
eb1a408b6a Merge remote-tracking branch 'origin/topic/jsiwek/improve-command-line-option-redefs'
* origin/topic/jsiwek/improve-command-line-option-redefs:
  Integrate review feedback: improve command-line option redef parsing
  Fix several issues with command-line option redefs
2020-06-26 16:27:42 +00:00
Jon Siwek
00a4865885 Merge branch 'dns-original-query-case' of https://github.com/rvictory/zeek
Changes during merge
- Changed the policy script to use an event handler that behaves
  for like the base script: &priority=5, msg$opcode != early-out,
  no record field existence checks
- Also extended dns_query_reply event with original_query param
- Removed ExtractName overload, and just use default param

* 'dns-original-query-case' of https://github.com/rvictory/zeek:
  Fixed some places where tabs became spaces
  Stricter checking if we have a dns field on the connection being processed
  Modified the DNS protocol analyzer to add a new parameter to the dns_request event which includes the DNS query in its original case. Added a policy script that will add the original_case to the dns.log file as well. Created new btests to test both.
2020-06-26 00:07:02 -07:00
Jon Siwek
94ba6303f3 Merge remote-tracking branch 'origin/topic/johanna/make-tables-slightly-smaller'
* origin/topic/johanna/make-tables-slightly-smaller:
  Remove last_access_time from TableEntryVal.
2020-06-19 13:34:07 -07:00
Johanna Amann
7ae5589469 Merge branch 'better_musl_support' of https://github.com/abenson/zeek
* 'better_musl_support' of https://github.com/abenson/zeek:
  cmake: Make musl support more distro agnostic
2020-06-18 20:42:33 +00:00
Johanna Amann
af2110cfc9 Merge remote-tracking branch 'origin/topic/jsiwek/reduce-ftp-cluster-msg-sizes'
* origin/topic/jsiwek/reduce-ftp-cluster-msg-sizes:
  Minimize data published for expected FTP data channel analysis
2020-06-18 20:07:26 +00:00
Jon Siwek
8d9e85b842 Fix location where CI places build.tgz 2020-06-15 11:04:50 -07:00
Jon Siwek
ebac462895 Merge remote-tracking branch 'origin/topic/timw/ci-benchmark'
* origin/topic/timw/ci-benchmark:
  Review cleanup
  Add new CI task for running benchmarks on a remote host
2020-06-15 10:28:14 -07:00
Jon Siwek
6cec268e43 Merge remote-tracking branch 'origin/topic/jsiwek/gh-977-improve-pcap-error-handling'
* origin/topic/jsiwek/gh-977-improve-pcap-error-handling:
  Compare pcap_next_ex() result to PCAP_ERROR/PCAP_ERROR_BREAK
  GH-977: Improve pcap error handling
  Remove not-useful code in iosource::Manager::OpenPktSrc
2020-06-11 23:22:19 -07:00
Jon Siwek
d4f3cad7d1 Merge remote-tracking branch 'origin/topic/timw/266-namespaces'
Merge adjustments:

- Preserved original `base_type_no_ref` argument type as ::TypeTag
- Removed superfluous #pragma guard around deprecated TableVal ctor
- Clarify NEWS regarding MetaHook{Pre,Post} deprecations
- Simplify some `::zeek::` qualifications to just `zeek::`
- Prefixed FORWARD_DECLARE_NAMESPACED macro with ZEEK_

* origin/topic/timw/266-namespaces:
  Disable some deprecation diagnostics for GCC
  Rename BroType to Type
  Update NEWS
  Review cleanup
  Move Type types to zeek namespace
  Move Flare/Pipe from the bro namespace to zeek::detail
  Move Attr to the zeek::detail namespace
  Move Trigger into the zeek::detail namespace
  Move ID to the zeek::detail namespace
  Move Anon.h into zeek::detail namespace
  Mark all of the aliased classes in plugin/Plugin.h deprecated, and fix all of the plugins that were using them
  Move all of the base plugin classes into the zeek::plugin namespace
  Expr: move all classes into zeek::detail
  Stmt: move Stmt classes into zeek::detail namespace
  Add utility macro for creating namespaced aliases for classes
2020-06-11 23:20:51 -07:00
Jon Siwek
0bc1d73c84 Optimize Connection::RemovalEvent() for bare-mode usage
It was creating RecordVals even if they wouldn't be used by any event
handler and that situation is common/expected for `zeek -b` mode.
Normally, there's at least the tunnel scripts with a `new_connection`
handler causing the connection RecordVals to be built.
2020-06-10 16:38:39 -07:00
Jon Siwek
e924370876 Restrict Cirrus CI to only zeek repo's branches 2020-06-09 15:19:29 -07:00
Jon Siwek
3255930738 Merge remote-tracking branch
* Prevent IP fragment reassembly on packets without minimal IP header
2020-06-09 09:46:46 -07:00
Johanna Amann
c4ebd7640d Merge remote-tracking branch 'origin/topic/jsiwek/gh-999-dhcp-client-id-hw0'
* origin/topic/jsiwek/gh-999-dhcp-client-id-hw0:
  GH-999: Stop formatting DHCP Client ID Hardware Type 0 as MAC

Fixes GH-999
2020-06-08 22:32:01 +00:00