Commit graph

12339 commits

Author SHA1 Message Date
Vern Paxson
509428a9dc removing -uu functionality and associated script analysis now no longer needed 2021-09-22 11:17:28 -07:00
zeek-bot
8a1b32c877 Update doc submodule [nomail] [skip ci] 2021-09-22 00:32:30 +00:00
Tim Wojtulewicz
c87e290bf1 Merge remote-tracking branch 'origin/topic/timw/fixing-homebrew-again'
* origin/topic/timw/fixing-homebrew-again:
  Use `brew update` instead of `brew update-reset` in CI macOS prepare script
2021-09-21 14:13:30 -07:00
Tim Wojtulewicz
1a4af90389 Use brew update instead of brew update-reset in CI macOS prepare script
Using `brew update-reset` causes homebrew to reset to homebrew's HEAD commit, which may
be buggy and broken. It appears whatever Cirrus was doing previously on their Catalina
VM is no longer a problem, and so update-reset isn't required anymore. Switch to
`brew update` to make sure we still get newer versions of the packages, but is
actually a versioned release of homebrew.
2021-09-21 13:05:33 -07:00
Tim Wojtulewicz
a507d4294d Merge remote-tracking branch 'origin/topic/timw/test-disabling-profiling'
* origin/topic/timw/test-disabling-profiling:
  Disable script profiling/coverage on sanitizer builds
2021-09-21 09:44:55 -07:00
Tim Wojtulewicz
8354bff91c Disable script profiling/coverage on sanitizer builds 2021-09-21 09:43:08 -07:00
Robin Sommer
74680bf4e6 Merge remote-tracking branch 'origin/topic/justin/software-framework-parse-cache'
* origin/topic/justin/software-framework-parse-cache:
  Restore behavior of Software::register event
  Optimzie software framework version parsing
2021-09-21 18:00:46 +02:00
Robin Sommer
2fc12d5bed Merge branch 'topic/foxds/dce_itype_opnums' of ssh://github.com/fox-ds/zeek
* 'topic/foxds/dce_itype_opnums' of ssh://github.com/fox-ds/zeek:
  Add IType opnum mapping
2021-09-21 17:56:13 +02:00
Tim Wojtulewicz
5c0669d9a4 Merge remote-tracking branch 'origin/topic/christian/rehash-comphash'
* origin/topic/christian/rehash-comphash:
  Add btests for new functionality
  Remove unused HashKey constructor and reorder for consistency
  Refactor CompHash class to use new HashKey buffering features
  Add debug string and ODesc support to HashKey class
  Refactor HashKey class to support read/write operations
  Add unit tests for memory helpers
  Add memory sizing/alignment helpers to util.cc/h
  Ensure table/set HashKey buffer reservation and writes happen in same order
2021-09-21 07:45:38 -07:00
Christian Kreibich
cfcf1f83cc Add btests for new functionality
- Expand language.set to cover sets of sets
- Expand language.table to cover tables indexed with tables
- Add language.table-nested-set-ordering to capture the reproducer from GHI-1753
2021-09-20 17:51:43 -07:00
Christian Kreibich
10e8d36340 Remove unused HashKey constructor and reorder for consistency
One of the HashKey constructors was only used in the old CompHash code.
This aso reorders some constructors and the destructor for readability.
2021-09-20 17:51:43 -07:00
Christian Kreibich
4d58b98c70 Refactor CompHash class to use new HashKey buffering features
This preserves the previous hash key buffer layout (so the testsuite still
passes) and overall approach but gets rid of the codepath for writing singleton
serializations. This code path required a fourth switch block over all types
(besides reads, writes, and size computation) and was inconsistent with the one
for writing non-atomic types.
2021-09-20 17:51:43 -07:00
Christian Kreibich
b6a11a69db Add debug string and ODesc support to HashKey class
This allows tracing of hash key buffer reservations, reads, and writes via a new
debug stream, and supports printing a summary of a HashKey object via
Describe(). The latter comes in handy e.g. in TableVal::Describe() (where
including the hash key is now available but commented out).
2021-09-20 17:51:43 -07:00
Christian Kreibich
82822b1e07 Refactor HashKey class to support read/write operations
This preserves the optimization of storing values directly in the key_u member
union when feasible, and using a variable size buffer otherwise. It also adds
bounds-checking for that buffer, moves size arguments to size_t, decouples
construction from hash computation, emulates the tagging feature found in
SerializationFormat to assist troubleshooting, and switches feasible
reinterpret_casts to static_casts.
2021-09-20 17:51:43 -07:00
Christian Kreibich
2585ccd873 Add unit tests for memory helpers 2021-09-20 17:51:43 -07:00
Christian Kreibich
c5cceaf5ad Add memory sizing/alignment helpers to util.cc/h
This functionality previously lived in the CompHash class, with one difference:
this removes a discrepancy between the offset aligner and the memory pointer
aligner/padder. The size aligner used to align the provided offset and then add an
additional alignment size (for example, 1 aligned to 4 wouldn't yield 4 but 8).
Like the memory aligners it now only rounds up as needed.

Includes unit tests.
2021-09-20 17:51:43 -07:00
Christian Kreibich
5fc8d89897 Ensure table/set HashKey buffer reservation and writes happen in same order
This takes the existing sorting for table index hashkeys we had in place during
hash key writes and applies it also during buffer size reservation. It changes
the approach slightly: the underlying map now points to the TableVal entry index
vals directly, rather than to the numerical index into an additional list that
gets built up to store those indexes. Doing so removes the need for that list.
2021-09-20 17:51:43 -07:00
zeek-bot
1260f6b585 Update doc submodule [nomail] [skip ci] 2021-09-21 00:43:52 +00:00
Justin Azoff
ef5fb790ef Restore behavior of Software::register event
Use an intermediary event to ensure that software versions are parsed
before calling Software::register.
2021-09-20 14:38:47 -04:00
Robin Sommer
31d3fb0f6c Merge https://github.com/gpotter2/zeek.
Changes during merge:
  - Add dedicated test (w/ trace "client_timestamp_enabled.pcapng" from Cloudshark)
  - Change types from signed to unsigned.
  - Add cast for bit-shifting operand.
  - clang-format run
2021-09-20 11:41:29 +02:00
FOX-DS
4a19acbef2 Add IType opnum mapping 2021-09-20 03:56:20 -04:00
Tim Wojtulewicz
c28fd3b610 Merge remote-tracking branch 'origin/topic/vern/changes-news-updates'
* origin/topic/vern/changes-news-updates:
  elevate CHANGES descriptions and add a NEWS item for -O ZAM
2021-09-18 16:35:41 -07:00
Vern Paxson
95af08b8be elevate CHANGES descriptions and add a NEWS item for -O ZAM 2021-09-17 07:36:33 -07:00
gpotter2
d4db9bf6d0 Add TSval and TSecr to TCPSyn 2021-09-17 11:35:16 +02:00
zeek-bot
a05b14711f Update doc submodule [nomail] [skip ci] 2021-09-17 00:41:27 +00:00
Tim Wojtulewicz
9068b71750 Merge remote-tracking branch 'origin/topic/timw/386-clang-format'
* origin/topic/timw/386-clang-format:
  Minor fixes for build problems after reformatting
  Reformat the world
  Fixes for includes ahead of reformatting
  GH-386: Add clang-format config and scripts
2021-09-16 15:37:21 -07:00
Tim Wojtulewicz
30aa4621c5 Minor fixes for build problems after reformatting 2021-09-16 15:36:09 -07:00
Tim Wojtulewicz
b2f171ec69 Reformat the world 2021-09-16 15:35:39 -07:00
Tim Wojtulewicz
194cb24547 Fixes for includes ahead of reformatting 2021-09-16 15:35:11 -07:00
Tim Wojtulewicz
a3ba7f638b GH-386: Add clang-format config and scripts 2021-09-16 15:35:11 -07:00
Robin Sommer
6c128a21e1 Merge branch 'topic/foxds/dcerpc_ms-oaut' of ssh://github.com/fox-ds/zeek
* 'topic/foxds/dcerpc_ms-oaut' of ssh://github.com/fox-ds/zeek:
  Added four new opnum mappings for MS-OAUT IDispatch methods
2021-09-16 11:15:36 +02:00
zeek-bot
9c2b3e97b4 Update doc submodule [nomail] [skip ci] 2021-09-16 00:34:09 +00:00
Tim Wojtulewicz
0dca1a70a7 Reformat docs in addrs.zeek to fix doc generation 2021-09-14 19:26:28 -07:00
Tim Wojtulewicz
619af69e42 Merge remote-tracking branch 'origin/topic/vern/zam-coverity'
* origin/topic/vern/zam-coverity:
  still more nit-squashing
  avoiding using back() for an empty std::string
  additional tweaks to address some compiler warnings
  tweaks to address some compiler warnings
  tweaks to address concerns flagged by Coverity
2021-09-14 18:39:44 -07:00
Vern Paxson
81e8354ada still more nit-squashing 2021-09-14 16:04:04 -07:00
Vern Paxson
10d8c2fe6b avoiding using back() for an empty std::string 2021-09-14 13:51:46 -07:00
Tim Wojtulewicz
cc162e2ab0 Merge remote-tracking branch 'awelzel/table-val-contains'
* awelzel/table-val-contains:
  ignore_checksums_nets: Add test for multiple subnets
2021-09-14 13:01:03 -07:00
Arne Welzel
b7690bb367 ignore_checksums_nets: Add test for multiple subnets
This test fails with Zeek 4.1 and Zeek 4.0.3.
2021-09-14 21:17:01 +02:00
Vern Paxson
57fdef573c additional tweaks to address some compiler warnings 2021-09-14 11:45:50 -07:00
FOX-DS
d3ca226e96 Added four new opnum mappings for MS-OAUT IDispatch methods 2021-09-14 15:31:08 +02:00
Vern Paxson
693fc14eb2 tweaks to address some compiler warnings 2021-09-13 19:55:26 -07:00
Vern Paxson
7f3993ca0e tweaks to address concerns flagged by Coverity 2021-09-13 12:57:15 -07:00
Tim Wojtulewicz
8414d13030 Merge remote-tracking branch 'origin/topic/vern/vector-hole-loops'
* origin/topic/vern/vector-hole-loops:
  fix for looping over vectors with holes per https://github.com/zeek/zeek/issues/1763
  btest for https://github.com/zeek/zeek/issues/1763
2021-09-08 12:04:56 -07:00
Tim Wojtulewicz
b7e264f8ef Merge remote-tracking branch 'origin/topic/seth/tsv-logs-utf8-by-default'
* origin/topic/seth/tsv-logs-utf8-by-default:
  Fix mis-usage of string::append that leads to an overflow
  Use json_escape_utf8 for all utf8 data in ODesc
  Switch the TSV Zeek logs to be UTF8 by default.
2021-09-08 12:04:13 -07:00
Tim Wojtulewicz
a251aa07f7 Merge remote-tracking branch 'origin/topic/vern/ZAM-remainder'
* origin/topic/vern/ZAM-remainder: (37 commits)
  fix race condition in btest output ordering
  whoops, forgot to canonicalize filenames in new btest
  extend btest to include a coercion overflow
  fixed a typo in a comment
  fixes for vector coercion overflows, typing, and holes
  factoring out logic to check for overflows during coercions
  test case for vector coercions, including holes
  low-level cleanups found by code review
  additional conversions of size() to empty() checks that were missed previously
  indentation nit
  flag loop that has slightly subtle logic
  use ## to start major sections
  a number of low-level tweaks from code review
  use std::find_if rather than explicit loop
  switch simple loops that don't need indices to being iterator-based
  use container empty() rather than size() where appropriate
  Baseline variants for "-a zam"
  new "-a ZAM" testing baseline alternative
  updates for usage issues: support for -uu, maybe/definitely distinctions
  enable reducer to track folding to enable constant propagation
  ...
2021-09-08 11:44:35 -07:00
Vern Paxson
acaaa0d36a fix race condition in btest output ordering 2021-09-08 10:23:39 -07:00
Vern Paxson
74fbfa3aae whoops, forgot to canonicalize filenames in new btest 2021-09-08 10:23:39 -07:00
Vern Paxson
9d896d114b extend btest to include a coercion overflow 2021-09-08 10:23:39 -07:00
Vern Paxson
7f3ab84e5c fixed a typo in a comment 2021-09-08 10:23:39 -07:00
Vern Paxson
196b8767e8 fixes for vector coercion overflows, typing, and holes 2021-09-08 10:23:39 -07:00