Commit graph

10798 commits

Author SHA1 Message Date
Jon Siwek
ba1611c21f Adjust Dict whitespace/style 2020-08-11 11:26:22 -07:00
Jon Siwek
ccf8e271a1 Adjust more btest timings 2020-08-11 11:26:22 -07:00
Jon Siwek
ec8c24b13b Improve termination reliability/speed for brokerstore btests 2020-08-11 11:26:22 -07:00
Jon Siwek
7967a5b0aa General btest cleanup
- Use `-b` most everywhere, it will save time.

- Start some intel tests upon the input file being fully read instead of
  at an arbitrary time.

- Improve termination condition for some sumstats/cluster tests.

- Filter uninteresting output from some supervisor tests.

- Test for `notice_policy.log` is no longer needed.
2020-08-11 11:26:22 -07:00
Jon Siwek
670bf02c95 Update NEWS about change in Dictionary implementation 2020-08-11 11:26:21 -07:00
Jon Siwek
3ad306355f Improve Intel expire-item btest to be less time-sensitive 2020-08-11 11:26:21 -07:00
Jon Siwek
af2262ddff Improve btests with unstable table/set output ordering
Particularly, the final output order of a table/set is sensitive to
order of input/insertions and some tests were converting
std::unordered_{set,map} to Zeek table/set and iteration over those
standard containers may not always loop through elements in the same
order across all platforms.
2020-08-11 11:26:21 -07:00
Jon Siwek
89b08a3983 Update doc submodule 2020-08-11 11:26:21 -07:00
Jon Siwek
48a243069c Adjust a few btests that were unstable due to time-sensitivity 2020-08-11 11:26:21 -07:00
Jon Siwek
fef4fda2f7 Fix DNS script deleting a table element while iterating
Doesn't typically cause any problems since the loop breaks after
deleting, except there's now an assert in debug builds catching
potential problems like this.
2020-08-11 11:26:21 -07:00
Jon Siwek
f613cb0c68 Improve a brokerstore btest to filter out Broker connection messages 2020-08-11 11:26:21 -07:00
Jon Siwek
b96b1593d8 Sort output of a few SumStats cluster tests
The order of $epoch_result() function calls among all keys within a
single epoch isn't consequential.
2020-08-11 11:26:21 -07:00
Jon Siwek
170752fa99 Fix extract_first_email_addr() to really return the first email
The use of find_all() in extract_email_addrs_vec() extracted occurrences
to an intermediate set and thus lost any sense of ordering.

This changes extract_email_addrs_vec() to use find_all_ordered() and
return all occurrences of email addresses found in the argument,
included duplicates, with their order of occurrence preserved.
2020-08-11 11:26:21 -07:00
Jon Siwek
a852ab4c39 Add find_all_ordered() BIF
Operates similar to find_all(), except returns a vector instead of
set to allow preservation of order/duplicates.
2020-08-11 11:26:21 -07:00
Jon Siwek
67835a6520 Extend external test suite canonifier with set-sorting logic
Two new canonifiers: one to sort the contents of conn.log "service"
field and another to sort the contents of any field of type "set".
2020-08-11 11:26:21 -07:00
Jon Siwek
94aee910d5 Update btests/baselines for OpenDict compat
Haven't checked different build configurations yet, but all except
a few SumStats tests are stable for me now.  The external tests
are also completely failing, but haven't looked at those yet.
2020-08-11 11:26:21 -07:00
Jon Siwek
37d3cfbd44 Fix new/malloc/delete/free mismatches in Dictionary code 2020-08-11 11:26:21 -07:00
Jon Siwek
d566b8ca31 Add explanation for a Dict TODO item 2020-08-11 11:26:21 -07:00
Jon Siwek
1d33dbc6bc Fix compiler warning in Dictionary debug/dump printf format string 2020-08-11 11:26:21 -07:00
Tim Wojtulewicz
0c7e4d62ee Overwrite old Dict.h/cc with new ones 2020-08-11 11:26:21 -07:00
Tim Wojtulewicz
78298e99b7 Extend the timeouts on a few intel tests, update baselines 2020-08-11 11:26:21 -07:00
Tim Wojtulewicz
30fa514d30 Updating test baselines for new dictionary code due to changes in ordering of fields in the dictionary 2020-08-11 11:26:21 -07:00
Tim Wojtulewicz
351aca5e52 Add namespaces for OpenDict files 2020-08-11 11:26:21 -07:00
Tim Wojtulewicz
59d5c85903 Add unit tests from Dict into OpenDict files 2020-08-11 11:26:21 -07:00
Tim Wojtulewicz
32653c86af Build fixups
- Fix some clang-tidy warnings
- Add std:: namespaces for ofstream and ios
- Remvoe some unused methods
2020-08-11 11:26:21 -07:00
jasonlue
394e067f7b Massive formatting cleanup 2020-08-11 11:26:21 -07:00
jasonlue
2a0609431c Remove key reference after insertion because the key may become invalid afterwards. 2020-08-11 11:26:21 -07:00
jasonlue
d3eeeb0f4c add Clustered Hashing based Open Addressing Dict. To replace the existing dict, #define USE_OPEN_DICT 2020-08-11 11:26:21 -07:00
Tim Wojtulewicz
5d764d6678 GH-178: Add new string bif methods based on python string utilities 2020-08-11 10:40:30 -07:00
Jon Siwek
6e7ff1e980 Update submodule(s)
[nomail]
2020-08-10 14:13:32 -07:00
Jon Siwek
747c681ecf Merge remote-tracking branch 'origin/topic/jsiwek/gh-1100-multipart-body-length'
* origin/topic/jsiwek/gh-1100-multipart-body-length:
  GH-1100: Fix reported body-length of HTTP messages w/ sub-entities
2020-08-10 09:58:01 -07:00
Tim Wojtulewicz
a243c0e4a6 Overwrite old Dict.h/cc with new ones 2020-08-09 21:27:23 -07:00
Tim Wojtulewicz
b8287a3375 Fix issue with sumstats script and fix baselines that were crashing previously 2020-08-09 21:13:12 -07:00
Tim Wojtulewicz
7496cf14c7 Extend the timeouts on a few intel tests, update baselines 2020-08-09 21:13:12 -07:00
Tim Wojtulewicz
21872aef39 Updating test baselines for new dictionary code due to changes in ordering of fields in the dictionary 2020-08-09 21:13:10 -07:00
Tim Wojtulewicz
10e4694f8e Add namespaces for OpenDict files 2020-08-09 21:12:19 -07:00
Tim Wojtulewicz
8f0e8037ba Add unit tests from Dict into OpenDict files 2020-08-09 21:12:19 -07:00
Tim Wojtulewicz
ed0f9f3564 Build fixups
- Fix some clang-tidy warnings
- Add std:: namespaces for ofstream and ios
- Remvoe some unused methods
2020-08-09 21:12:19 -07:00
jasonlue
a2476ed91d Massive formatting cleanup 2020-08-09 21:12:19 -07:00
jasonlue
b0bd432f57 Remove key reference after insertion because the key may become invalid afterwards. 2020-08-09 21:12:19 -07:00
jasonlue
19f063f2a3 add Clustered Hashing based Open Addressing Dict. To replace the existing dict, #define USE_OPEN_DICT 2020-08-09 21:12:19 -07:00
Justin Azoff
211789bbb3 Accept bracketed ipv6 addresses 2020-08-05 17:18:36 -04:00
Jon Siwek
8f4138262f Merge remote-tracking branch 'origin/topic/jsiwek/fix-fuzz-target-warnings'
* origin/topic/jsiwek/fix-fuzz-target-warnings:
  Fix deprecation warnings in fuzz targets
  Remove dbg_read_internal_state()
2020-08-05 12:36:33 -07:00
Jon Siwek
5d5adcbffd Fix deprecation warnings in fuzz targets 2020-08-04 16:28:19 -07:00
Jon Siwek
c8c2c6952f Remove dbg_read_internal_state()
It's never had a definition anyway and the namespacing/deprecation
process was now causing a compile error in debug/fuzzing builds for me w/ GCC:

src/fuzzers/libzeek_fuzzer_shared.so: error: undefined reference to 'zeek::detail::dbg_read_internal_state()'
2020-08-04 16:23:26 -07:00
Jon Siwek
363b167bd2 GH-1100: Fix reported body-length of HTTP messages w/ sub-entities
The body-lengths of sub-entities, like multipart messages, got counted
twice by mistake: once upon the end of the sub-entity and then again
upon the end of the top-level entity that contains all sub-entities.
The size of just the top-level entity is the correct one to use.
2020-08-04 14:21:03 -07:00
Johanna Amann
8e99d4b170 Merge remote-tracking branch 'origin/topic/jsiwek/gh-1080-error-on-enum-redefinition-conflict'
* origin/topic/jsiwek/gh-1080-error-on-enum-redefinition-conflict:
  Fix incorrect conflict detection of namespaced-enum-names
  Improve error message for an enum name conflicting with non-enum ID
  GH-1080: Treat enum name re-use across different enum types as an error
  GH-1080: Rename conflicting NetControl::DROP enum definitions

Fixes GH-1080
2020-08-04 20:25:40 +00:00
Johanna Amann
e31dda6989 Update submodule
[nomail]
2020-08-04 13:08:57 -07:00
Seth Hall
7f4914c219 Fix config reader regular expression for MUSL.
It was not dealing with multiple spaces between the key and the value
with MUSL correctly. This change ensures that if a value exists, that it
begins and ends with a non-blank character.
2020-08-04 12:35:20 -04:00
Tim Wojtulewicz
dbd8a4fc9f Merge remote-tracking branch 'origin/topic/jsiwek/silence-type-counter-warnings'
* origin/topic/jsiwek/silence-type-counter-warnings:
  Silence compiler warnings related to deprecated TYPE_COUNTER
2020-08-04 08:47:26 -07:00