Commit graph

246 commits

Author SHA1 Message Date
Tim Wojtulewicz
271d3c4038 GH-2054: Allow nulls as separators for join_string_vec 2022-05-25 17:03:26 -07:00
Tim Wojtulewicz
aaaca5d5a6 Merge remote-tracking branch 'origin/topic/bbannier/spicy'
* origin/topic/bbannier/spicy:
  Add NEWS item for new requirements due to default-enabled Spicy support.
  Include headers in binary dir before other headers.
  Increase memory in CI.
  Add tests for bundled Spicy infrastructure.
  Include spicy in build.
  Add `bare_mode` bif.
  Change test so included plugins can load notice framework.
  Do not log function arguments in test.
2022-05-17 08:50:33 -07:00
Tim Wojtulewicz
55297a2e73 Merge remote-tracking branch 'origin/topic/timw/2103-bifs-footprint-leak'
* origin/topic/timw/2103-bifs-footprint-leak:
  GH-2103: Disable leak detection for bifs.footprint in another way
2022-05-16 09:09:17 -07:00
Benjamin Bannier
e83c4c834d Add bare_mode bif. 2022-05-16 09:07:11 +02:00
Tim Wojtulewicz
bd1ea5edda Merge remote-tracking branch 'origin/topic/johanna/bloomfilter'
* origin/topic/johanna/bloomfilter:
  Bloom filter changes: address review feedback
  Bloom-filter: fix documentation syntax problem
  Bloomfilter related clang-format fixes
  Correcly typify bloom filters in all cases during merge
  Add intersect operation for bloom filters
  Add bloomfilter_decrement bif
2022-05-13 16:49:13 -07:00
Tim Wojtulewicz
56fe393691 GH-2103: Disable leak detection for bifs.footprint in another way 2022-05-13 16:26:41 -07:00
Vern Paxson
a387157ead skip new BiF test for ASAN CI runs 2022-05-05 17:07:46 -07:00
Vern Paxson
d5f60cfaba btest update to include recursive value that doesn't require a record 2022-05-05 16:55:03 -07:00
Johanna Amann
42bc6db359 Merge remote-tracking branch 'origin/master' into topic/johanna/bloomfilter 2022-05-03 16:43:36 +01:00
Vern Paxson
edf276520a make including count of container elements non-optional 2022-04-29 09:03:35 -07:00
Vern Paxson
b670046a69 btest for mutually-recursive case 2022-04-29 08:44:58 -07:00
Vern Paxson
16c37034de added value_footprint() and global_container_footprints() BiFs 2022-04-28 16:42:14 -07:00
Tim Wojtulewicz
2c9296120e Merge remote-tracking branch 'ynadji/add-enum-to-int-ordering-test'
* ynadji/add-enum-to-int-ordering-test:
  Add test to ensure enum_to_int's return values are ordered
2022-04-13 10:53:07 -07:00
Yacin Nadji
8e225efd57 Add test to ensure enum_to_int's return values are ordered 2022-04-11 13:10:36 -04:00
Tim Wojtulewicz
2386b650cd Merge remote-tracking branch 'ynadji/raw_bytes_to_v6_addr'
* ynadji/raw_bytes_to_v6_addr:
  add raw_bytes_to_v6_addr in docs when raw_bytes_to_v4_addr is present
  Zero out bytes by default for consistent return value on error
  Add tests for raw_bytes_to_v6_addr
  Add raw_bytes_to_v6_addr function
2022-03-10 11:09:49 -07:00
Yacin Nadji
45ab85c09e Add tests for raw_bytes_to_v6_addr 2022-03-08 17:50:26 -05:00
Christian Kreibich
19bfa071e0 Expand testcases around is_num(), is_alpha(), is_alnum(), is_ascii() BiFs 2022-02-28 13:09:32 -08:00
Johanna Amann
796e18ecfc Add intersect operation for bloom filters
Intersecting two bloom filters yields a bloom filter that returns true
when an element was contained in both bloom filters. The false positive
rate is potentially a bit higher than in the original bloom filters.

This operation also works for counting bloom filters, however the
counters are discarded and the bloomfilters are converted to basic bloom
filters. The reason is that there is no obvious meaning to the counters
when two bloom filters are intersected - besides the fact if an element
was inserted at all.
2022-01-20 13:34:07 +00:00
Johanna Amann
aa58b6b37b Add bloomfilter_decrement bif
This bif implements the decrement operation for counting bloom filters.

It also clarifies some of the documentation.
2022-01-18 17:58:10 +00:00
Tim Wojtulewicz
7de9a7d76f GH-1860: Add double_to_int() bif 2021-12-13 10:46:14 -07:00
Tim Wojtulewicz
612212568a Add analyzer_confirmation and analyzer_violation events 2021-11-23 19:36:50 -07:00
Johanna Amann
d451fb8184 Merge branch 'topic/AbdelSaTd/case-insensitive-find'
* topic/AbdelSaTd/case-insensitive-find:
  testing-do-find-str_case-insensitive
  case-insensitive-search-features-for-do_find_str

I did a few small style-fixes while merging this and expanded the test
a bit. I also removed unecessary commits.

Closes GH-1828
2021-11-10 11:33:47 +00:00
Abdel
effa8c403f testing-do-find-str_case-insensitive 2021-11-10 11:25:20 +00:00
Tim Wojtulewicz
ceaec09024 GH-693: use pcap_dump_open_append where supported 2021-11-02 17:09:39 -07:00
Johanna Amann
e4b2fa50a9 Merge remote-tracking branch 'origin/master' into topic/johanna/gh-859 2021-06-29 15:09:56 +01:00
Johanna Amann
833168090a Add ability to check if hostname is valid for a specific cert
This commit adds two new bifs, x509_check_hostname and
x509_check_cert_hostname. These bifs can be used to check if a given
hostname which can, e.g., be sent in a SNI is valid for a specific
certificate.

This PR furthermore modifies the ssl logs again, and adds information
about this to the log-file. Furthermore we now by default remove the
server certificate information from ssl.log - I doubt that this is often
looked at, it is not present in TLS 1.3, we do still have the SNI, and
if you need it you have the information in x509.log.

This also fixes a small potential problem in X509.cc assuming there
might be SAN-entries that contain null-bytes.

Baseline update will follow in another commit.
2021-06-29 15:00:48 +01:00
Vern Paxson
b6e9776a11 updates to test suite tests for compatibility with upcoming ZAM functionality 2021-06-01 09:25:30 -07:00
Tim Wojtulewicz
ad67d810be Merge remote-tracking branch 'origin/topic/jsiwek/fix-netbios-decode-bifs'
* origin/topic/jsiwek/fix-netbios-decode-bifs:
  Fixes to `decode_netbios_name` and `decode_netbios_name_type` BIFs
2021-04-30 09:40:46 -07:00
Jon Siwek
76fb1e7fd0 Fixes to decode_netbios_name and decode_netbios_name_type BIFs
Fixes to `decode_netbios_name`:

* Improve validation that input string is a NetBIOS encoding
  (32 bytes, with characters ranging from 'A' to 'P').  This helps
  prevent Undefined Behavior of left-shifting negative values.
  Invalid encodings now cause a return-value of an empty string.

* More liberal in what decoded characters are allowed.  Namely,
  spaces are now allowed (but any trailing null-bytes and spaces
  are trimmed, similar to before).

Fixes to `decode_netbios_name_type`:

* Improve validation that input string is a NetBIOS encoding
  (32 bytes, with characters ranging from 'A' to 'P').  This helps
  prevent Undefined Behavior of left-shifting negative values and
  a heap-buffer-overread when the input string is too small.
  Invalid encodings now cause a return-value of 256.
2021-04-27 15:27:04 -07:00
Jon Siwek
d51bd4bc46 Fix using clear_table() within an &expire_func
This previously crashed since clear_table()/TableVal::RemoveAll() left
behind a stale iterator to the old table causing a heap-use-after-free
when resuming table expiry iteration in TableVal::DoExpire().
2021-04-26 22:49:44 -07:00
Jon Siwek
3e9ac00f21 Separate stdout from stderr in btest baselines
Redirecting both to the same file can show platform-specific differences
(e.g. Alpine), likely due to different buffering defaults.
2021-03-30 16:23:23 -07:00
Vern Paxson
98ada4e419 test suite updates for expanded sort() BiF semantics (bools, doubles) 2021-02-25 17:13:50 -08:00
Tim Wojtulewicz
7292b52f66 Remove some deprected methods/events from bif files 2021-01-27 10:52:40 -07:00
Vern Paxson
bd2c94521a split bifs.string_utils into a non-error test and an only-errors test, to help control for differing error propagation 2021-01-10 14:21:51 -08:00
Christian Kreibich
b04082c140 Additional use of btest-diff --binary 2020-12-06 20:19:52 -08:00
Yacin Nadji
607af85ac1 Add count_to_double and int_to_double bif functions 2020-12-01 16:35:49 -05:00
Jon Siwek
f147c47271 Add enum_names() BIF to return names of an enum type's values 2020-11-06 18:31:41 -08:00
Jon Siwek
77514f234f Add type_aliases() BIF for introspecting type-names of types/values 2020-11-06 17:18:44 -08:00
Jon Siwek
bfb7afc600 Fix lookup_ID() BIF to return enum values
Looking up an enum value from a string equal to its name previously
returned "<no ID value>".
2020-11-06 15:37:54 -08:00
Jon Siwek
a96440ea79 GH-1155: Recursively check table index for unsupported types
Previously, container types used within a table/set index were not
deeply checked to ensure all constituents could be part of an index.
2020-09-14 19:06:11 -07:00
Tim Wojtulewicz
b89935107d Merge remote-tracking branch 'origin/topic/timw/178-string-functions'
* origin/topic/timw/178-string-functions:
  GH-178: Add new string bif methods based on python string utilities
2020-08-14 10:00:29 -07:00
Jon Siwek
7967a5b0aa General btest cleanup
- Use `-b` most everywhere, it will save time.

- Start some intel tests upon the input file being fully read instead of
  at an arbitrary time.

- Improve termination condition for some sumstats/cluster tests.

- Filter uninteresting output from some supervisor tests.

- Test for `notice_policy.log` is no longer needed.
2020-08-11 11:26:22 -07:00
Jon Siwek
a852ab4c39 Add find_all_ordered() BIF
Operates similar to find_all(), except returns a vector instead of
set to allow preservation of order/duplicates.
2020-08-11 11:26:21 -07:00
Tim Wojtulewicz
5d764d6678 GH-178: Add new string bif methods based on python string utilities 2020-08-11 10:40:30 -07:00
Jon Siwek
887b53b7f3 GH-1076: Fix bro_srandom() to replace 0 seeds with 1
The bro_prng() implementation cannot generate 0 as a result since it
causes every subsequent number from the PRNG to also be 0, so use the
number 1 instead of 0.
2020-07-22 14:01:33 -07:00
Jon Siwek
0f4eb9af02 GH-1076: Fix bro_prng() implementation
The intermediate result of the PRNG used unsigned storage, preventing
the ( result < 0 ) branch from ever being evaluated.  This could cause
return values to exceed the modulus as well as RAND_MAX.

One interesting effect of this is potential for the rand() BIF to
return values outside the requested maximum limit.

Another interesting effect of this is that a PacketFilter may start
randomly dropping packets even if it was not configured for
random-packet-drops.
2020-07-22 14:01:33 -07:00
Jon Siwek
dba764386b GH-1076: Fix use of getrandom()
The availability and use of getrandom() actually caused unrandom and
deterministic results in terms of Zeek's random number generation.
2020-07-22 14:01:33 -07:00
Johanna Amann
66f586c02c Update Mozilla CA & Google CT lists 2020-07-17 23:26:37 +00:00
Jon Siwek
6908d1b919 GH-1019: deprecate icmp_conn params for ICMP events
Previously, a single `icmp_conn` record was built per ICMP "connection"
and re-used for all events generated from it.  This may have been a
historical attempt at performance optimization, but:

  * By default, Zeek does not load any scripts that handle ICMP events.

  * The one script Zeek ships with that does handle ICMP events,
    "detect-traceroute", is already noted as being disabled due to
    potential performance problems of doing that kind of analysis.

  * Re-use of the original `icmp_conn` record tends to misreport
    TTL and length values since they come from original packet instead
    of the current one.

  * Even if we chose to still re-use `icmp_conn` records and just fill
    in a new TTL and length value each packet, a user script could have
    stored a reference to the record and not be expecting those values
    to be changed out from underneath them.

Now, a new `icmp_info` record is created/populated in all ICMP events
and should be used instead of `icmp_conn`.  It also removes the
orig_h/resp_h fields as those are redundant with what's already
available in the connection record.
2020-07-10 11:06:28 -07:00
Jon Siwek
7b15b82009 Merge remote-tracking branch 'origin/topic/timw/1040-str-split'
* origin/topic/timw/1040-str-split:
  GH-1040: Add zero-indexed version of str_split
2020-07-06 21:06:51 -07:00