Commit graph

13262 commits

Author SHA1 Message Date
Christian Kreibich
4b5584a85d Move redefs of ClusterController::Request::Request to their places of use
The Request module does not need to know about additional state tucked onto it
by its users.
2022-01-31 18:29:58 -08:00
Christian Kreibich
f9ac03d6e3 Simplify ClusterController::API::set_configuration_request/response
It's easier to track outstanding controller/agent requests via a simple set of
pending agent names, and we can remove all of the result aggregation logic since
we can simply re-use the results reported by the agents.

This can serve as a template for request-response patterns where a client's
request triggers a request to all agents, followed by a response to the client
once all agents have responded. Once we have a few more of those, it'll become
clearer how to abstract this further.
2022-01-31 17:45:14 -08:00
Robin Sommer
9b0d525728
Let our TCP-based application analyzers operate without any TCP parent analyzer.
Conceptually, a TCP-based application analyzer should not need any
knowledge about the underlying TCP analysis; it's supposed to just
process its reassembled input stream as it's handed over. But our
analyzers break that assumption at a few places because sometimes
knowledge about the TCP state of the connection can be helpful for
heuristics. This is fine as long as there actually *is* a TCP parent
analyzer available. Sometimes, however, there isn't: if the payload
stream is encapsulated inside another application-layer protocol, the
semantic link to TCP is broken. And if the outer connection is even
UDP, then we don't have a TCP analyzer at all.

We didn't handle this situation well so far. Most analyzers needing
TCP state would just crash if there's no TCP analyzer (in debug mode
with an `assert`, in release mode with a null pointer deref ...). Only
HTTP did the right thing already: check if TCP is available and adapt
accordingly.

We know extend that check to all other analyzers as well: all accesses
to `TCP()` are guarded, with reasonable defaults if not available.
It's actually a pretty small change overall, which is evidence for how
little this layering violation actually matters.

The existing behavior is what's causing
https://github.com/corelight/zeek-spicy-openvpn/issues/3.
2022-01-28 16:25:27 +01:00
Tim Wojtulewicz
0793a38cc5 Merge remote-tracking branch 'origin/topic/vern/vec-slice-bug'
* origin/topic/vern/vec-slice-bug:
  btest to catch vector slice assignment issues
  bug fix for vector slice assignment
2022-01-25 13:52:19 -07:00
Vern Paxson
c378017269 btest to catch vector slice assignment issues 2022-01-25 10:47:33 -08:00
Vern Paxson
63bfae180a bug fix for vector slice assignment 2022-01-25 10:47:15 -08:00
Johanna Amann
5411c431dc Merge remote-tracking branch 'origin/topic/vern/de-bro-comments'
* origin/topic/vern/de-bro-comments:
  updated Bro->Zeek in comments in the source tree
2022-01-25 12:26:13 +00:00
Vern Paxson
d758585e42 updated Bro->Zeek in comments in the source tree 2022-01-24 14:26:20 -08:00
Christian Kreibich
fab8f9f2fe Merge branch 'topic/christian/ci-updates'
* topic/christian/ci-updates:
  CI update: add Ubuntu 21.10
  CI update: remove OpenSUSE Leap 15.2 (EOL)
  CI update: add CentOS Stream 9
2022-01-24 13:42:01 -08:00
Christian Kreibich
8e24f84857 CI update: add Ubuntu 21.10 2022-01-24 13:40:11 -08:00
Christian Kreibich
1de58030f2 CI update: remove OpenSUSE Leap 15.2 (EOL) 2022-01-24 13:40:11 -08:00
Christian Kreibich
404c43e36c CI update: add CentOS Stream 9 2022-01-24 13:40:11 -08:00
Zeke
7ec2fa2ac9 Consistently warn about mixing vector and scalar operand depreciaton
Resolves #1890
2022-01-21 14:34:48 -08:00
Johanna Amann
dfd79cc785 Correcly typify bloom filters in all cases during merge
Before, there were edge-cases where the type was not transferred if one
of the involved bloom-filters was untypified.

Fixes GH-1938
2022-01-20 13:41:20 +00:00
Johanna Amann
796e18ecfc Add intersect operation for bloom filters
Intersecting two bloom filters yields a bloom filter that returns true
when an element was contained in both bloom filters. The false positive
rate is potentially a bit higher than in the original bloom filters.

This operation also works for counting bloom filters, however the
counters are discarded and the bloomfilters are converted to basic bloom
filters. The reason is that there is no obvious meaning to the counters
when two bloom filters are intersected - besides the fact if an element
was inserted at all.
2022-01-20 13:34:07 +00:00
Johanna Amann
aa58b6b37b Add bloomfilter_decrement bif
This bif implements the decrement operation for counting bloom filters.

It also clarifies some of the documentation.
2022-01-18 17:58:10 +00:00
Johanna Amann
b78f30339f TLS decryption: refactoring, more comments, less bare pointers
This commit refactors TLS decryption, adds more comments in scripts and
in C++ source-code, and removes use of bare pointers, instead relying
more on stl data types.
2022-01-17 15:04:44 +00:00
Johanna Amann
689b06d9bd Merge remote-tracking branch 'origin/master' into topic/johanna/tls12-decryption 2022-01-17 10:56:06 +00:00
Johanna Amann
becc966106 Merge remote-tracking branch 'origin/topic/vern/vec-del-bug'
* origin/topic/vern/vec-del-bug:
  new btest to catch regressions to previous behavior/crash
  fix for adding a non-managed type to an empty vector
2022-01-17 10:35:51 +00:00
Vern Paxson
d6a68ada22 new btest to catch regressions to previous behavior/crash 2022-01-14 14:10:15 -08:00
Tim Wojtulewicz
3d9d6e953b Merge remote-tracking branch 'origin/topic/vern/when-lambda'
* origin/topic/vern/when-lambda:
  explicitly provide the frame for evaluating a "when" timeout expression
  attempt to make "when" btest deterministic
  tests for new "when" semantics/errors
  update existing test suite usage of "when" statements to include captures
  update uses of "when" in base scripts to include captures
  captures for "when" statements update Triggers to IntrusivePtr's and simpler AST traversal introduce IDSet type, migrate associated "ID*" types to "const ID*"
  logic (other than in profiling) for assignments that yield separate values
  option for internal use to mark a function type as allowing non-expression returns
  removed some now-obsolete profiling functionality
  minor commenting clarifications
2022-01-14 14:41:42 -07:00
Tim Wojtulewicz
c6078cd6d5 Merge remote-tracking branch 'origin/topic/johanna/fedora-33-eol'
* origin/topic/johanna/fedora-33-eol:
  Remove Fedora 33 from CI.
2022-01-14 12:22:19 -07:00
Tim Wojtulewicz
cdf39840bb Merge remote-tracking branch 'origin/topic/timw/make-dist-cleanup'
* origin/topic/timw/make-dist-cleanup:
  Have `make dist` cleanup a few more wayward files before tarring
2022-01-14 12:21:34 -07:00
Tim Wojtulewicz
a26e2f511b Have make dist cleanup a few more wayward files before tarring 2022-01-14 12:20:24 -07:00
Johanna Amann
1f52366ffc Merge remote-tracking branch 'origin/topic/johanna/StringValconst'
* origin/topic/johanna/StringValconst:
  Make some StringVal functions const
2022-01-14 16:00:23 +00:00
Johanna Amann
3a3f9af40d Make some StringVal functions const
I noticed these while working on the TLS changes - I do not see a good
reason why they should not be const.
2022-01-14 11:04:11 +00:00
Vern Paxson
57f6487111 fix for adding a non-managed type to an empty vector 2022-01-12 09:39:02 -08:00
Tim Wojtulewicz
3932fd930f Merge remote-tracking branch 'origin/topic/timw/fix-freebsd-jemalloc'
* origin/topic/timw/fix-freebsd-jemalloc:
  Fix --with-jemalloc configure option on FreeBSD
2022-01-12 10:16:46 -07:00
Tim Wojtulewicz
059120003f Fix --with-jemalloc configure option on FreeBSD 2022-01-12 10:16:15 -07:00
Tim Wojtulewicz
2accf6dfbf Merge remote-tracking branch 'origin/topic/vern/CPP-type-switch'
* origin/topic/vern/CPP-type-switch:
  support for compiling type-based switches to C++
  make encountering uncompilable "standalone" functions to be a hard error
  fix: ZAM could misinterpret a "type" switch that starts with a "default"
  fix: don't treat pseudo-identifiers in type cases as local variables
  removed development helper scripts now obsolete with --optimize-files=
2022-01-11 15:02:52 -07:00
Tim Wojtulewicz
ce9ca91d6e Merge remote-tracking branch 'origin/topic/timw/add-null-check'
* origin/topic/timw/add-null-check:
  Bump CI sanitizer task to Ubuntu 20.04
  Fix uninitialized variable warning
  Enable null check for UBSan
2022-01-11 14:58:13 -07:00
Tim Wojtulewicz
62ce62085b Bump CI sanitizer task to Ubuntu 20.04 2022-01-11 13:25:23 -07:00
Johanna Amann
543c992e66 Small code fix and test baseline update.
After this, tests hopefully should pass consistently.
2022-01-11 11:20:05 +00:00
Johanna Amann
304a06bb88 Merge remote-tracking branch 'origin/master' into topic/johanna/tls12-decryption 2022-01-11 11:04:20 +00:00
Johanna Amann
76a0cad1d0 Remove Fedora 33 from CI.
EOL date was 2021‑11‑30
2022-01-11 11:02:00 +00:00
zeek-bot
1dfc9176d5 Update doc submodule [nomail] [skip ci] 2022-01-11 00:37:24 +00:00
Tim Wojtulewicz
9834793386 Fix uninitialized variable warning 2022-01-10 17:06:39 -07:00
Tim Wojtulewicz
c786dc8f48 Enable null check for UBSan 2022-01-10 11:11:04 -07:00
Tim Wojtulewicz
ebc4d45004 Merge remote-tracking branch 'origin/topic/vern/nil-closure-fix'
* origin/topic/vern/nil-closure-fix:
  fix for avoiding de-ref of nil pointer
2022-01-10 10:31:13 -07:00
Tim Wojtulewicz
e9803113d1 NEWS: move note for http header changes into 4.2 2022-01-10 10:22:32 -07:00
Robin Sommer
964293209b
Merge remote-tracking branch 'origin/topic/robin/gh1844-host'
* origin/topic/robin/gh1844-host:
  Fix host header normalization in intel framework.
  Switch to recording unmodified HTTP header.
2022-01-10 14:43:30 +01:00
Vern Paxson
98a05538b7 explicitly provide the frame for evaluating a "when" timeout expression 2022-01-08 08:24:15 -08:00
Vern Paxson
fa848167bb attempt to make "when" btest deterministic 2022-01-07 21:52:11 -08:00
Vern Paxson
253630311e fix for avoiding de-ref of nil pointer 2022-01-07 18:18:00 -08:00
Vern Paxson
c5ab916710 tests for new "when" semantics/errors 2022-01-07 14:54:47 -08:00
Vern Paxson
b59ee83979 update existing test suite usage of "when" statements to include captures 2022-01-07 14:54:06 -08:00
Vern Paxson
98cd3f2213 update uses of "when" in base scripts to include captures 2022-01-07 14:53:33 -08:00
Vern Paxson
f895008c34 captures for "when" statements
update Triggers to IntrusivePtr's and simpler AST traversal
introduce IDSet type, migrate associated "ID*" types to "const ID*"
2022-01-07 14:50:35 -08:00
Vern Paxson
fa142438fe logic (other than in profiling) for assignments that yield separate values 2022-01-07 14:18:42 -08:00
Vern Paxson
e22d279fdf option for internal use to mark a function type as allowing non-expression returns 2022-01-07 11:50:40 -08:00