Commit graph

16273 commits

Author SHA1 Message Date
Christian Kreibich
4e45a3462b Update btest baselines to reflect introduction of mmdb.bif 2024-01-10 20:28:41 -08:00
Christian Kreibich
8406959ae2 Move MaxMind/GeoIP BiF functionality into separate file 2024-01-10 20:28:37 -08:00
Christian Kreibich
06642d185b Provide script-level configurability of MaxMind DB placement on disk
This lifts the list of fallback directories in which Zeek will look for Maxmind
DBs into the script layer, and makes the names of the DB files themselves
(previously hardwired) configurable as well.

This does not yet change the in-core code; that commit follows.
2024-01-10 20:14:24 -08:00
Christian Kreibich
b4725c113f Sort toplevel .bif list in CMakeLists
Not important, but here it is safe to do so and the list is getting lengthy.
2024-01-10 20:14:24 -08:00
zeek-bot
4c960fd7d5 Update doc submodule [nomail] [skip ci] 2024-01-11 00:12:54 +00:00
Tim Wojtulewicz
2b4005b820 Merge branch 'topic/timw/move-bifs'
* topic/timw/move-bifs:
  Use std::move in return values from bif methods to avoid copies
  Use bool return values instead of int in a couple zeek.bif static methods
2024-01-10 12:32:58 -07:00
Tim Wojtulewicz
010306f6f6 Use std::move in return values from bif methods to avoid copies 2024-01-10 12:32:15 -07:00
Tim Wojtulewicz
d0cb3888b4 Use bool return values instead of int in a couple zeek.bif static methods 2024-01-10 12:32:15 -07:00
Arne Welzel
bddd74dcc1 Merge remote-tracking branch 'origin/topic/awelzel/quic-draft-mvfst-versions'
* origin/topic/awelzel/quic-draft-mvfst-versions:
  quic: Handle and log unhandled_version
  quic: Support decryption of a few more versions
2024-01-10 14:08:01 +01:00
Arne Welzel
8de7808cf9 Merge remote-tracking branch 'origin/topic/awelzel/3531-at-if-record-definition'
* origin/topic/awelzel/3531-at-if-record-definition:
  parse: Support @if conditionals in record definitions
2024-01-10 14:05:22 +01:00
Arne Welzel
60d8872969 Known: Keep &create_expire on local tables/sets valid
After switching the known scripts away from broker stores, the
&create_expire value of the local tables/sets of the known scripts
wasn't in effect due to Cluster::node_up() and Cluster::node_down()
re-assigning these without keeping the &create_expire attribute
intact. This broke the "log hosts every 24h" behavior.

Closes #3540
2024-01-10 09:07:43 +01:00
Arne Welzel
ec9ed81250 quic: Handle and log unhandled_version 2024-01-09 17:10:11 +01:00
Arne Welzel
4ca6f690d7 quic: Support decryption of a few more versions 2024-01-09 17:10:11 +01:00
Arne Welzel
f1e94594f9 Merge remote-tracking branch 'origin/topic/awelzel/3523-expiration-iteration-at-termination'
* origin/topic/awelzel/3523-expiration-iteration-at-termination:
  Dict: Invalidate iterators during Clear()
2024-01-09 09:16:01 +01:00
Arne Welzel
96823dab73 Dict: Invalidate iterators during Clear()
If we don't do this, a RobustDictIterator owned by someone else holds
dangling pointers to the Dictionary that is being cleared.

Closes #3523
2024-01-09 09:12:21 +01:00
Tim Wojtulewicz
985e1f9024 Merge branch 'topic/timw/werror'
* topic/timw/werror:
  CI: Remove unused openssl30_config
  CPP-gen: Don't emit extra braces if only one element
  Use <poll.h> instead of <sys/poll.h>
  ZAM: Create ListValPtr directly instead of a stack object
  Bump zeekjs to pick up dprintf warning fix
  Avoid unused-result warning in Supervisor
  Update src/3rdparty submodule to fix sprintf warning in modp
  Fix warning with attribute string lookup
  Set -Werror / /WX via target_compile_options to force warnings as errors
2024-01-08 14:10:55 -07:00
Tim Wojtulewicz
652ba502aa CI: Remove unused openssl30_config 2024-01-08 13:26:57 -07:00
Tim Wojtulewicz
df65b668b7 CPP-gen: Don't emit extra braces if only one element 2024-01-08 13:26:57 -07:00
Tim Wojtulewicz
016121b6f7 Use <poll.h> instead of <sys/poll.h> 2024-01-08 13:26:57 -07:00
Tim Wojtulewicz
ca29793ecc ZAM: Create ListValPtr directly instead of a stack object 2024-01-08 13:26:57 -07:00
Tim Wojtulewicz
ac59b11f33 Bump zeekjs to pick up dprintf warning fix 2024-01-08 13:26:57 -07:00
Tim Wojtulewicz
d87e2ec70c Avoid unused-result warning in Supervisor 2024-01-08 13:26:57 -07:00
Tim Wojtulewicz
3d5aaf9aec Update src/3rdparty submodule to fix sprintf warning in modp 2024-01-08 13:26:57 -07:00
Tim Wojtulewicz
b639f1426f Fix warning with attribute string lookup 2024-01-08 13:25:08 -07:00
Tim Wojtulewicz
bbcc4b00fb Set -Werror / /WX via target_compile_options to force warnings as errors 2024-01-08 13:25:05 -07:00
Tim Wojtulewicz
3f70998896 Merge remote-tracking branch 'origin/topic/timw/update-broker'
* origin/topic/timw/update-broker:
  Update broker to pick up Windows 2-minute exception fix
2024-01-08 10:05:14 -07:00
Tim Wojtulewicz
3ae6a56c00 Update broker to pick up Windows 2-minute exception fix 2024-01-08 09:28:28 -07:00
Arne Welzel
dd42e00fc5 Bump cmake submodule 2024-01-08 16:00:14 +01:00
Arne Welzel
f3cbfa9c25 parse: Support @if conditionals in record definitions
Closes #3531
2024-01-08 15:21:48 +01:00
Dominik Charousset
bd3e5bedbb Integrate review feedback 2024-01-06 13:48:14 +01:00
Dominik Charousset
6132cbd13e Fix formatting 2024-01-06 13:39:08 +01:00
Dominik Charousset
1bc5fda591 Backward compatibility for OpaqueVal serialization
External plugins depend on the API for `OpaqueVal`. This set of changes
brings back the previous signature for the `Serialize` and `Unserialize`
member functions. The new set of functions that operate on the recently
added `BrokerData` API were renamed accordingly and use a `Data` suffix to
distinguish between the old and new interface.

For the transition period, `OpaqueVal` now has two "sets" of
serialization functions: old and new (using the suffix). By default, the
new functions call the old API and then convert to the new types. Hence,
plugins that override the old set of member functions will continue to
work. New code should only override the new set of functions.

Since the macro `DECLARE_OPAQUE_VALUE` (a convenience macro for adding a
default set of member functions to a subtype of `OpaqueVal`) might be
used by 3rd parties, the macro has been "restored" to its previous
behavior, i.e., it will override the old set of member functions. The
new macro `DECLARE_OPAQUE_VALUE_V2` is similar but overrides the new set
of functions instead.

The class `BloomFilter` uses the same member function signatures as
`OpaqueVal` for serialization. Hence, the same old/new split was
implemented to keep the APIs consistent.
2024-01-06 10:52:06 +01:00
zeek-bot
5ff99f7d0b Update doc submodule [nomail] [skip ci] 2024-01-06 00:12:24 +00:00
Arne Welzel
e52d401c85 Bump cmake submodule 2024-01-05 16:08:40 +01:00
Arne Welzel
23a47181b3 Merge remote-tracking branch 'origin/topic/awelzel/ldap-search-substring-parsing'
* origin/topic/awelzel/ldap-search-substring-parsing:
  ldap: Fix substring filter parsing and rendering
2024-01-05 16:07:16 +01:00
Arne Welzel
6a6cc7f551 ldap: Fix substring filter parsing and rendering
The initial (prefix) and final (suffix) strings are specified individually
with a variable number of "any" matches that can occur between these.
The previous implementation assumed a single string and rendered it
as *<string>*.

Reported and PCAP provided by @martinvanhensbergen, thanks!

Closes zeek/spicy-ldap#27
2024-01-05 16:06:23 +01:00
Arne Welzel
fe0f981f87 Merge remote-tracking branch 'origin/topic/awelzel/3503-quic-v2'
* origin/topic/awelzel/3503-quic-v2:
  quic: tests: Require have-spicy
  quic: analyzer: Recognize and report unknown versions better
  quic: tests: Add QUIC v2 test cases
  quic: analyzer: Support QUIC v2
  quic: decrypt_crypto: Support QUIC v2
2024-01-05 14:44:20 +01:00
Arne Welzel
0796a191c6 quic: tests: Require have-spicy 2024-01-05 11:37:35 +01:00
Arne Welzel
50cdac922f quic: analyzer: Recognize and report unknown versions better
This makes the analyzer.log entry more informative by including the
actual version and also allows to handle this scenario in script land
if needed.
2024-01-05 11:37:03 +01:00
Arne Welzel
727091ed67 quic: tests: Add QUIC v2 test cases
Produced using examples from the go-quic project, patching the clients
to force QUIC v2.
2024-01-05 11:36:57 +01:00
Arne Welzel
dabe85ebbf quic: analyzer: Support QUIC v2
QUIC v2 changed the version *and* the packet type enumeration to prevent
protocol ossification. Use an intermediary unit to handle the difference.
2024-01-05 11:36:57 +01:00
Arne Welzel
0b6f4ef443 quic: decrypt_crypto: Support QUIC v2
Attempt to refactor in order to re-use common code between the two
versions.
2024-01-05 11:36:57 +01:00
Benjamin Bannier
6c4a9510da Merge branch 'topic/bbannier/bump-spicy' 2024-01-04 16:40:24 +01:00
Benjamin Bannier
1e16530538 Bump auxil/spicy to latest development snapshot
This bump mostly brings in small incremental performance improvements
for runtime throughput for Spicy-generated parsers. With a large
internal analyzer at Corelight these changes improve throughput by
around (3.8±1.2)%.
2024-01-04 14:53:00 +01:00
zeek-bot
72724d4e88 Update doc submodule [nomail] [skip ci] 2024-01-04 00:21:04 +00:00
Arne Welzel
4ebd81fb23 Merge remote-tracking branch 'origin/topic/awelzel/3504-ldap-logs-scalars'
* origin/topic/awelzel/3504-ldap-logs-scalars:
  Update external baselines
  ldap: Use scalar values in logs where appropriate
  ldap: Rename LDAP::search_result to LDAP::search_result_entry
2024-01-03 12:35:51 +01:00
Arne Welzel
d01b0bafdb Merge remote-tracking branch 'origin/topic/awelzel/no-more-segment-profiler'
* origin/topic/awelzel/no-more-segment-profiler:
  segment_profiling: Remove SegmentProfiler and load_sample event
2024-01-03 12:25:20 +01:00
Arne Welzel
48a92a58f3 Update external baselines 2024-01-03 11:57:31 +01:00
Arne Welzel
242db4981d ldap: Use scalar values in logs where appropriate
Skimming through the RFC, the previous approach of having containers for most
fields seems unfounded for normal protocol operation. The new weirds could just
as well be considered protocol violations. Outside of duplicated or missed data
they just shouldn't happen for well-behaved client/server behavior.
Additionally, with non-conformant traffic it would be trivial to cause
unbounded state growth and immense log record sizes.

Unfortunately, things have become a bit clunky now.

Closes #3504
2024-01-03 11:57:31 +01:00
Arne Welzel
46d0287b49 ldap: Rename LDAP::search_result to LDAP::search_result_entry
To avoid confusion potential confusion between SearchResultEntry
and SearchResultDone.
2024-01-03 11:57:30 +01:00