Commit graph

16159 commits

Author SHA1 Message Date
Arne Welzel
96823dab73 Dict: Invalidate iterators during Clear()
If we don't do this, a RobustDictIterator owned by someone else holds
dangling pointers to the Dictionary that is being cleared.

Closes #3523
2024-01-09 09:12:21 +01:00
Tim Wojtulewicz
985e1f9024 Merge branch 'topic/timw/werror'
* topic/timw/werror:
  CI: Remove unused openssl30_config
  CPP-gen: Don't emit extra braces if only one element
  Use <poll.h> instead of <sys/poll.h>
  ZAM: Create ListValPtr directly instead of a stack object
  Bump zeekjs to pick up dprintf warning fix
  Avoid unused-result warning in Supervisor
  Update src/3rdparty submodule to fix sprintf warning in modp
  Fix warning with attribute string lookup
  Set -Werror / /WX via target_compile_options to force warnings as errors
2024-01-08 14:10:55 -07:00
Tim Wojtulewicz
652ba502aa CI: Remove unused openssl30_config 2024-01-08 13:26:57 -07:00
Tim Wojtulewicz
df65b668b7 CPP-gen: Don't emit extra braces if only one element 2024-01-08 13:26:57 -07:00
Tim Wojtulewicz
016121b6f7 Use <poll.h> instead of <sys/poll.h> 2024-01-08 13:26:57 -07:00
Tim Wojtulewicz
ca29793ecc ZAM: Create ListValPtr directly instead of a stack object 2024-01-08 13:26:57 -07:00
Tim Wojtulewicz
ac59b11f33 Bump zeekjs to pick up dprintf warning fix 2024-01-08 13:26:57 -07:00
Tim Wojtulewicz
d87e2ec70c Avoid unused-result warning in Supervisor 2024-01-08 13:26:57 -07:00
Tim Wojtulewicz
3d5aaf9aec Update src/3rdparty submodule to fix sprintf warning in modp 2024-01-08 13:26:57 -07:00
Tim Wojtulewicz
b639f1426f Fix warning with attribute string lookup 2024-01-08 13:25:08 -07:00
Tim Wojtulewicz
bbcc4b00fb Set -Werror / /WX via target_compile_options to force warnings as errors 2024-01-08 13:25:05 -07:00
Tim Wojtulewicz
3f70998896 Merge remote-tracking branch 'origin/topic/timw/update-broker'
* origin/topic/timw/update-broker:
  Update broker to pick up Windows 2-minute exception fix
2024-01-08 10:05:14 -07:00
Tim Wojtulewicz
3ae6a56c00 Update broker to pick up Windows 2-minute exception fix 2024-01-08 09:28:28 -07:00
Arne Welzel
dd42e00fc5 Bump cmake submodule 2024-01-08 16:00:14 +01:00
Arne Welzel
f3cbfa9c25 parse: Support @if conditionals in record definitions
Closes #3531
2024-01-08 15:21:48 +01:00
Dominik Charousset
bd3e5bedbb Integrate review feedback 2024-01-06 13:48:14 +01:00
Dominik Charousset
6132cbd13e Fix formatting 2024-01-06 13:39:08 +01:00
Dominik Charousset
1bc5fda591 Backward compatibility for OpaqueVal serialization
External plugins depend on the API for `OpaqueVal`. This set of changes
brings back the previous signature for the `Serialize` and `Unserialize`
member functions. The new set of functions that operate on the recently
added `BrokerData` API were renamed accordingly and use a `Data` suffix to
distinguish between the old and new interface.

For the transition period, `OpaqueVal` now has two "sets" of
serialization functions: old and new (using the suffix). By default, the
new functions call the old API and then convert to the new types. Hence,
plugins that override the old set of member functions will continue to
work. New code should only override the new set of functions.

Since the macro `DECLARE_OPAQUE_VALUE` (a convenience macro for adding a
default set of member functions to a subtype of `OpaqueVal`) might be
used by 3rd parties, the macro has been "restored" to its previous
behavior, i.e., it will override the old set of member functions. The
new macro `DECLARE_OPAQUE_VALUE_V2` is similar but overrides the new set
of functions instead.

The class `BloomFilter` uses the same member function signatures as
`OpaqueVal` for serialization. Hence, the same old/new split was
implemented to keep the APIs consistent.
2024-01-06 10:52:06 +01:00
zeek-bot
5ff99f7d0b Update doc submodule [nomail] [skip ci] 2024-01-06 00:12:24 +00:00
Arne Welzel
e52d401c85 Bump cmake submodule 2024-01-05 16:08:40 +01:00
Arne Welzel
23a47181b3 Merge remote-tracking branch 'origin/topic/awelzel/ldap-search-substring-parsing'
* origin/topic/awelzel/ldap-search-substring-parsing:
  ldap: Fix substring filter parsing and rendering
2024-01-05 16:07:16 +01:00
Arne Welzel
6a6cc7f551 ldap: Fix substring filter parsing and rendering
The initial (prefix) and final (suffix) strings are specified individually
with a variable number of "any" matches that can occur between these.
The previous implementation assumed a single string and rendered it
as *<string>*.

Reported and PCAP provided by @martinvanhensbergen, thanks!

Closes zeek/spicy-ldap#27
2024-01-05 16:06:23 +01:00
Arne Welzel
fe0f981f87 Merge remote-tracking branch 'origin/topic/awelzel/3503-quic-v2'
* origin/topic/awelzel/3503-quic-v2:
  quic: tests: Require have-spicy
  quic: analyzer: Recognize and report unknown versions better
  quic: tests: Add QUIC v2 test cases
  quic: analyzer: Support QUIC v2
  quic: decrypt_crypto: Support QUIC v2
2024-01-05 14:44:20 +01:00
Arne Welzel
0796a191c6 quic: tests: Require have-spicy 2024-01-05 11:37:35 +01:00
Arne Welzel
50cdac922f quic: analyzer: Recognize and report unknown versions better
This makes the analyzer.log entry more informative by including the
actual version and also allows to handle this scenario in script land
if needed.
2024-01-05 11:37:03 +01:00
Arne Welzel
727091ed67 quic: tests: Add QUIC v2 test cases
Produced using examples from the go-quic project, patching the clients
to force QUIC v2.
2024-01-05 11:36:57 +01:00
Arne Welzel
dabe85ebbf quic: analyzer: Support QUIC v2
QUIC v2 changed the version *and* the packet type enumeration to prevent
protocol ossification. Use an intermediary unit to handle the difference.
2024-01-05 11:36:57 +01:00
Arne Welzel
0b6f4ef443 quic: decrypt_crypto: Support QUIC v2
Attempt to refactor in order to re-use common code between the two
versions.
2024-01-05 11:36:57 +01:00
Benjamin Bannier
6c4a9510da Merge branch 'topic/bbannier/bump-spicy' 2024-01-04 16:40:24 +01:00
Benjamin Bannier
1e16530538 Bump auxil/spicy to latest development snapshot
This bump mostly brings in small incremental performance improvements
for runtime throughput for Spicy-generated parsers. With a large
internal analyzer at Corelight these changes improve throughput by
around (3.8±1.2)%.
2024-01-04 14:53:00 +01:00
zeek-bot
72724d4e88 Update doc submodule [nomail] [skip ci] 2024-01-04 00:21:04 +00:00
Arne Welzel
4ebd81fb23 Merge remote-tracking branch 'origin/topic/awelzel/3504-ldap-logs-scalars'
* origin/topic/awelzel/3504-ldap-logs-scalars:
  Update external baselines
  ldap: Use scalar values in logs where appropriate
  ldap: Rename LDAP::search_result to LDAP::search_result_entry
2024-01-03 12:35:51 +01:00
Arne Welzel
d01b0bafdb Merge remote-tracking branch 'origin/topic/awelzel/no-more-segment-profiler'
* origin/topic/awelzel/no-more-segment-profiler:
  segment_profiling: Remove SegmentProfiler and load_sample event
2024-01-03 12:25:20 +01:00
Arne Welzel
48a92a58f3 Update external baselines 2024-01-03 11:57:31 +01:00
Arne Welzel
242db4981d ldap: Use scalar values in logs where appropriate
Skimming through the RFC, the previous approach of having containers for most
fields seems unfounded for normal protocol operation. The new weirds could just
as well be considered protocol violations. Outside of duplicated or missed data
they just shouldn't happen for well-behaved client/server behavior.
Additionally, with non-conformant traffic it would be trivial to cause
unbounded state growth and immense log record sizes.

Unfortunately, things have become a bit clunky now.

Closes #3504
2024-01-03 11:57:31 +01:00
Arne Welzel
46d0287b49 ldap: Rename LDAP::search_result to LDAP::search_result_entry
To avoid confusion potential confusion between SearchResultEntry
and SearchResultDone.
2024-01-03 11:57:30 +01:00
Arne Welzel
3f7881a57b segment_profiling: Remove SegmentProfiler and load_sample event
While it seems interesting functionality, this hasn't been documented,
maintained or knowingly leveraged for many years.

There are various other approaches today, too:

* We track the number of event handler invocations regardless of
  profiling. It's possible to approximate a load_sample event by
  comparing the result of two get_event_stats() calls. Or, visualize
  the corresponding counters in a Prometheus setup to get an idea of
  event/s broken down by event names.

* HookCallFunction() allows to intercept script execution, including
  measuring the time execution takes.

* The global call_stack and g_frame_stack can be used from plugins
  (and even external processes) to walk the Zeek script stack at certain
  points to implement a sampling profiler.

* USDT probes or more plugin hooks will likely be preferred over Zeek
  builtin functionality in the future.

Relates to #3458
2024-01-03 11:55:54 +01:00
Johanna Amann
df37cadbe8 Merge remote-tracking branch 'origin/topic/neverlord/data-to-threading-field'
* origin/topic/neverlord/data-to-threading-field:
  Avoid extra copies in threading_field_to_data
2024-01-03 09:44:35 +00:00
zeek-bot
190753ab45 Update doc submodule [nomail] [skip ci] 2024-01-03 00:12:44 +00:00
Tim Wojtulewicz
5740dbcf20 Merge remote-tracking branch 'origin/topic/awelzel/smb-unbounded-recent-files-growth'
* origin/topic/awelzel/smb-unbounded-recent-files-growth:
  smb: Fix &read_expire not in effect due to &default=string_set() usage
2024-01-02 11:13:36 -07:00
Dominik Charousset
12a45c2ff1 Avoid extra copies in threading_field_to_data
Passing the `broker::data` paramter for `threading_field_to_data` by
value forces copies. However, the function only needs const access to
its parameter. Hence, taking the parameter as const-ref is the logical
choice and avoids unnecessary copies of individual data fields.
2023-12-22 11:58:14 +01:00
Arne Welzel
fea8ee2260 smb: Fix &read_expire not in effect due to &default=string_set() usage
The SMB::State$recent_files field is meant to have expiring entries.
However, due to usage of &default=string_set(), the &read_expire
attribute is not respected causing unbounded state growth. Replace
&default=string_set() with &default=set().

Thanks to ya-sato on Slack for reporting!

Related: zeek/zeek-docs#179, #3513.
2023-12-17 15:02:05 +01:00
Arne Welzel
7a9a40f822 Merge remote-tracking branch 'origin/topic/vern/table-create-opt'
* origin/topic/vern/table-create-opt:
  TableType: Convert table_hash unique_ptr
  streamlining of constructing script-level tables
2023-12-16 17:57:58 +01:00
Arne Welzel
be0a22fec2 TableType: Convert table_hash unique_ptr
Cannot remove the destructor as otherwise the compiler attempts to create
its implementation in Type.h where CompositeHash isn't a complete type
yet and std::unique_ptr's delete fails to be instantiated.
2023-12-16 17:33:48 +01:00
Vern Paxson
a11ee9038b streamlining of constructing script-level tables 2023-12-16 17:33:46 +01:00
Tim Wojtulewicz
d1d9b9a1be Merge remote-tracking branch 'origin/topic/neverlord/broker-format'
* origin/topic/neverlord/broker-format:
  Update broker submodule
2023-12-15 14:59:42 -07:00
Dominik Charousset
a9954b363f Update broker submodule 2023-12-15 14:59:04 -07:00
Tim Wojtulewicz
43edd3c945 Merge remote-tracking branch 'stevesmoot/master'
* stevesmoot/master:
  improve search-ability in  zeek.spicy
2023-12-15 13:10:48 -07:00
Arne Welzel
83caf6108e Merge remote-tracking branch 'origin/topic/awelzel/3494-no-more-btest-script-coverage-locally'
* origin/topic/awelzel/3494-no-more-btest-script-coverage-locally:
  ci/btest: Remove ZEEK_PROFILER_FILE from btest.cfg, set in ci/test.sh explicitly
  ci: Remove ZEEK_CI_DISABLE_SCRIPT_PROFILING logic
2023-12-15 18:58:25 +01:00
Arne Welzel
43fec1f2c8 ci/btest: Remove ZEEK_PROFILER_FILE from btest.cfg, set in ci/test.sh explicitly
The produced coverage files are of little use in current local workflows
and usually just end-up taking up disk space. ZEEK_PROFILER_FILE can be
set explicitly if there's a one-off need to produce these locally, too.
2023-12-15 15:23:52 +01:00