Commit graph

535 commits

Author SHA1 Message Date
赵富鹏
074faf202f Fix &ordered attribute not preserved in table initializer assignments 2025-06-04 17:07:34 +08:00
Arne Welzel
53b0f0ad64 Event: Deprecate default network timestamp metadata
This deprecates the Event constructor and the ``ts`` parameter of Enqueue()
Instead, versions are introduced that take a detail::MetadataVectorPtr which
can hold the network timestamp metadata and is meant to be allocated by the
caller instead of automatically during Enqueue() or within the Event
constructor.

This also introduces a BifConst ``EventMetadata::add_network_timestamp`` to
opt-in adding network timestamps to events globally. It's disabled by
default as there are not a lot of known use cases that need this.
2025-05-23 19:32:23 +02:00
Arne Welzel
85b8c8866b testing/btest/*zeek: Comment all @TEST lines 2025-04-17 16:30:23 +02:00
Evan Typanski
4237239325 Fix printing type constants like double
Fixes #4268
2025-03-07 11:54:04 -05:00
Evan Typanski
7160e074f6 Do not export locals
Fixes #4227
2025-02-12 13:49:12 -08:00
Evan Typanski
0c27aa6c50 Make types into constants
Closes #4173

This allows types to be used in expressions, but they can't be
reassigned. Note that this was meant to be a special "type expression" -
but that is unnecessary complexity.

Type expressions would allow access to the type without going through
its constant value, but the constant value is never changed, so it's
simply a few more checks if necessary when functionality gets expanded.
This way, ZAM and other code will not need updates, so the potential for
increased work in the future is probably not worth caring about.
2025-01-24 13:57:28 -05:00
Arne Welzel
cd8adb3da5 Expr: Fix folding of pattern values to support == and !=
The fatal error is actually triggered at runtime, so it's a bit
dangerous for users, but not sure there's many use-cases to
compare vectors of patterns.

Closes #4084
2024-12-16 10:56:02 +01:00
Vern Paxson
8876ec6f2d BTest updates in support of compiling-scripts-to-C++ 2024-12-12 14:30:23 -08:00
Arne Welzel
93a3a11d36 Merge remote-tracking branch 'origin/topic/vern/script-opt-keep-asserts'
* origin/topic/vern/script-opt-keep-asserts:
  ZAM documentation updates for asserts and event handler run-time errors
  BTest updates for ZAM support of (optionally) keeping "assert" statements
  command-line options for controlling script optimization: keeping asserts, avoiding event handler coalescence
  ZAM support for option to not coalesce event handlers
  ZAM support for keeping "assert" statements
  internal support for script optimization options for keeping asserts, not consolidating event handlers
  ZAM operations to support asserts
  simplified "assert" by not trying to catch messages that themselves have errors

Fixed some TEST-REQUIRES "${ZEEK_ZAM}" == "1" to use "=" instead to
be /bin/sh compatible.
2024-12-05 21:45:07 +01:00
Vern Paxson
47a555da87 BTest updates for ZAM support of (optionally) keeping "assert" statements 2024-12-03 10:40:41 -07:00
Vern Paxson
05e913db1b simplified "assert" by not trying to catch messages that themselves have errors 2024-12-02 10:37:10 -08:00
Arne Welzel
18bfdb8a2b Merge remote-tracking branch 'origin/topic/awelzel/deprecate-broker-auto-publish'
* origin/topic/awelzel/deprecate-broker-auto-publish:
  sumstats: Remove copy() for Broker::publish() calls
  broker/Publish: Use event time instead of network time
  broker/Eventhandler: Deprecate Broker::auto_publish() for v8.1
  btest: Remove Broker::auto_publish() usages
  frameworks/control: Remove Broker::auto_publish()
  catch-and-release: Remove Broker::auto_publish()
  ssl/validate-certs: Remove Broker::auto_publish()
  sumstats: Remove Broker::auto_publish()
  cluster_started: No Broker::auto_publish() use
  openflow: Remove Broker::auto_publish()
  dhcp: Remove Broker::auto_publish()
  frameworks/notice: Remove Broker::auto_publish()
  netcontrol: Replace Broker::auto_publish()
  intel: Switch to Cluster::publish()
  broker: Support publish() of unspecified set() / table()
  types: Fix table() resulting in table_type->IsSet() == true
2024-11-14 14:17:13 +01:00
Tim Wojtulewicz
35ec9733c0 Add conn.log entries for connections with unhandled IP protocols 2024-11-13 11:25:40 -07:00
Arne Welzel
189565d131 types: Fix table() resulting in table_type->IsSet() == true
typename(table()) apparently always resulted in a set[] type
being rendered. Make the yield type of an unspecified table
ANY so that type->IsSet() ends up false.

While at it, also render unspecified types as table(), set() and
vector() rather than vector of void, set[] or table[] of any which
IMO should help to figure out what's going.
2024-11-06 13:28:57 +01:00
Evan Typanski
08348cd177 Add enum value negative check
There was one already at parse time, this adds a check later so that
cases like overflows or internal enums with negative values get caught.
2024-09-17 12:04:19 -04:00
Evan Typanski
d3dd8a155d Fix port/enum values SizeOf not being a count
Really, they both should be count. But, they were getting provided as an
integer. Port is easy since it is backed by an unsigned value. Enums
*should* be unsigned, but aren't. This doesn't address that, it just
takes the other name for this operator (absolute value) and makes the
enum value positive if it's negative.

This fixes a case where using the size of operator on enum/port values
in certain contexts (like the default parameter of a struct) would cause
an internal error.
2024-09-17 10:55:45 -04:00
Arne Welzel
ec1088c3ef Merge remote-tracking branch 'origin/topic/vern/zam-regularization'
* origin/topic/vern/zam-regularization: (33 commits)
  simpler and more robust identification of function parameters for AST profiling
  fixes to limit AST traversal in the face of recursive types
  address some script optimization compiler warnings under Linux
  fix for -O C++ construction of variable names that use multiple module namespaces
  fix for script optimization of "opaque" values that are run-time constants
  fix for script optimization of nested switch statements
  script optimization fix for complex "in" expressions in conditionals
  updates to typos allow-list reflecting ZAM regularization changes
  BTest updates for ZAM regularization changes
  convert new ZAM operations to use typed operands
  complete migration of ZAM to use only public ZVal methods
  "-O validate-ZAM" option to validate generated ZAM instructions
  internal option to suppress control-flow optimization
  exposing some functionality for greater flexibility in structuring run-time execution
  rework ZAM compilation of type switches to leverage value switches
  add tracking of control flow information
  factoring of ZAM operation specifications into separate files
  updates to ZAM operations / gen-zam regularization, other than the operations themselves
  type-checking fix for vector-of-string operations
  ZVal constructor for booleans
  ...
2024-08-16 12:10:33 +02:00
Vern Paxson
bf3cf9da48 BTest updates for ZAM regularization changes 2024-08-16 11:18:58 +02:00
Tim Wojtulewicz
2d68b1d834 Return an error if GLOBAL:: prefix is used 2024-08-07 11:58:22 -07:00
Arne Welzel
52bcc92e40 Merge remote-tracking branch 'origin/topic/vern/script-opt-maint.Aug24'
* origin/topic/vern/script-opt-maint.Aug24:
  minor optimization of boolean comparisons
  fix & regression test for GH-3839 (spurious warnings for "when" constructs)
2024-07-24 11:18:18 +02:00
Vern Paxson
e960c29acb fix & regression test for GH-3839 (spurious warnings for "when" constructs) 2024-07-23 15:18:21 -07:00
Vern Paxson
a53e28be87 minor script optimization updates to reflect recent changes, Coverity findings 2024-07-06 16:44:27 -07:00
Christian Kreibich
0179a5e75c Support JSON roundtripping via to_json()/from_json() for patterns
This needed a small tweak in the deserialization, since each roundtrip
would otherwise pad the prior pattern with an extra /^?(...)$?/.

This expands the language.set test to also verify serializing/unserializing for
sets, similarly to tables in the previous commit.
2024-07-02 14:46:16 -07:00
Christian Kreibich
92c1098e97 Support table deserialization in from_json()
This allows additional data roundtripping through JSON since to_json() already
supports tables. There are some subtleties around the formatting of strings in
JSON object keys, for which this adds a bit of helper infrastructure.

This also expands the language.table test to verify the roundtrips, and adapts
bif.from_json to include a table in the test record.
2024-07-02 14:46:16 -07:00
Vern Paxson
4b719ef45a script optimization for record operations sourced (in part) from other records 2024-06-24 09:38:37 +02:00
Vern Paxson
033cf2d8dd factored BTest constants into globals to prevent premature ZAM optimization when testing 2024-04-27 19:40:39 -07:00
Tim Wojtulewicz
9abf8ea8b7 Check for valid identifier in WhenStmt::Describe 2024-04-10 16:20:36 -07:00
Arne Welzel
2d1588277e Merge remote-tracking branch 'origin/topic/awelzel/3640-void-size-expr-assign-stmt'
* origin/topic/awelzel/3640-void-size-expr-assign-stmt:
  Expr: Handle TYPE_VOID in SizeExpr and AssignExpr::Typecheck()
2024-03-12 10:31:46 +01:00
Vern Paxson
54877e9dec fixed bug where appending an empty vector to a vector-of-vector's had no effect 2024-03-08 19:09:43 +01:00
Arne Welzel
8cb1a1518f Expr: Handle TYPE_VOID in SizeExpr and AssignExpr::Typecheck()
@vpax reported surprising behavior when working with "void values".
While these are not exposed to script land, plumb the places he
pointed out are causing confusing behavior.

Closes #3640.
2024-03-07 11:24:28 +01:00
Arne Welzel
6572324b8c Stmt: Fix assert evaluating cond twice
Since 81a9745fb3, the assert condition is
evaluated twice. This leads to unexpected behavior when cond has a side
effect like publishing a message or creating a log stream or filter.

Found while using the following in ad-hoc testing code and wondering
why two messages were published.

    assert publish(Cluster::worker_topic, hello, "abc")
2024-03-04 15:30:09 +01:00
Arne Welzel
2f1893bc58 Allow delete statement for tables, sets and vectors
Relates to #3472. This allow "delete tbl" as an alternative for
clear_table(tbl). Also works for vectors.
2024-02-21 18:00:00 +01:00
Vern Paxson
71a1aa0afd fix for needing to always flush optimization information for identifiers 2024-01-15 15:03:56 +01:00
Arne Welzel
f3cbfa9c25 parse: Support @if conditionals in record definitions
Closes #3531
2024-01-08 15:21:48 +01:00
Arne Welzel
c113b9b297 Expr/Val: Add support for in set[pattern] 2023-11-21 10:34:17 +01:00
Arne Welzel
e39f280e3d zeek.bif: Implement table_pattern_matcher_stats() bif for introspection
Provide a script accessible way to introspect the DFA stats that can be
leveraged to gather runtime statistics of the underlying DFA. This
re-uses the existing MatcherStats used by ``get_matcher_stats()``.
2023-11-21 10:34:17 +01:00
Arne Welzel
c72d4a4427 Expr: Implement string in table[pattern] of X
Not sure how useful this is (and the implementation isn't optimized in
any way), but seems reasonable for consistency.

Vern suggested that set[pattern] can already be achieved via
set_to_regex(), so left out any set[pattern] variants.
2023-11-21 10:34:17 +01:00
Arne Welzel
074f51fc96 btest: Add test for pattern tables and when 2023-11-21 10:34:16 +01:00
Vern Paxson
fd1094a184 BTests for indexing "table[pattern] of T" with strings 2023-11-21 10:34:15 +01:00
Vern Paxson
280acc51bf BTest updates for latest ZAM maintenance 2023-11-10 09:57:35 +01:00
Arne Welzel
8109bbc52f Var/Func: Render function parameters using comma, not semicolon
A bit larger follow-up to what Tim pointed out: Function prototype descriptions
previously used semicolons to separate parameters.

Switch to use commas when a RecordType is used as function parameter.
Use existing "func_args" naming for consistency.
2023-10-04 17:42:30 +02:00
Arne Welzel
8ede22f6ec Var: Fix null-pointer deref on redefinition of lambdas
Closes #3314
2023-10-04 15:18:24 +02:00
Vern Paxson
3addda28d3 removed order-of-evaluation ambiguity from new BTest 2023-09-27 11:36:22 -07:00
Arne Welzel
f4322f9584 Merge remote-tracking branch 'origin/topic/vern/script-opt-maint.Sep23'
* origin/topic/vern/script-opt-maint.Sep23:
  fix for ZAM statement-level profiling (broken by GH-3199)
  ZAM fixes for compatibility with GH-3249 changes
  -O gen-C++ fixes for compatibility with GH-3249 changes minor -O gen-C++ BTest updates
  minor BTest reordering to diminish differences with script optimization
2023-09-08 16:47:29 +02:00
Arne Welzel
aaa81cae5d CompositeHash: Skip record initialization when recovering vals
Initializing fields of recovered records caused running &default expression
of fields just so that they are re-assigned in the next step with the
recovered fields. The second test case still shows that the loop var
is initialized as well even though that's not needed.

Add tests for iterating over records with &default attributes for both,
tables and vectors.

Fixes #3267
2023-09-08 13:02:34 +02:00
Vern Paxson
85cd1d69a4 minor BTest reordering to diminish differences with script optimization 2023-09-01 12:17:06 -07:00
Vern Paxson
8e5dac3900 BTests for any/vector-of-any fixes 2023-08-24 15:49:13 -07:00
Tim Wojtulewicz
e8ef169b27 Merge remote-tracking branch 'origin/topic/timw/3059-set-vector-conversion'
* origin/topic/timw/3059-set-vector-conversion:
  Fix conversion with record types
  Add conversion between set and vector using 'as' keyword
  Add std::move for a couple of variables passed by value
2023-08-11 10:35:06 -07:00
Tim Wojtulewicz
fe9926e538 Fix conversion with record types 2023-08-10 13:42:23 -07:00
Tim Wojtulewicz
af9e852c28 Add conversion between set and vector using 'as' keyword 2023-08-09 14:41:54 -07:00