Commit graph

524 commits

Author SHA1 Message Date
Arne Welzel
18bfdb8a2b Merge remote-tracking branch 'origin/topic/awelzel/deprecate-broker-auto-publish'
* origin/topic/awelzel/deprecate-broker-auto-publish:
  sumstats: Remove copy() for Broker::publish() calls
  broker/Publish: Use event time instead of network time
  broker/Eventhandler: Deprecate Broker::auto_publish() for v8.1
  btest: Remove Broker::auto_publish() usages
  frameworks/control: Remove Broker::auto_publish()
  catch-and-release: Remove Broker::auto_publish()
  ssl/validate-certs: Remove Broker::auto_publish()
  sumstats: Remove Broker::auto_publish()
  cluster_started: No Broker::auto_publish() use
  openflow: Remove Broker::auto_publish()
  dhcp: Remove Broker::auto_publish()
  frameworks/notice: Remove Broker::auto_publish()
  netcontrol: Replace Broker::auto_publish()
  intel: Switch to Cluster::publish()
  broker: Support publish() of unspecified set() / table()
  types: Fix table() resulting in table_type->IsSet() == true
2024-11-14 14:17:13 +01:00
Tim Wojtulewicz
35ec9733c0 Add conn.log entries for connections with unhandled IP protocols 2024-11-13 11:25:40 -07:00
Arne Welzel
189565d131 types: Fix table() resulting in table_type->IsSet() == true
typename(table()) apparently always resulted in a set[] type
being rendered. Make the yield type of an unspecified table
ANY so that type->IsSet() ends up false.

While at it, also render unspecified types as table(), set() and
vector() rather than vector of void, set[] or table[] of any which
IMO should help to figure out what's going.
2024-11-06 13:28:57 +01:00
Evan Typanski
08348cd177 Add enum value negative check
There was one already at parse time, this adds a check later so that
cases like overflows or internal enums with negative values get caught.
2024-09-17 12:04:19 -04:00
Evan Typanski
d3dd8a155d Fix port/enum values SizeOf not being a count
Really, they both should be count. But, they were getting provided as an
integer. Port is easy since it is backed by an unsigned value. Enums
*should* be unsigned, but aren't. This doesn't address that, it just
takes the other name for this operator (absolute value) and makes the
enum value positive if it's negative.

This fixes a case where using the size of operator on enum/port values
in certain contexts (like the default parameter of a struct) would cause
an internal error.
2024-09-17 10:55:45 -04:00
Arne Welzel
ec1088c3ef Merge remote-tracking branch 'origin/topic/vern/zam-regularization'
* origin/topic/vern/zam-regularization: (33 commits)
  simpler and more robust identification of function parameters for AST profiling
  fixes to limit AST traversal in the face of recursive types
  address some script optimization compiler warnings under Linux
  fix for -O C++ construction of variable names that use multiple module namespaces
  fix for script optimization of "opaque" values that are run-time constants
  fix for script optimization of nested switch statements
  script optimization fix for complex "in" expressions in conditionals
  updates to typos allow-list reflecting ZAM regularization changes
  BTest updates for ZAM regularization changes
  convert new ZAM operations to use typed operands
  complete migration of ZAM to use only public ZVal methods
  "-O validate-ZAM" option to validate generated ZAM instructions
  internal option to suppress control-flow optimization
  exposing some functionality for greater flexibility in structuring run-time execution
  rework ZAM compilation of type switches to leverage value switches
  add tracking of control flow information
  factoring of ZAM operation specifications into separate files
  updates to ZAM operations / gen-zam regularization, other than the operations themselves
  type-checking fix for vector-of-string operations
  ZVal constructor for booleans
  ...
2024-08-16 12:10:33 +02:00
Vern Paxson
bf3cf9da48 BTest updates for ZAM regularization changes 2024-08-16 11:18:58 +02:00
Tim Wojtulewicz
2d68b1d834 Return an error if GLOBAL:: prefix is used 2024-08-07 11:58:22 -07:00
Arne Welzel
52bcc92e40 Merge remote-tracking branch 'origin/topic/vern/script-opt-maint.Aug24'
* origin/topic/vern/script-opt-maint.Aug24:
  minor optimization of boolean comparisons
  fix & regression test for GH-3839 (spurious warnings for "when" constructs)
2024-07-24 11:18:18 +02:00
Vern Paxson
e960c29acb fix & regression test for GH-3839 (spurious warnings for "when" constructs) 2024-07-23 15:18:21 -07:00
Vern Paxson
a53e28be87 minor script optimization updates to reflect recent changes, Coverity findings 2024-07-06 16:44:27 -07:00
Christian Kreibich
0179a5e75c Support JSON roundtripping via to_json()/from_json() for patterns
This needed a small tweak in the deserialization, since each roundtrip
would otherwise pad the prior pattern with an extra /^?(...)$?/.

This expands the language.set test to also verify serializing/unserializing for
sets, similarly to tables in the previous commit.
2024-07-02 14:46:16 -07:00
Christian Kreibich
92c1098e97 Support table deserialization in from_json()
This allows additional data roundtripping through JSON since to_json() already
supports tables. There are some subtleties around the formatting of strings in
JSON object keys, for which this adds a bit of helper infrastructure.

This also expands the language.table test to verify the roundtrips, and adapts
bif.from_json to include a table in the test record.
2024-07-02 14:46:16 -07:00
Vern Paxson
4b719ef45a script optimization for record operations sourced (in part) from other records 2024-06-24 09:38:37 +02:00
Vern Paxson
033cf2d8dd factored BTest constants into globals to prevent premature ZAM optimization when testing 2024-04-27 19:40:39 -07:00
Tim Wojtulewicz
9abf8ea8b7 Check for valid identifier in WhenStmt::Describe 2024-04-10 16:20:36 -07:00
Arne Welzel
2d1588277e Merge remote-tracking branch 'origin/topic/awelzel/3640-void-size-expr-assign-stmt'
* origin/topic/awelzel/3640-void-size-expr-assign-stmt:
  Expr: Handle TYPE_VOID in SizeExpr and AssignExpr::Typecheck()
2024-03-12 10:31:46 +01:00
Vern Paxson
54877e9dec fixed bug where appending an empty vector to a vector-of-vector's had no effect 2024-03-08 19:09:43 +01:00
Arne Welzel
8cb1a1518f Expr: Handle TYPE_VOID in SizeExpr and AssignExpr::Typecheck()
@vpax reported surprising behavior when working with "void values".
While these are not exposed to script land, plumb the places he
pointed out are causing confusing behavior.

Closes #3640.
2024-03-07 11:24:28 +01:00
Arne Welzel
6572324b8c Stmt: Fix assert evaluating cond twice
Since 81a9745fb3, the assert condition is
evaluated twice. This leads to unexpected behavior when cond has a side
effect like publishing a message or creating a log stream or filter.

Found while using the following in ad-hoc testing code and wondering
why two messages were published.

    assert publish(Cluster::worker_topic, hello, "abc")
2024-03-04 15:30:09 +01:00
Arne Welzel
2f1893bc58 Allow delete statement for tables, sets and vectors
Relates to #3472. This allow "delete tbl" as an alternative for
clear_table(tbl). Also works for vectors.
2024-02-21 18:00:00 +01:00
Vern Paxson
71a1aa0afd fix for needing to always flush optimization information for identifiers 2024-01-15 15:03:56 +01:00
Arne Welzel
f3cbfa9c25 parse: Support @if conditionals in record definitions
Closes #3531
2024-01-08 15:21:48 +01:00
Arne Welzel
c113b9b297 Expr/Val: Add support for in set[pattern] 2023-11-21 10:34:17 +01:00
Arne Welzel
e39f280e3d zeek.bif: Implement table_pattern_matcher_stats() bif for introspection
Provide a script accessible way to introspect the DFA stats that can be
leveraged to gather runtime statistics of the underlying DFA. This
re-uses the existing MatcherStats used by ``get_matcher_stats()``.
2023-11-21 10:34:17 +01:00
Arne Welzel
c72d4a4427 Expr: Implement string in table[pattern] of X
Not sure how useful this is (and the implementation isn't optimized in
any way), but seems reasonable for consistency.

Vern suggested that set[pattern] can already be achieved via
set_to_regex(), so left out any set[pattern] variants.
2023-11-21 10:34:17 +01:00
Arne Welzel
074f51fc96 btest: Add test for pattern tables and when 2023-11-21 10:34:16 +01:00
Vern Paxson
fd1094a184 BTests for indexing "table[pattern] of T" with strings 2023-11-21 10:34:15 +01:00
Vern Paxson
280acc51bf BTest updates for latest ZAM maintenance 2023-11-10 09:57:35 +01:00
Arne Welzel
8109bbc52f Var/Func: Render function parameters using comma, not semicolon
A bit larger follow-up to what Tim pointed out: Function prototype descriptions
previously used semicolons to separate parameters.

Switch to use commas when a RecordType is used as function parameter.
Use existing "func_args" naming for consistency.
2023-10-04 17:42:30 +02:00
Arne Welzel
8ede22f6ec Var: Fix null-pointer deref on redefinition of lambdas
Closes #3314
2023-10-04 15:18:24 +02:00
Vern Paxson
3addda28d3 removed order-of-evaluation ambiguity from new BTest 2023-09-27 11:36:22 -07:00
Arne Welzel
f4322f9584 Merge remote-tracking branch 'origin/topic/vern/script-opt-maint.Sep23'
* origin/topic/vern/script-opt-maint.Sep23:
  fix for ZAM statement-level profiling (broken by GH-3199)
  ZAM fixes for compatibility with GH-3249 changes
  -O gen-C++ fixes for compatibility with GH-3249 changes minor -O gen-C++ BTest updates
  minor BTest reordering to diminish differences with script optimization
2023-09-08 16:47:29 +02:00
Arne Welzel
aaa81cae5d CompositeHash: Skip record initialization when recovering vals
Initializing fields of recovered records caused running &default expression
of fields just so that they are re-assigned in the next step with the
recovered fields. The second test case still shows that the loop var
is initialized as well even though that's not needed.

Add tests for iterating over records with &default attributes for both,
tables and vectors.

Fixes #3267
2023-09-08 13:02:34 +02:00
Vern Paxson
85cd1d69a4 minor BTest reordering to diminish differences with script optimization 2023-09-01 12:17:06 -07:00
Vern Paxson
8e5dac3900 BTests for any/vector-of-any fixes 2023-08-24 15:49:13 -07:00
Tim Wojtulewicz
e8ef169b27 Merge remote-tracking branch 'origin/topic/timw/3059-set-vector-conversion'
* origin/topic/timw/3059-set-vector-conversion:
  Fix conversion with record types
  Add conversion between set and vector using 'as' keyword
  Add std::move for a couple of variables passed by value
2023-08-11 10:35:06 -07:00
Tim Wojtulewicz
fe9926e538 Fix conversion with record types 2023-08-10 13:42:23 -07:00
Tim Wojtulewicz
af9e852c28 Add conversion between set and vector using 'as' keyword 2023-08-09 14:41:54 -07:00
Arne Welzel
431767d04b Add &default_insert attribute for tables
This is based on the discussion in zeek/zeek#2668. Using &default with tables
can be confusing as the default value is not inserted. The following example
prints an empty table at the end even new Service records was instantiated.

    type Service: record {
        occurrences: count &default=0;
        last_seen: time &default=network_time();
    };

    global services: table[string] of Service &default=Service();

    event zeek_init()
        {
        services["http"]$occurrences += 1;
        services["http"]$last_seen = network_time();

        print services;
        }

Changing above &default to &default_insert will insert the newly created
default value upon a missed lookup and act less surprising.

Other examples that caused confusion previously revolved around table of sets
 or table of vectors and `add` or `+=` not working as expected.

    tbl_of_vector["http"] += 1
    add tbl_of_set["http"][1];
2023-08-04 12:30:36 +02:00
Arne Welzel
ab388844bf UsageAnalyzer: Collect identifiers found in attributes as seeds
This marks every identifier used within an attribute as seeds. The scenario
this avoids is functions referenced through attributes on unused tables or
record types (&default, &expire_func, ...) being dinged as unused as
that's rather confusing.

Also adds test for the above and a light smoke test into language/ as it
doesn't appear we had coverage here.

Closes #3122
2023-08-01 15:46:10 +02:00
Arne Welzel
12a5616f4d Support :: prefix to reference global namespace, deprecate GLOBAL::
Closes #2709
2023-07-11 12:51:21 +02:00
Vern Paxson
a1d25670fc Fix for nested "when" statements leading to confusion over scoping 2023-07-05 17:10:38 -07:00
Vern Paxson
a4cf10e34d tweaks to BTests for ZAM feature completeness; new ZAM-only btest 2023-06-30 09:36:14 +02:00
Vern Paxson
5165a04ae0 ZAM ignores new "assert" statement 2023-06-14 17:41:41 -07:00
Vern Paxson
40d7e6949a BTest to check for invalid negative vector indices 2023-06-14 17:35:08 -07:00
Tim Wojtulewicz
d5b19c61ff Change table initialization deprecation to error 2023-06-14 10:07:23 -07:00
Tim Wojtulewicz
0110c15ee2 Make break/next/continue outside loops an error (6.1 deprecation) 2023-06-14 10:07:22 -07:00
Arne Welzel
0b0f6e509f Stmt: Rework assertion hooks break semantics
Using break in either of the hooks allows to suppress the default reporter
error message rather than suppressing solely based on the existence of an
assertion_failure() handler.
2023-06-13 16:18:01 +02:00
Arne Welzel
25ea678626 Stmt: Introduce assert statement and related hooks
including two hooks called assertion_failure() and assertion_result() for
customization and tracking of assertion results.
2023-06-12 18:16:02 +02:00