Commit graph

97 commits

Author SHA1 Message Date
Arne Welzel
993cdd03e0 plugin: Add InitPreExecution()
Currently, plugins do not have a way to further inspect or even mutate
script functions after ZAM optimization ran. One use-case here is
zeek-perf-support [1]. This plugin wraps Stmt instances of functions,
events and hooks hooks with a small assembly stub to support JIT map
files [2] and for integration with perf tools.

This change introduces a new InitPreExecution() hook that runs after
ZAM optimization completed, just before the zeek_init() event is enqueued.

Additionally, remove the existing CPP_activation_hook. It doesn't seem
to be used. If it becomes necessary in the future, the new
InitPreExecution() hook can be leveraged instead.

[1] https://github.com/zeek/zeek-perf-support
[2] https://github.com/torvalds/linux/blob/master/tools/perf/Documentation/jit-interface.txt
2024-11-18 13:53:56 +01:00
Vern Paxson
e0e6c462ce "-O dump-final-ZAM" option similar to "dump-ZAM" only prints final version of functions 2024-09-27 13:24:30 -07:00
Vern Paxson
a68ffc5316 extended "-O allow-cond" to apply to both gen-C++ and gen-standalone-C++ 2024-09-27 13:23:06 -07:00
Arne Welzel
71e9c8d436 script_opt: Use Func::GetName() 2024-09-27 15:11:17 +02:00
Arne Welzel
9d1d4e28b3 ScriptOpt: Ensure global statements have non-null scope
The ProfileFunc() logic assumed that GetScope() returned a non-nullptr.
This holds except for the synthetic global statements function.

Fix the latter and add an assert, also add a name to the type so it's
easier to recognize in a debugger what's going on, otherwise the name
is "".

This was found by UBSAN due to it seeing the ->OrderedVars() call on a
nullptr. Elsewhere, num_params == 0 shielded from that access and so
didn't lead to crashes.
2024-08-16 13:59:24 +02:00
Arne Welzel
ec1088c3ef Merge remote-tracking branch 'origin/topic/vern/zam-regularization'
* origin/topic/vern/zam-regularization: (33 commits)
  simpler and more robust identification of function parameters for AST profiling
  fixes to limit AST traversal in the face of recursive types
  address some script optimization compiler warnings under Linux
  fix for -O C++ construction of variable names that use multiple module namespaces
  fix for script optimization of "opaque" values that are run-time constants
  fix for script optimization of nested switch statements
  script optimization fix for complex "in" expressions in conditionals
  updates to typos allow-list reflecting ZAM regularization changes
  BTest updates for ZAM regularization changes
  convert new ZAM operations to use typed operands
  complete migration of ZAM to use only public ZVal methods
  "-O validate-ZAM" option to validate generated ZAM instructions
  internal option to suppress control-flow optimization
  exposing some functionality for greater flexibility in structuring run-time execution
  rework ZAM compilation of type switches to leverage value switches
  add tracking of control flow information
  factoring of ZAM operation specifications into separate files
  updates to ZAM operations / gen-zam regularization, other than the operations themselves
  type-checking fix for vector-of-string operations
  ZVal constructor for booleans
  ...
2024-08-16 12:10:33 +02:00
Vern Paxson
1457099df3 "-O validate-ZAM" option to validate generated ZAM instructions 2024-08-16 11:18:58 +02:00
Vern Paxson
db22448270 internal option to suppress control-flow optimization 2024-08-16 11:18:58 +02:00
Vern Paxson
1253af42f9 mark functions skipped by ZAM compilation as such 2024-08-16 11:18:57 +02:00
Vern Paxson
e6fe20f140 fix for -O report-C++ 2024-08-16 11:18:57 +02:00
Vern Paxson
857df9f063 support for more in-depth AST profiling 2024-08-16 11:18:57 +02:00
Vern Paxson
d2c6208421 allow profiling without updating of hash values 2024-08-16 11:18:57 +02:00
Vern Paxson
769a3d958a some minor tidying of -O gen-C++ sources 2024-08-13 14:29:26 -07:00
Tim Wojtulewicz
e2b03681d1 Remove EventRegistry::Used and EventRegistry::SetUsed 2024-08-07 11:58:21 -07:00
Vern Paxson
4b719ef45a script optimization for record operations sourced (in part) from other records 2024-06-24 09:38:37 +02:00
Vern Paxson
a474106fce improve ZAM's estimation of profiling overheads 2024-04-09 15:12:00 -07:00
Vern Paxson
c29db63fdd ZAM execution changes to support richer profiling 2024-03-19 10:59:49 -07:00
Vern Paxson
dc376953fa framework for --enable-ZAM-profiling configuration
profiles go to zprof.log rather than stdout
2024-03-19 10:59:49 -07:00
Vern Paxson
71a1aa0afd fix for needing to always flush optimization information for identifiers 2024-01-15 15:03:56 +01:00
Vern Paxson
709d410fcd regularized (some) types of pointers used in script optimization 2023-12-12 09:45:19 +01:00
Vern Paxson
740a087765 reworked AST optimizers analysis of side effects during aggregate operations & calls 2023-12-06 12:26:27 -08:00
Vern Paxson
3d21d80dac coalescing of event handlers (ZAM optimization) 2023-11-16 12:58:28 -08:00
Vern Paxson
4ec9a23ce6 retention of superseded AST elements to prevent pointer mis-aliasing 2023-11-10 11:06:16 +01:00
Vern Paxson
b489cfc508 greater ZAM optimization of inlined function calls 2023-11-10 09:57:35 +01:00
Vern Paxson
fadda05782 added "-O noinline" option to turn off ZAM inlining, to help with diagnosing optimization problems 2023-11-10 09:57:35 +01:00
Vern Paxson
1dc74eaa9c fixes for a number of ZAM optimization bugs 2023-11-10 09:56:59 +01:00
Arne Welzel
ff34a4aa7f EventHandler: Deprecate SetUsed() and Used() as well.
Seems the latter isn't used outside of the functions that were deprecated
in the previous commit and with UsageAnalyzer not making use of this
information unclear why we should keep it around.

Relates to #3187.
2023-11-07 16:06:35 +01:00
Benjamin Bannier
f5a76c1aed Reformat Zeek in Spicy style
This largely copies over Spicy's `.clang-format` configuration file. The
one place where we deviate is header include order since Zeek depends on
headers being included in a certain order.
2023-10-30 09:40:55 +01:00
Vern Paxson
a36c06e5cd fix for ZAM statement-level profiling (broken by GH-3199) 2023-09-01 12:20:46 -07:00
Vern Paxson
6af0014a7b fixes for compiling lambdas to C++ 2023-08-16 17:03:37 -07:00
Vern Paxson
3f64858335 fixes for correctly tracking which functions have been fully inlined 2023-07-26 13:32:00 -07:00
Vern Paxson
91d70e6dd4 support for discarding ASTs once compiled via ZAM script optimization 2023-07-26 13:32:00 -07:00
Vern Paxson
79c53c9ed6 some code simplifications and streamlining 2023-07-20 09:35:07 -07:00
Vern Paxson
1ff490b41c more extensive ZAM inlining & compilation of lambdas 2023-07-17 16:31:30 -07:00
Tim Wojtulewicz
64b78f6fb9 Use emplace_back over push_back where appropriate 2023-07-07 09:17:05 -07:00
Vern Paxson
cb15e0d4f1 Address PR review feedback on zam-feature-complete
* insert_or_assign usage
  * master -> primary
  * FunctionIngredientsPtr
  * FuncType::Capture deprecations
  * no new ScriptFunc constructor
2023-06-30 09:59:10 +02:00
Vern Paxson
ecc93606c4 feature completeness for ZAM 2023-06-30 09:36:14 +02:00
Vern Paxson
528aa6766a more debugging information when dumping script optimization data structures 2023-06-30 09:36:14 +02:00
Vern Paxson
c173cfdbe7 added ZEEK_REPORT_UNCOMPILABLE environment variable for "-O report-uncompilable" 2023-05-05 12:00:55 -07:00
Vern Paxson
65a2900bb7 removed -O add-C++ option and updated documentation 2023-01-24 10:52:58 -08:00
Vern Paxson
134f8f2ef5 script optimization fixes:
new initialization model for standalone C++ scripts
 type coercion fix
 ZAM fix for compiling using C++ optimizer
 disambiguate empty constructors
2022-11-20 12:16:25 -08:00
Vern Paxson
ee0a6f6835 Script optimization maintenance and updates:
maintenance fixes for variadic run-time checks, '_' placeholder identifier
  "-O allow-cond" permits compiling scripts to C++ when influenced by @if conditionals
  more robust standalone compile-to-C++ properties
  fix for nested "when" statements
  test suite updates
2022-11-08 11:56:32 -08:00
Arne Welzel
2ad609cbbb Reintroduce event groups
This started with reverting commit 52cd02173d
and then rewriting it to be per handler rather than handler identifier
and adding support for hooks as well as adding implicit module groups.
2022-10-25 18:03:26 +02:00
Vern Paxson
96668ca0d3 fixes for compiling "standalone" C++ scripts 2022-09-24 14:56:42 -07:00
Tim Wojtulewicz
5ca0bb79c8 Fixes from review, post-rebase 2022-05-26 18:39:01 -07:00
Vern Paxson
d9479c0502 --no-usage-warnings flag to suppress analysis 2022-05-26 17:39:15 -07:00
Vern Paxson
fb6725a9ce "-O use-C++" now fails if no C++ bodies wound up being applicable 2022-05-12 14:08:54 -07:00
Vern Paxson
34b448ee7b update environemnt variable names to match command-line arguments 2022-05-12 14:08:17 -07:00
Vern Paxson
03194949c9 reduce interpreter frames for compiled function bodies 2022-04-01 18:00:11 -07:00
Tim Wojtulewicz
16ee942a8d Merge remote-tracking branch 'origin/topic/vern/use-C++-fix'
* origin/topic/vern/use-C++-fix:
  fix existing checks for looking to use C++ when it's not available
2022-02-04 10:33:55 -07:00