Commit graph

106 commits

Author SHA1 Message Date
Vern Paxson
0a813a53c7 -O gen-standalone-C++ fixes for recent more aggressive AST profiling 2024-12-12 14:27:12 -08:00
Tim Wojtulewicz
6e75417032 Merge remote-tracking branch 'origin/topic/vern/CPP-streamlining'
* origin/topic/vern/CPP-streamlining:
  BTest baseline updates for compile-to-C++
  mark ZAM regression BTests as not suitable for compile-to-C++
  fix for -O gen-C++ maintenance helper to skip BTest intermediary files
  introduced simplified initialization for non-standalone -O gen-C++ code tied -O gen-standalone-C++ to use of --optimize-files
  streamline generated -O C++ code by relying on per-function profiles rather than aggregate profile
  when reporting available/unavailble C++ script bodies, flag those that are skipped
  modified AST profiling to mark (and fully skip) non-optimizable functions
  modified merge_types() to skip work if given identical types, which also preserves type names (useful for -O gen-C++)
2024-12-09 12:16:00 -07:00
Arne Welzel
0ebcd9608f Merge remote-tracking branch 'origin/topic/vern/script-opt-uncompilable-AST'
* origin/topic/vern/script-opt-uncompilable-AST:
  ScriptOpt: Fail compilation if known exprs/stmts is outdated
  skip optimization of functions with AST nodes unknown to script optimization
2024-12-09 14:04:01 +01:00
Arne Welzel
28f76d5da9 ScriptOpt: Fail compilation if known exprs/stmts is outdated 2024-12-09 11:03:20 +01:00
Vern Paxson
2e69a8870a introduced simplified initialization for non-standalone -O gen-C++ code
tied -O gen-standalone-C++ to use of --optimize-files
2024-12-06 16:25:22 -08:00
Vern Paxson
79c5790bbf when reporting available/unavailble C++ script bodies, flag those that are skipped 2024-12-06 16:18:43 -08:00
Vern Paxson
62f891dcba modified AST profiling to mark (and fully skip) non-optimizable functions 2024-12-06 16:17:44 -08:00
Vern Paxson
a328185a8f internal support for script optimization options for keeping asserts, not
consolidating event handlers
2024-12-03 10:35:35 -07:00
Vern Paxson
35e157a0ab skip optimization of functions with AST nodes unknown to script optimization 2024-11-29 16:12:05 -08:00
Arne Welzel
993cdd03e0 plugin: Add InitPreExecution()
Currently, plugins do not have a way to further inspect or even mutate
script functions after ZAM optimization ran. One use-case here is
zeek-perf-support [1]. This plugin wraps Stmt instances of functions,
events and hooks hooks with a small assembly stub to support JIT map
files [2] and for integration with perf tools.

This change introduces a new InitPreExecution() hook that runs after
ZAM optimization completed, just before the zeek_init() event is enqueued.

Additionally, remove the existing CPP_activation_hook. It doesn't seem
to be used. If it becomes necessary in the future, the new
InitPreExecution() hook can be leveraged instead.

[1] https://github.com/zeek/zeek-perf-support
[2] https://github.com/torvalds/linux/blob/master/tools/perf/Documentation/jit-interface.txt
2024-11-18 13:53:56 +01:00
Vern Paxson
e0e6c462ce "-O dump-final-ZAM" option similar to "dump-ZAM" only prints final version of functions 2024-09-27 13:24:30 -07:00
Vern Paxson
a68ffc5316 extended "-O allow-cond" to apply to both gen-C++ and gen-standalone-C++ 2024-09-27 13:23:06 -07:00
Arne Welzel
71e9c8d436 script_opt: Use Func::GetName() 2024-09-27 15:11:17 +02:00
Arne Welzel
9d1d4e28b3 ScriptOpt: Ensure global statements have non-null scope
The ProfileFunc() logic assumed that GetScope() returned a non-nullptr.
This holds except for the synthetic global statements function.

Fix the latter and add an assert, also add a name to the type so it's
easier to recognize in a debugger what's going on, otherwise the name
is "".

This was found by UBSAN due to it seeing the ->OrderedVars() call on a
nullptr. Elsewhere, num_params == 0 shielded from that access and so
didn't lead to crashes.
2024-08-16 13:59:24 +02:00
Arne Welzel
ec1088c3ef Merge remote-tracking branch 'origin/topic/vern/zam-regularization'
* origin/topic/vern/zam-regularization: (33 commits)
  simpler and more robust identification of function parameters for AST profiling
  fixes to limit AST traversal in the face of recursive types
  address some script optimization compiler warnings under Linux
  fix for -O C++ construction of variable names that use multiple module namespaces
  fix for script optimization of "opaque" values that are run-time constants
  fix for script optimization of nested switch statements
  script optimization fix for complex "in" expressions in conditionals
  updates to typos allow-list reflecting ZAM regularization changes
  BTest updates for ZAM regularization changes
  convert new ZAM operations to use typed operands
  complete migration of ZAM to use only public ZVal methods
  "-O validate-ZAM" option to validate generated ZAM instructions
  internal option to suppress control-flow optimization
  exposing some functionality for greater flexibility in structuring run-time execution
  rework ZAM compilation of type switches to leverage value switches
  add tracking of control flow information
  factoring of ZAM operation specifications into separate files
  updates to ZAM operations / gen-zam regularization, other than the operations themselves
  type-checking fix for vector-of-string operations
  ZVal constructor for booleans
  ...
2024-08-16 12:10:33 +02:00
Vern Paxson
1457099df3 "-O validate-ZAM" option to validate generated ZAM instructions 2024-08-16 11:18:58 +02:00
Vern Paxson
db22448270 internal option to suppress control-flow optimization 2024-08-16 11:18:58 +02:00
Vern Paxson
1253af42f9 mark functions skipped by ZAM compilation as such 2024-08-16 11:18:57 +02:00
Vern Paxson
e6fe20f140 fix for -O report-C++ 2024-08-16 11:18:57 +02:00
Vern Paxson
857df9f063 support for more in-depth AST profiling 2024-08-16 11:18:57 +02:00
Vern Paxson
d2c6208421 allow profiling without updating of hash values 2024-08-16 11:18:57 +02:00
Vern Paxson
769a3d958a some minor tidying of -O gen-C++ sources 2024-08-13 14:29:26 -07:00
Tim Wojtulewicz
e2b03681d1 Remove EventRegistry::Used and EventRegistry::SetUsed 2024-08-07 11:58:21 -07:00
Vern Paxson
4b719ef45a script optimization for record operations sourced (in part) from other records 2024-06-24 09:38:37 +02:00
Vern Paxson
a474106fce improve ZAM's estimation of profiling overheads 2024-04-09 15:12:00 -07:00
Vern Paxson
c29db63fdd ZAM execution changes to support richer profiling 2024-03-19 10:59:49 -07:00
Vern Paxson
dc376953fa framework for --enable-ZAM-profiling configuration
profiles go to zprof.log rather than stdout
2024-03-19 10:59:49 -07:00
Vern Paxson
71a1aa0afd fix for needing to always flush optimization information for identifiers 2024-01-15 15:03:56 +01:00
Vern Paxson
709d410fcd regularized (some) types of pointers used in script optimization 2023-12-12 09:45:19 +01:00
Vern Paxson
740a087765 reworked AST optimizers analysis of side effects during aggregate operations & calls 2023-12-06 12:26:27 -08:00
Vern Paxson
3d21d80dac coalescing of event handlers (ZAM optimization) 2023-11-16 12:58:28 -08:00
Vern Paxson
4ec9a23ce6 retention of superseded AST elements to prevent pointer mis-aliasing 2023-11-10 11:06:16 +01:00
Vern Paxson
b489cfc508 greater ZAM optimization of inlined function calls 2023-11-10 09:57:35 +01:00
Vern Paxson
fadda05782 added "-O noinline" option to turn off ZAM inlining, to help with diagnosing optimization problems 2023-11-10 09:57:35 +01:00
Vern Paxson
1dc74eaa9c fixes for a number of ZAM optimization bugs 2023-11-10 09:56:59 +01:00
Arne Welzel
ff34a4aa7f EventHandler: Deprecate SetUsed() and Used() as well.
Seems the latter isn't used outside of the functions that were deprecated
in the previous commit and with UsageAnalyzer not making use of this
information unclear why we should keep it around.

Relates to #3187.
2023-11-07 16:06:35 +01:00
Benjamin Bannier
f5a76c1aed Reformat Zeek in Spicy style
This largely copies over Spicy's `.clang-format` configuration file. The
one place where we deviate is header include order since Zeek depends on
headers being included in a certain order.
2023-10-30 09:40:55 +01:00
Vern Paxson
a36c06e5cd fix for ZAM statement-level profiling (broken by GH-3199) 2023-09-01 12:20:46 -07:00
Vern Paxson
6af0014a7b fixes for compiling lambdas to C++ 2023-08-16 17:03:37 -07:00
Vern Paxson
3f64858335 fixes for correctly tracking which functions have been fully inlined 2023-07-26 13:32:00 -07:00
Vern Paxson
91d70e6dd4 support for discarding ASTs once compiled via ZAM script optimization 2023-07-26 13:32:00 -07:00
Vern Paxson
79c53c9ed6 some code simplifications and streamlining 2023-07-20 09:35:07 -07:00
Vern Paxson
1ff490b41c more extensive ZAM inlining & compilation of lambdas 2023-07-17 16:31:30 -07:00
Tim Wojtulewicz
64b78f6fb9 Use emplace_back over push_back where appropriate 2023-07-07 09:17:05 -07:00
Vern Paxson
cb15e0d4f1 Address PR review feedback on zam-feature-complete
* insert_or_assign usage
  * master -> primary
  * FunctionIngredientsPtr
  * FuncType::Capture deprecations
  * no new ScriptFunc constructor
2023-06-30 09:59:10 +02:00
Vern Paxson
ecc93606c4 feature completeness for ZAM 2023-06-30 09:36:14 +02:00
Vern Paxson
528aa6766a more debugging information when dumping script optimization data structures 2023-06-30 09:36:14 +02:00
Vern Paxson
c173cfdbe7 added ZEEK_REPORT_UNCOMPILABLE environment variable for "-O report-uncompilable" 2023-05-05 12:00:55 -07:00
Vern Paxson
65a2900bb7 removed -O add-C++ option and updated documentation 2023-01-24 10:52:58 -08:00
Vern Paxson
134f8f2ef5 script optimization fixes:
new initialization model for standalone C++ scripts
 type coercion fix
 ZAM fix for compiling using C++ optimizer
 disambiguate empty constructors
2022-11-20 12:16:25 -08:00