Mirror/zeek
1
0
Fork 0
mirror of https://github.com/zeek/zeek.git synced 2025-10-02 14:48:21 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 2

Merge remote-tracking branch 'origin/topic/vern/zam-regularization'

Browse source 
* origin/topic/vern/zam-regularization: (33 commits)
  simpler and more robust identification of function parameters for AST profiling
  fixes to limit AST traversal in the face of recursive types
  address some script optimization compiler warnings under Linux
  fix for -O C++ construction of variable names that use multiple module namespaces
  fix for script optimization of "opaque" values that are run-time constants
  fix for script optimization of nested switch statements
  script optimization fix for complex "in" expressions in conditionals
  updates to typos allow-list reflecting ZAM regularization changes
  BTest updates for ZAM regularization changes
  convert new ZAM operations to use typed operands
  complete migration of ZAM to use only public ZVal methods
  "-O validate-ZAM" option to validate generated ZAM instructions
  internal option to suppress control-flow optimization
  exposing some functionality for greater flexibility in structuring run-time execution
  rework ZAM compilation of type switches to leverage value switches
  add tracking of control flow information
  factoring of ZAM operation specifications into separate files
  updates to ZAM operations / gen-zam regularization, other than the operations themselves
  type-checking fix for vector-of-string operations
  ZVal constructor for booleans
  ...
This commit is contained in:
Arne Welzel 2024-08-16 12:08:48 +02:00
commit ec1088c3ef
78 changed files with 4410 additions and 3917 deletions
Download patch file Download diff file Expand all files Collapse all files

9
.typos.toml
View file

@ -9,6 +9,7 @@ extend-ignore-re = [
"Remove in v6.1.*SupressWeird",
"max_repititions:.*Remove in v6.1",
"mis-aliasing of",
"mis-indexing",
# On purpose
"\"THE NETBIOS NAM\"",
# NFS stuff.
@ -20,6 +21,12 @@ extend-ignore-re = [
"ot->Tag\\(\\) == TYPE_.*",
"auto.* ot =",
"ot = OP_.*",
"ot\\[",
"ot.size",
"ot.empty",
"ot_i",
"ot.c_str",
"have_ot",
"if \\( ot == OP_.*",
"ot->Yield\\(\\)->InternalType\\(\\)",
"switch \\( ot \\)",
@ -53,7 +60,7 @@ ND_REDIRECT = "ND_REDIRECT"
NED_ACK = "NED_ACK"
NFS3ERR_ACCES = "NFS3ERR_ACCES"
NO_SEH = "NO_SEH"
OP_SWITCHS_VVV = "OP_SWITCHS_VVV"
OP_SWITCHS_Vii = "OP_SWITCHS_Vii"
O_WRONLY = "O_WRONLY"
RPC_NT_CALL_FAILED_DNE = "RPC_NT_CALL_FAILED_DNE"
RpcAddPrintProvidor = "RpcAddPrintProvidor"

68
CHANGES
View file

@ -1,3 +1,71 @@
7.1.0-dev.198 | 2024-08-16 12:10:00 +0200
* simpler and more robust identification of function parameters for AST profiling (Vern Paxson, Corelight)
* fixes to limit AST traversal in the face of recursive types (Vern Paxson, Corelight)
* address some script optimization compiler warnings under Linux (Vern Paxson, Corelight)
* fix for -O C++ construction of variable names that use multiple module namespaces (Vern Paxson, Corelight)
* fix for script optimization of "opaque" values that are run-time constants (Vern Paxson, Corelight)
* fix for script optimization of nested switch statements (Vern Paxson, Corelight)
* script optimization fix for complex "in" expressions in conditionals (Vern Paxson, Corelight)
* updates to typos allow-list reflecting ZAM regularization changes (Vern Paxson, Corelight)
* BTest updates for ZAM regularization changes (Vern Paxson, Corelight)
* convert new ZAM operations to use typed operands (Vern Paxson, Corelight)
* complete migration of ZAM to use only public ZVal methods (Vern Paxson, Corelight)
* "-O validate-ZAM" option to validate generated ZAM instructions (Vern Paxson, Corelight)
* internal option to suppress control-flow optimization (Vern Paxson, Corelight)
* exposing some functionality for greater flexibility in structuring run-time execution (Vern Paxson, Corelight)
* rework ZAM compilation of type switches to leverage value switches (Vern Paxson, Corelight)
* add tracking of control flow information (Vern Paxson, Corelight)
* factoring of ZAM operation specifications into separate files (Vern Paxson, Corelight)
* updates to ZAM operations / gen-zam regularization, other than the operations themselves (Vern Paxson, Corelight)
* type-checking fix for vector-of-string operations (Vern Paxson, Corelight)
* ZVal constructor for booleans (Vern Paxson, Corelight)
* fix for nit in base/protocols/krb/main.zeek (Vern Paxson, Corelight)
* mark functions skipped by ZAM compilation as such (Vern Paxson, Corelight)
* fix for avoiding inadvertent interpreter errors in CallExpr::IsPure() (Vern Paxson, Corelight)
* support for traversing ZAM code similar to AST traversal (Vern Paxson, Corelight)
* run-time warnings for scripts compiled to C++ (Vern Paxson, Corelight)
* allow C++ script compiler access to type internals (Vern Paxson, Corelight)
* fixes for script optimization of coerce-to-any expressions (Vern Paxson, Corelight)
* fix to correctly track whether a capture needs deep-copying (Vern Paxson, Corelight)
* fix for -O report-C++ (Vern Paxson, Corelight)
* support for more in-depth AST profiling (Vern Paxson, Corelight)
* allow profiling without updating of hash values (Vern Paxson, Corelight)
* ListVal method to clear the list to allow reusing w/o new construction (Vern Paxson, Corelight)
* accessor for smart-pointer version of FileVal's value (Vern Paxson, Corelight)
7.1.0-dev.164 | 2024-08-15 10:30:37 +0200
* Func: Add SetCapturesVec() (Arne Welzel, Corelight)

2
VERSION
View file

@ -1 +1 @@
7.1.0-dev.164
7.1.0-dev.198

2
auxil/gen-zam

@ -1 +1 @@
Subproject commit 610cf8527dad7033b971595a1d556c2c95294f2b
Subproject commit cfc0c7b9de63f44419c2a57040ae6b7081a66a33

3
scripts/base/protocols/krb/main.zeek
View file

@ -190,8 +190,7 @@ event krb_as_response(c: connection, msg: KDC_Response) &priority=-5
event krb_ap_request(c: connection, ticket: KRB::Ticket, opts: KRB::AP_Options) &priority=5
{
if ( set_session(c) )
return;
set_session(c);
}
event krb_tgs_request(c: connection, msg: KDC_Request) &priority=5

19
src/CMakeLists.txt
View file

@ -162,7 +162,23 @@ list(APPEND BINPAC_OUTPUTS "${BINPAC_OUTPUT_CC}")
include(Gen-ZAM)
set(GEN_ZAM_SRC_DIR ${CMAKE_CURRENT_SOURCE_DIR}/script_opt/ZAM/OPs)
set(GEN_ZAM_SRC ${GEN_ZAM_SRC_DIR}/ZAM.op)
set(ZAM_OP_SRCS
${GEN_ZAM_SRC_DIR}/aggr-assignments.op
${GEN_ZAM_SRC_DIR}/binary-exprs.op
${GEN_ZAM_SRC_DIR}/calls.op
${GEN_ZAM_SRC_DIR}/coercions.op
${GEN_ZAM_SRC_DIR}/constructors.op
${GEN_ZAM_SRC_DIR}/indexing.op
${GEN_ZAM_SRC_DIR}/internal.op
${GEN_ZAM_SRC_DIR}/iterations.op
${GEN_ZAM_SRC_DIR}/macros.op
${GEN_ZAM_SRC_DIR}/non-uniform.op
${GEN_ZAM_SRC_DIR}/rel-exprs.op
${GEN_ZAM_SRC_DIR}/script-idioms.op
${GEN_ZAM_SRC_DIR}/stmts.op
${GEN_ZAM_SRC_DIR}/unary-exprs.op
${GEN_ZAM_SRC_DIR}/ZBI.op)
set(GEN_ZAM_SRC ${ZAM_OP_SRCS})
gen_zam_target(${GEN_ZAM_SRC_DIR})
@ -430,6 +446,7 @@ set(MAIN_SRCS
script_opt/ZAM/Profile.cc
script_opt/ZAM/Stmt.cc
script_opt/ZAM/Support.cc
script_opt/ZAM/Validate.cc
script_opt/ZAM/Vars.cc
script_opt/ZAM/ZBody.cc
script_opt/ZAM/ZInst.cc

14
src/Expr.cc
View file

@ -1985,8 +1985,10 @@ EqExpr::EqExpr(ExprTag arg_tag, ExprPtr arg_op1, ExprPtr arg_op2)
}
}
else if ( bt1 == TYPE_PATTERN && bt2 == TYPE_STRING )
;
else if ( (bt1 == TYPE_PATTERN && bt2 == TYPE_STRING) || (bt1 == TYPE_STRING && bt2 == TYPE_PATTERN) ) {
if ( op1->GetType()->Tag() == TYPE_VECTOR )
ExprError("cannot compare string vectors with pattern vectors");
}
else
ExprError("type clash in comparison");
@ -4063,11 +4065,15 @@ bool CallExpr::IsPure() const {
if ( IsError() )
return true;
if ( ! func->IsPure() )
if ( func->Tag() != EXPR_NAME )
// Indirect call, can't resolve up front.
return false;
auto func_val = func->Eval(nullptr);
auto func_id = func->AsNameExpr()->Id();
if ( ! func_id->IsGlobal() )
return false;
auto func_val = func_id->GetVal();
if ( ! func_val )
return false;

3
src/Expr.h
View file

@ -1651,6 +1651,9 @@ class CoerceToAnyExpr : public UnaryExpr {
public:
CoerceToAnyExpr(ExprPtr op);
bool IsReduced(Reducer* c) const override;
ExprPtr Reduce(Reducer* c, StmtPtr& red_stmt) override;
protected:
ValPtr Fold(Val* v) const override;

6
src/Func.h
View file

@ -300,6 +300,9 @@ protected:
*/
virtual void SetCaptures(Frame* f);
// Captures when using ZVal block instead of a Frame.
std::unique_ptr<std::vector<ZVal>> captures_vec;
private:
size_t frame_size = 0;
@ -313,9 +316,6 @@ private:
OffsetMap* captures_offset_mapping = nullptr;
// Captures when using ZVal block instead of a Frame.
std::unique_ptr<std::vector<ZVal>> captures_vec;
// The most recently added/updated body ...
StmtPtr current_body;

11
src/ID.cc
View file

@ -100,20 +100,15 @@ ID::ID(const char* arg_name, IDScope arg_scope, bool arg_is_export) {
name = util::copy_string(arg_name);
scope = arg_scope;
is_export = arg_is_export;
is_option = false;
is_blank = name && extract_var_name(name) == "_";
is_const = false;
is_enum_const = false;
is_type = false;
offset = 0;
if ( is_blank )
if ( name && extract_var_name(name) == "_" ) {
is_blank = true;
SetType(base_type(TYPE_ANY));
}
opt_info = new IDOptInfo(this);
infer_return_type = false;
SetLocationInfo(&start_location, &end_location);
}

10
src/ID.h
View file

@ -81,7 +81,6 @@ public:
}
bool IsType() const { return is_type; }
void MakeType() { is_type = true; }
void SetVal(ValPtr v);
@ -160,9 +159,14 @@ protected:
const char* name;
IDScope scope;
bool is_export;
bool infer_return_type;
TypePtr type;
bool is_const, is_enum_const, is_type, is_option, is_blank;
bool is_capture = false;
bool is_const = false;
bool is_enum_const = false;
bool is_type = false;
bool is_option = false;
bool is_blank = false;
bool infer_return_type = false;
int offset;
ValPtr val;
AttributesPtr attrs;

9
src/Options.cc
View file

@ -194,6 +194,7 @@ static void print_analysis_help() {
fprintf(stderr, " optimize-AST optimize the (transformed) AST; implies xform\n");
fprintf(stderr, " profile-ZAM generate to zprof.out a ZAM execution profile; implies -O ZAM\n");
fprintf(stderr, " report-recursive report on recursive functions and exit\n");
fprintf(stderr, " validate-ZAM perform internal assessment of synthesized ZAM instructions and exit\n");
fprintf(stderr, " xform transform scripts to \"reduced\" form\n");
fprintf(stderr, "\n--optimize options when generating C++:\n");
@ -220,14 +221,14 @@ static void set_analysis_option(const char* opt, Options& opts) {
exit(0);
}
if ( util::streq(opt, "dump-uds") )
if ( util::streq(opt, "allow-cond") )
a_o.allow_cond = true;
else if ( util::streq(opt, "dump-uds") )
a_o.activate = a_o.dump_uds = true;
else if ( util::streq(opt, "dump-xform") )
a_o.activate = a_o.dump_xform = true;
else if ( util::streq(opt, "dump-ZAM") )
a_o.activate = a_o.dump_ZAM = true;
else if ( util::streq(opt, "allow-cond") )
a_o.allow_cond = true;
else if ( util::streq(opt, "gen-C++") )
a_o.gen_CPP = true;
else if ( util::streq(opt, "gen-standalone-C++") )
@ -254,6 +255,8 @@ static void set_analysis_option(const char* opt, Options& opts) {
a_o.report_uncompilable = true;
else if ( util::streq(opt, "use-C++") )
a_o.use_CPP = true;
else if ( util::streq(opt, "validate-ZAM") )
a_o.validate_ZAM = true;
else if ( util::streq(opt, "xform") )
a_o.activate = true;

8
src/Reporter.cc
View file

@ -203,6 +203,14 @@ void Reporter::CPPRuntimeError(const char* fmt, ...) {
throw InterpreterException();
}
void Reporter::CPPRuntimeWarning(const char* fmt, ...) {
va_list ap;
va_start(ap, fmt);
FILE* out = EmitToStderr(warnings_to_stderr) ? stderr : nullptr;
DoLog("runtime warning in compiled code", reporter_error, out, nullptr, nullptr, true, true, "", fmt, ap);
va_end(ap);
}
void Reporter::InternalError(const char* fmt, ...) {
va_list ap;
va_start(ap, fmt);

3
src/Reporter.h
View file

@ -115,6 +115,9 @@ public:
// function will not return but raise an InterpreterException.
[[noreturn]] void CPPRuntimeError(const char* fmt, ...) __attribute__((format(printf, 2, 3)));
// Similar, but for warnings. This function does return.
void CPPRuntimeWarning(const char* fmt, ...) __attribute__((format(printf, 2, 3)));
// Report a traffic weirdness, i.e., an unexpected protocol situation
// that may lead to incorrectly processing a connection.
void Weird(const char* name, const char* addl = "",

10
src/ScriptValidation.cc
View file

@ -58,6 +58,15 @@ public:
return TC_CONTINUE;
}
TraversalCode PreType(const Type* t) override {
if ( types_seen.count(t) > 0 )
return TC_ABORTSTMT;
types_seen.insert(t);
return TC_CONTINUE;
}
void SetHookDepth(int hd) { hook_depth = hd; }
bool IsValid() const { return valid_script; }
@ -83,6 +92,7 @@ private:
}
std::unordered_map<StmtTag, int> stmt_depths;
std::unordered_set<const Type*> types_seen;
int hook_depth = 0;
bool report; // whether to report problems via "reporter"
bool valid_script = true;

3
src/Stmt.cc
View file

@ -1843,7 +1843,8 @@ void WhenInfo::Build(StmtPtr ws) {
auto else_branch = timeout_s ? timeout_s : empty;
auto do_bodies = make_intrusive<IfStmt>(two_test, s, else_branch);
auto dummy_return = make_intrusive<ReturnStmt>(true_const);
auto any_true_const = make_intrusive<CoerceToAnyExpr>(true_const);
auto dummy_return = make_intrusive<ReturnStmt>(any_true_const);
auto shebang = make_intrusive<StmtList>(do_test, do_bodies, dummy_return);

3
src/Type.cc
View file

@ -601,6 +601,9 @@ SetType::~SetType() = default;
FuncType::Capture::Capture(detail::IDPtr _id, bool _deep_copy) : id(std::move(_id)), deep_copy(_deep_copy) {
is_managed = id ? ZVal::IsManagedType(id->GetType()) : false;
if ( ! is_managed )
// For non-managed types, deep copying isn't applicable.
deep_copy = false;
}
FuncType::FuncType(RecordTypePtr arg_args, TypePtr arg_yield, FunctionFlavor arg_flavor)

3
src/Type.h
View file

@ -342,6 +342,9 @@ public:
void Append(TypePtr t);
void AppendEvenIfNotPure(TypePtr t);
// Resets the list to be empty.
void Clear() { types.clear(); }
detail::TraversalCode Traverse(detail::TraversalCallback* cb) const override;
protected:

2
src/Val.cc
View file

@ -1277,6 +1277,8 @@ FileVal::FileVal(FilePtr f) : Val(make_intrusive<FileType>(base_type(TYPE_STRING
assert(file_val->GetType()->Tag() == TYPE_STRING);
}
FilePtr FileVal::AsFilePtr() const { return file_val; }
ValPtr FileVal::SizeVal() const { return make_intrusive<DoubleVal>(file_val->Size()); }
void FileVal::ValDescribe(ODesc* d) const { file_val->Describe(d); }

10
src/Val.h
View file

@ -609,6 +609,8 @@ class FileVal final : public Val {
public:
explicit FileVal(FilePtr f);
FilePtr AsFilePtr() const;
ValPtr SizeVal() const override;
File* Get() const { return file_val.get(); }
@ -682,6 +684,14 @@ public:
*/
void Append(ValPtr v);
/**
* Empties the list.
*/
void Clear() {
vals.clear();
type->AsTypeList()->Clear();
}
// Returns a Set representation of the list (which must be homogeneous).
TableValPtr ToSetVal() const;

2
src/ZVal.h
View file

@ -62,6 +62,7 @@ union ZVal {
ZVal(const TypePtr& t);
// Construct directly.
ZVal(bool v) { int_val = v; }
ZVal(zeek_int_t v) { int_val = v; }
ZVal(zeek_uint_t v) { uint_val = v; }
ZVal(double v) { double_val = v; }
@ -160,7 +161,6 @@ union ZVal {
private:
friend class RecordVal;
friend class VectorVal;
friend class zeek::detail::ZBody;
// Used for bool, int, enum.
zeek_int_t int_val;

10
src/script_opt/CSE.h
View file

@ -24,6 +24,13 @@ public:
TraversalCode PreExpr(const Expr*) override;
TraversalCode PostExpr(const Expr*) override;
TraversalCode PreType(const Type* t) override {
if ( types_seen.count(t) > 0 )
return TC_ABORTSTMT;
types_seen.insert(t);
return TC_CONTINUE;
}
// Returns the ultimate verdict re safety.
bool IsValid() const {
if ( ! is_valid )
@ -105,6 +112,9 @@ protected:
//
// A count to allow for nesting.
int in_aggr_mod_expr = 0;
// Used to limit traversal of recursive types.
std::unordered_set<const Type*> types_seen;
};
// Used for debugging, to communicate which expression wasn't

17
src/script_opt/Expr.cc
View file

@ -3114,6 +3114,23 @@ CoerceToAnyExpr::CoerceToAnyExpr(ExprPtr arg_op) : UnaryExpr(EXPR_TO_ANY_COERCE,
type = base_type(TYPE_ANY);
}
bool CoerceToAnyExpr::IsReduced(Reducer* c) const { return HasReducedOps(c); }
ExprPtr CoerceToAnyExpr::Reduce(Reducer* c, StmtPtr& red_stmt) {
if ( c->Optimizing() )
op = c->UpdateExpr(op);
red_stmt = nullptr;
if ( ! op->IsSingleton(c) )
op = op->ReduceToSingleton(c, red_stmt);
if ( c->Optimizing() )
return ThisPtr();
else
return AssignToTemporary(c, red_stmt);
}
ValPtr CoerceToAnyExpr::Fold(Val* v) const { return {NewRef{}, v}; }
ExprPtr CoerceToAnyExpr::Duplicate() { return SetSucc(new CoerceToAnyExpr(op->Duplicate())); }

84
src/script_opt/ProfileFunc.cc
View file

@ -24,11 +24,12 @@ p_hash_type p_hash(const Obj* o) {
ProfileFunc::ProfileFunc(const Func* func, const StmtPtr& body, bool _abs_rec_fields) {
profiled_func = func;
profiled_scope = profiled_func->GetScope();
profiled_body = body.get();
abs_rec_fields = _abs_rec_fields;
auto ft = func->GetType()->AsFuncType();
auto& fcaps = ft->GetCaptures();
profiled_func_t = cast_intrusive<FuncType>(func->GetType());
auto& fcaps = profiled_func_t->GetCaptures();
if ( fcaps ) {
int offset = 0;
@ -40,7 +41,7 @@ ProfileFunc::ProfileFunc(const Func* func, const StmtPtr& body, bool _abs_rec_fi
}
}
Profile(ft, body);
Profile(profiled_func_t.get(), body);
}
ProfileFunc::ProfileFunc(const Stmt* s, bool _abs_rec_fields) {
@ -56,6 +57,9 @@ ProfileFunc::ProfileFunc(const Expr* e, bool _abs_rec_fields) {
if ( e->Tag() == EXPR_LAMBDA ) {
auto func = e->AsLambdaExpr();
ASSERT(func->GetType()->Tag() == TYPE_FUNC);
profiled_scope = func->GetScope();
profiled_func_t = cast_intrusive<FuncType>(func->GetType());
int offset = 0;
@ -75,6 +79,11 @@ ProfileFunc::ProfileFunc(const Expr* e, bool _abs_rec_fields) {
void ProfileFunc::Profile(const FuncType* ft, const StmtPtr& body) {
num_params = ft->Params()->NumFields();
auto& ov = profiled_scope->OrderedVars();
for ( int i = 0; i < num_params; ++i )
params.insert(ov[i].get());
TrackType(ft);
body->Traverse(this);
}
@ -181,28 +190,10 @@ TraversalCode ProfileFunc::PreExpr(const Expr* e) {
TrackType(id->GetType());
if ( id->IsGlobal() ) {
globals.insert(id);
all_globals.insert(id);
const auto& t = id->GetType();
if ( t->Tag() == TYPE_FUNC )
if ( t->AsFuncType()->Flavor() == FUNC_FLAVOR_EVENT )
events.insert(id->Name());
PreID(id);
break;
}
// This is a tad ugly. Unfortunately due to the weird way
// that Zeek function *declarations* work, there's no reliable
// way to get the list of parameters for a function *definition*,
// since they can have different names than what's present in the
// declaration. So we identify them directly, by knowing that
// they come at the beginning of the frame ... and being careful
// to avoid misconfusing a lambda capture with a low frame offset
// as a parameter.
if ( captures.count(id) == 0 && id->Offset() < num_params )
params.insert(id);
locals.insert(id);
break;
@ -426,11 +417,6 @@ TraversalCode ProfileFunc::PreExpr(const Expr* e) {
for ( const auto& i : l->OuterIDs() ) {
locals.insert(i);
TrackID(i);
// See above re EXPR_NAME regarding the following
// logic.
if ( captures.count(i) == 0 && i->Offset() < num_params )
params.insert(i);
}
// In general, we don't want to recurse into the body.
@ -487,10 +473,29 @@ TraversalCode ProfileFunc::PreExpr(const Expr* e) {
TraversalCode ProfileFunc::PreID(const ID* id) {
TrackID(id);
if ( id->IsGlobal() ) {
globals.insert(id);
all_globals.insert(id);
const auto& t = id->GetType();
TrackType(t);
if ( t->Tag() == TYPE_FUNC )
if ( t->AsFuncType()->Flavor() == FUNC_FLAVOR_EVENT )
events.insert(id->Name());
}
// There's no need for any further analysis of this ID.
return TC_ABORTSTMT;
}
TraversalCode ProfileFunc::PreType(const Type* t) {
TrackType(t);
// There's no need for any further analysis of this type.
return TC_ABORTSTMT;
}
void ProfileFunc::TrackType(const Type* t) {
if ( ! t )
return;
@ -514,6 +519,11 @@ void ProfileFunc::TrackID(const ID* id) {
// Already tracked.
return;
if ( id->IsGlobal() ) {
globals.insert(id);
all_globals.insert(id);
}
ordered_ids.push_back(id);
}
@ -546,7 +556,9 @@ void ProfileFunc::CheckRecordConstructor(TypePtr t) {
}
}
ProfileFuncs::ProfileFuncs(std::vector<FuncInfo>& funcs, is_compilable_pred pred, bool _full_record_hashes) {
ProfileFuncs::ProfileFuncs(std::vector<FuncInfo>& funcs, is_compilable_pred pred, bool _compute_func_hashes,
bool _full_record_hashes) {
compute_func_hashes = _compute_func_hashes;
full_record_hashes = _full_record_hashes;
for ( auto& f : funcs ) {
@ -558,6 +570,11 @@ ProfileFuncs::ProfileFuncs(std::vector<FuncInfo>& funcs, is_compilable_pred pred
// Track the profile even if we're not compiling the function, since
// the AST optimizer will still need it to reason about function-call
// side effects.
// Propagate previous hash if requested.
if ( ! compute_func_hashes && f.Profile() )
pf->SetHashVal(f.Profile()->HashVal());
f.SetProfile(std::move(pf));
func_profs[f.Func()] = f.ProfilePtr();
}
@ -805,15 +822,18 @@ void ProfileFuncs::ComputeTypeHashes(const std::vector<const Type*>& types) {
}
void ProfileFuncs::ComputeBodyHashes(std::vector<FuncInfo>& funcs) {
for ( auto& f : funcs )
if ( ! f.ShouldSkip() )
ComputeProfileHash(f.ProfilePtr());
if ( compute_func_hashes )
for ( auto& f : funcs )
if ( ! f.ShouldSkip() )
ComputeProfileHash(f.ProfilePtr());
for ( auto& l : lambdas ) {
auto pf = ExprProf(l);
func_profs[l->PrimaryFunc().get()] = pf;
lambda_primaries[l->Name()] = l->PrimaryFunc().get();
ComputeProfileHash(pf);
if ( compute_func_hashes )
ComputeProfileHash(pf);
}
}

23
src/script_opt/ProfileFunc.h
View file

@ -83,6 +83,7 @@ public:
// Returns the function, body, or expression profiled. Each can be
// null depending on the constructor used.
const Func* ProfiledFunc() const { return profiled_func; }
const ScopePtr& ProfiledScope() const { return profiled_scope; }
const Stmt* ProfiledBody() const { return profiled_body; }
const Expr* ProfiledExpr() const { return profiled_expr; }
@ -139,6 +140,7 @@ protected:
TraversalCode PreStmt(const Stmt*) override;
TraversalCode PreExpr(const Expr*) override;
TraversalCode PreID(const ID*) override;
TraversalCode PreType(const Type*) override;
// Take note of the presence of a given type.
void TrackType(const Type* t);
@ -157,6 +159,8 @@ protected:
// The function, body, or expression profiled. Can be null
// depending on which constructor was used.
const Func* profiled_func = nullptr;
ScopePtr profiled_scope; // null when not in a full function context
FuncTypePtr profiled_func_t; // null when not in a full function context
const Stmt* profiled_body = nullptr;
const Expr* profiled_expr = nullptr;
@ -347,13 +351,15 @@ using is_compilable_pred = bool (*)(const ProfileFunc*, const char** reason);
// Collectively profile an entire collection of functions.
class ProfileFuncs {
public:
// Updates entries in "funcs" to include profiles. If pred is
// non-nil, then it is called for each profile to see whether it's
// compilable, and, if not, the FuncInfo is marked as ShouldSkip().
// "full_record_hashes" controls whether the hashes for extended
// records covers their final, full form, or should only their
// original fields.
ProfileFuncs(std::vector<FuncInfo>& funcs, is_compilable_pred pred, bool full_record_hashes);
// Updates entries in "funcs" to include profiles. If pred is non-nil,
// then it is called for each profile to see whether it's compilable,
// and, if not, the FuncInfo is marked as ShouldSkip().
// "compute_func_hashes" governs whether we compute hashes for the
// FuncInfo entries, or keep their existing ones. "full_record_hashes"
// controls whether the hashes for extended records covers their final,
// full form, or should only their original fields.
ProfileFuncs(std::vector<FuncInfo>& funcs, is_compilable_pred pred, bool compute_func_hashes,
bool full_record_hashes);
// The following accessors provide a global profile across all of
// the (non-skipped) functions in "funcs". See the comments for
@ -604,6 +610,9 @@ protected:
// These can arise for example due to lambdas or record attributes.
std::vector<const Expr*> pending_exprs;
// Whether to compute new hashes for the FuncInfo entries.
bool compute_func_hashes;
// Whether the hashes for extended records should cover their final,
// full form, or only their original fields.
bool full_record_hashes;

43
src/script_opt/ScriptOpt.cc
View file

@ -271,6 +271,7 @@ static void init_options() {
check_env_opt("ZEEK_REPORT_UNCOMPILABLE", analysis_options.report_uncompilable);
check_env_opt("ZEEK_ZAM_CODE", analysis_options.gen_ZAM_code);
check_env_opt("ZEEK_NO_ZAM_OPT", analysis_options.no_ZAM_opt);
check_env_opt("ZEEK_NO_ZAM_CONTROL_FLOW_OPT", analysis_options.no_ZAM_control_flow_opt);
check_env_opt("ZEEK_DUMP_ZAM", analysis_options.dump_ZAM);
check_env_opt("ZEEK_PROFILE", analysis_options.profile_ZAM);
@ -391,7 +392,7 @@ static void use_CPP() {
int num_used = 0;
auto pfs = std::make_unique<ProfileFuncs>(funcs, is_CPP_compilable, false);
auto pfs = std::make_unique<ProfileFuncs>(funcs, is_CPP_compilable, true, false);
for ( auto& f : funcs ) {
auto hash = f.Profile()->HashVal();
@ -429,18 +430,16 @@ static void use_CPP() {
reporter->FatalError("no C++ functions found to use");
}
static void generate_CPP() {
static void generate_CPP(std::shared_ptr<ProfileFuncs> pfs) {
const auto gen_name = CPP_dir + "CPP-gen.cc";
const bool standalone = analysis_options.gen_standalone_CPP;
const bool report = analysis_options.report_uncompilable;
auto pfs = std::make_shared<ProfileFuncs>(funcs, is_CPP_compilable, false);
CPPCompile cpp(funcs, pfs, gen_name, standalone, report);
}
static void analyze_scripts_for_ZAM() {
static void analyze_scripts_for_ZAM(std::shared_ptr<ProfileFuncs> pfs) {
if ( analysis_options.usage_issues > 0 && analysis_options.optimize_AST ) {
fprintf(stderr,
"warning: \"-O optimize-AST\" option is incompatible with -u option, "
@ -448,8 +447,6 @@ static void analyze_scripts_for_ZAM() {
analysis_options.optimize_AST = false;
}
auto pfs = std::make_shared<ProfileFuncs>(funcs, nullptr, true);
if ( analysis_options.profile_ZAM ) {
#ifdef ENABLE_ZAM_PROFILE
AST_blocks = std::make_unique<ASTBlockAnalyzer>(funcs);
@ -505,12 +502,12 @@ static void analyze_scripts_for_ZAM() {
if ( ! analysis_options.compile_all && ! is_lambda && inl && inl->WasFullyInlined(func.get()) &&
func_used_indirectly.count(func.get()) == 0 ) {
// No need to compile as it won't be called directly.
// We'd like to zero out the body to recover the
// memory, but a *few* such functions do get called,
// such as by the event engine reaching up, or
// BiFs looking for them, so we can't safely zero
// them.
// No need to compile as it won't be called directly. We'd
// like to zero out the body to recover the memory, but a *few*
// such functions do get called, such as by the event engine
// reaching up, or BiFs looking for them, so we can't safely
// zero them.
f.SetSkip(true);
continue;
}
@ -558,6 +555,11 @@ void clear_script_analysis() {
void analyze_scripts(bool no_unused_warnings) {
init_options();
if ( analysis_options.validate_ZAM ) {
validate_ZAM_insts();
return;
}
// Any standalone compiled scripts have already been instantiated
// at this point, but may require post-loading-of-scripts finalization.
for ( auto cb : standalone_finalizations )
@ -601,6 +603,7 @@ void analyze_scripts(bool no_unused_warnings) {
}
if ( analysis_options.report_CPP ) {
auto pfs = std::make_unique<ProfileFuncs>(funcs, is_CPP_compilable, true, false);
report_CPP();
exit(0);
}
@ -608,17 +611,23 @@ void analyze_scripts(bool no_unused_warnings) {
if ( analysis_options.use_CPP )
use_CPP();
std::shared_ptr<ProfileFuncs> pfs;
// Note, in the following it's not clear whether the final argument
// for absolute/relative record fields matters any more ...
if ( generating_CPP )
pfs = std::make_shared<ProfileFuncs>(funcs, is_CPP_compilable, true, false);
else
pfs = std::make_shared<ProfileFuncs>(funcs, nullptr, true, true);
if ( generating_CPP ) {
if ( analysis_options.gen_ZAM )
reporter->FatalError("-O ZAM and -O gen-C++ conflict");
generate_CPP();
generate_CPP(pfs);
exit(0);
}
// At this point we're done with C++ considerations, so instead
// are compiling to ZAM.
analyze_scripts_for_ZAM();
analyze_scripts_for_ZAM(pfs);
if ( reporter->Errors() > 0 )
reporter->FatalError("Optimized script execution aborted due to errors");

12
src/script_opt/ScriptOpt.h
View file

@ -61,6 +61,10 @@ struct AnalyOpt {
// recursive, and exit. Only germane if running the inliner.
bool report_recursive = false;
// If true, assess the instructions generated from ZAM templates
// for validity, and exit.
bool validate_ZAM = false;
// If true, generate ZAM code for applicable function bodies,
// activating all optimizations.
bool gen_ZAM = false;
@ -72,6 +76,9 @@ struct AnalyOpt {
// Deactivate the low-level ZAM optimizer.
bool no_ZAM_opt = false;
// Deactivate ZAM optimization of control flow.
bool no_ZAM_control_flow_opt = false;
// Produce a profile of ZAM execution.
bool profile_ZAM = false;
@ -241,6 +248,11 @@ extern bool should_analyze(const ScriptFuncPtr& f, const StmtPtr& body);
// suppressed by the flag) and optimization.
extern void analyze_scripts(bool no_unused_warnings);
// Conduct internal validation of ZAM instructions. Upon success, generates
// a terse report to stdout. Exits with an internal error if a problem is
// encountered.
extern void validate_ZAM_insts();
// Called when all script processing is complete and we can discard
// unused ASTs and associated state.
extern void clear_script_analysis();

148
src/script_opt/ZAM/AM-Opt.cc
View file

@ -144,6 +144,9 @@ bool ZAMCompiler::RemoveDeadCode() {
if ( ! i0->live )
continue;
if ( analysis_options.no_ZAM_control_flow_opt )
continue;
auto i1 = NextLiveInst(i0);
// Look for degenerate branches.
@ -181,6 +184,9 @@ bool ZAMCompiler::RemoveDeadCode() {
}
bool ZAMCompiler::CollapseGoTos() {
if ( analysis_options.no_ZAM_control_flow_opt )
return false;
bool did_change = false;
for ( auto& i0 : insts1 ) {
@ -303,7 +309,7 @@ bool ZAMCompiler::PruneUnused() {
if ( assignmentless_op.count(inst->op) == 0 )
reporter->InternalError("inconsistency in re-flavoring instruction with side effects");
inst->op_type = assignmentless_op_type[inst->op];
inst->op_type = assignmentless_op_class[inst->op];
inst->op = assignmentless_op[inst->op];
inst->v1 = inst->v2;
@ -336,8 +342,8 @@ void ZAMCompiler::ComputeFrameLifetimes() {
// Some special-casing.
switch ( inst->op ) {
case OP_NEXT_TABLE_ITER_VV:
case OP_NEXT_TABLE_ITER_VAL_VAR_VVV: {
case OP_NEXT_TABLE_ITER_fb:
case OP_NEXT_TABLE_ITER_VAL_VAR_Vfb: {
// These assign to an arbitrary long list of variables.
auto& iter_vars = inst->aux->loop_vars;
auto depth = inst->loop_depth;
@ -361,21 +367,21 @@ void ZAMCompiler::ComputeFrameLifetimes() {
}
// No need to check the additional "var" associated
// with OP_NEXT_TABLE_ITER_VAL_VAR_VVV as that's
// with OP_NEXT_TABLE_ITER_VAL_VAR_Vfb as that's
// a slot-1 assignment. However, similar to other
// loop variables, mark this as a usage.
if ( inst->op == OP_NEXT_TABLE_ITER_VAL_VAR_VVV )
if ( inst->op == OP_NEXT_TABLE_ITER_VAL_VAR_Vfb )
ExtendLifetime(inst->v1, EndOfLoop(inst, depth));
} break;
case OP_NEXT_TABLE_ITER_NO_VARS_VV: break;
case OP_NEXT_TABLE_ITER_NO_VARS_fb: break;
case OP_NEXT_TABLE_ITER_VAL_VAR_NO_VARS_VVV: {
case OP_NEXT_TABLE_ITER_VAL_VAR_NO_VARS_Vfb: {
auto depth = inst->loop_depth;
ExtendLifetime(inst->v1, EndOfLoop(inst, depth));
} break;
case OP_NEXT_VECTOR_ITER_VAL_VAR_VVVV: {
case OP_NEXT_VECTOR_ITER_VAL_VAR_VVsb: {
CheckSlotAssignment(inst->v2, inst);
auto depth = inst->loop_depth;
@ -383,13 +389,13 @@ void ZAMCompiler::ComputeFrameLifetimes() {
ExtendLifetime(inst->v2, EndOfLoop(inst, depth));
} break;
case OP_NEXT_VECTOR_BLANK_ITER_VAL_VAR_VVV: {
case OP_NEXT_VECTOR_BLANK_ITER_VAL_VAR_Vsb: {
auto depth = inst->loop_depth;
ExtendLifetime(inst->v1, EndOfLoop(inst, depth));
} break;
case OP_NEXT_VECTOR_ITER_VVV:
case OP_NEXT_STRING_ITER_VVV:
case OP_NEXT_VECTOR_ITER_Vsb:
case OP_NEXT_STRING_ITER_Vsb:
// Sometimes loops are written that don't actually
// use the iteration variable. However, we still
// need to mark the variable as having usage
@ -401,12 +407,12 @@ void ZAMCompiler::ComputeFrameLifetimes() {
ExtendLifetime(inst->v1, EndOfLoop(inst, inst->loop_depth));
break;
case OP_NEXT_VECTOR_BLANK_ITER_VV:
case OP_NEXT_STRING_BLANK_ITER_VV: break;
case OP_NEXT_VECTOR_BLANK_ITER_sb:
case OP_NEXT_STRING_BLANK_ITER_sb: break;
case OP_INIT_TABLE_LOOP_VV:
case OP_INIT_VECTOR_LOOP_VV:
case OP_INIT_STRING_LOOP_VV: {
case OP_INIT_TABLE_LOOP_Vf:
case OP_INIT_VECTOR_LOOP_Vs:
case OP_INIT_STRING_LOOP_Vs: {
// For all of these, the scope of the aggregate being
// looped over is the entire loop, even if it doesn't
// directly appear in it, and not just the initializer.
@ -423,14 +429,30 @@ void ZAMCompiler::ComputeFrameLifetimes() {
continue;
}
case OP_STORE_GLOBAL_V: {
case OP_STORE_GLOBAL_g: {
// Use of the global goes to here.
auto slot = frame_layout1[globalsI[inst->v1].id.get()];
ExtendLifetime(slot, EndOfLoop(inst, 1));
break;
}
case OP_LAMBDA_VV: {
case OP_DETERMINE_TYPE_MATCH_VV: {
auto aux = inst->aux;
int n = aux->n;
for ( int i = 0; i < n; ++i ) {
auto slot_i = aux->elems[i].Slot();
if ( slot_i >= 0 ) {
CheckSlotAssignment(slot_i, inst);
// The variable gets used in the switch that
// immediately follows this instruction, hence
// "i + 1" in the following.
ExtendLifetime(slot_i, insts1[i + 1]);
}
}
break;
}
case OP_LAMBDA_Vi: {
auto aux = inst->aux;
int n = aux->n;
for ( int i = 0; i < n; ++i ) {
@ -486,8 +508,7 @@ void ZAMCompiler::ReMapFrame() {
auto vars = inst_beginnings[inst];
for ( auto v : vars ) {
// Don't remap variables whose values aren't actually
// used.
// Don't remap variables whose values aren't actually used.
int slot = frame_layout1[v];
if ( denizen_ending.count(slot) > 0 )
ReMapVar(v, slot, i);
@ -549,9 +570,15 @@ void ZAMCompiler::ReMapFrame() {
// Handle special cases.
switch ( inst->op ) {
case OP_NEXT_TABLE_ITER_VV:
case OP_NEXT_TABLE_ITER_VAL_VAR_VVV: {
// Rewrite iteration variables.
case OP_INIT_TABLE_LOOP_Vf:
case OP_NEXT_TABLE_ITER_fb:
case OP_NEXT_TABLE_ITER_VAL_VAR_Vfb: {
// Rewrite iteration variables. Strictly speaking we only
// need to do this for the INIT, not the NEXT, since the
// latter currently doesn't access the variables directly but
// instead uses pointers set up by the INIT. We do both types
// here, though, to keep things consistent and to help avoid
// surprises if the implementation changes in the future.
auto& iter_vars = inst->aux->loop_vars;
for ( auto& v : iter_vars ) {
if ( v < 0 )
@ -941,16 +968,75 @@ void ZAMCompiler::KillInst(zeek_uint_t i) {
}
}
if ( num_labels == 0 )
// No labels to propagate.
return;
ZInstI* succ = nullptr;
for ( auto j = i + 1; j < insts1.size(); ++j ) {
auto succ = insts1[j];
if ( succ->live ) {
succ->num_labels += num_labels;
break;
if ( num_labels > 0 ) {
for ( auto j = i + 1; j < insts1.size(); ++j ) {
if ( insts1[j]->live ) {
succ = insts1[j];
break;
}
}
if ( succ )
succ->num_labels += num_labels;
}
// Look into propagating control flow info.
if ( inst->aux && ! inst->aux->cft.empty() ) {
auto& cft = inst->aux->cft;
if ( cft.count(CFT_ELSE) > 0 ) {
// Push forward unless this was the end of the block.
if ( cft.count(CFT_BLOCK_END) == 0 ) {
ASSERT(succ);
AddCFT(succ, CFT_ELSE);
}
else
// But if it *was* the end of the block, remove that block.
--cft[CFT_BLOCK_END];
}
if ( cft.count(CFT_BREAK) > 0 ) {
// ### Factor this with the following
// Propagate breaks backwards.
int j = i;
while ( --j >= 0 )
if ( insts1[j]->live )
break;
ASSERT(j >= 0);
// Make sure the CFT entry is created.
AddCFT(insts1[j], CFT_BREAK);
auto be_cnt = cft[CFT_BREAK];
--be_cnt; // we already did one above
insts1[j]->aux->cft[CFT_BREAK] += be_cnt;
}
if ( cft.count(CFT_BLOCK_END) > 0 ) {
// Propagate block-ends backwards.
int j = i;
while ( --j >= 0 )
if ( insts1[j]->live )
break;
ASSERT(j >= 0);
// Make sure the CFT entry is created.
AddCFT(insts1[j], CFT_BLOCK_END);
auto be_cnt = cft[CFT_BLOCK_END];
--be_cnt; // we already did one above
insts1[j]->aux->cft[CFT_BLOCK_END] += be_cnt;
}
// If's can be killed because their bodies become empty,
// break's because they just lead to their following instruction,
// and next's if they become dead code.
// However, loop's and next's should not be killed.
ASSERT(cft.count(CFT_LOOP) == 0);
ASSERT(cft.count(CFT_LOOP_COND) == 0);
}
}

11
src/script_opt/ZAM/Branches.cc
View file

@ -10,9 +10,12 @@ namespace zeek::detail {
void ZAMCompiler::PushGoTos(GoToSets& gotos) { gotos.emplace_back(); }
void ZAMCompiler::ResolveGoTos(GoToSets& gotos, const InstLabel l) {
for ( auto& gi : gotos.back() )
void ZAMCompiler::ResolveGoTos(GoToSets& gotos, const InstLabel l, ControlFlowType cft) {
for ( auto& gi : gotos.back() ) {
SetGoTo(gi, l);
if ( cft != CFT_NONE )
AddCFT(insts1[gi.stmt_num], cft);
}
gotos.pop_back();
}
@ -25,13 +28,13 @@ ZAMStmt ZAMCompiler::GenGoTo(GoToSet& v) {
}
ZAMStmt ZAMCompiler::GoToStub() {
ZInstI z(OP_GOTO_V, 0);
ZInstI z(OP_GOTO_b, 0);
z.op_type = OP_V_I1;
return AddInst(z);
}
ZAMStmt ZAMCompiler::GoTo(const InstLabel l) {
ZInstI inst(OP_GOTO_V, 0);
ZInstI inst(OP_GOTO_b, 0);
inst.target = l;
inst.target_slot = 1;
inst.op_type = OP_V_I1;

87
src/script_opt/ZAM/BuiltIn.cc
View file

@ -27,8 +27,10 @@ SimpleZBI::SimpleZBI(std::string name, ZOp _const_op, ZOp _op, bool _ret_val_mat
bool SimpleZBI::Build(ZAMCompiler* zam, const NameExpr* n, const ExprPList& args) const {
ZInstI z;
if ( nargs == 0 ) {
if ( n )
if ( n ) {
z = ZInstI(op, zam->Frame1Slot(n, OP1_WRITE));
z.is_managed = ZVal::IsManagedType(n->GetType());
}
else
z = ZInstI(op);
}
@ -59,12 +61,9 @@ bool SimpleZBI::Build(ZAMCompiler* zam, const NameExpr* n, const ExprPList& args
z.c = ZVal(args[0]->AsConstExpr()->ValuePtr(), t);
}
z.t = t;
z.SetType(t);
}
if ( n )
z.is_managed = ZVal::IsManagedType(n->GetType());
zam->AddInst(z);
return true;
@ -104,7 +103,7 @@ bool CondZBI::BuildCond(ZAMCompiler* zam, const ExprPList& args, int& branch_v)
auto a0_slot = zam->FrameSlot(a0->AsNameExpr());
z = ZInstI(cond_op, a0_slot, 0);
z.op_type = OP_VV_I2;
z.t = a0->GetType();
z.SetType(a0->GetType());
branch_v = 2;
}
@ -129,7 +128,7 @@ bool OptAssignZBI::Build(ZAMCompiler* zam, const NameExpr* n, const ExprPList& a
ASSERT(nargs == 1);
auto a0 = zam->FrameSlot(args[0]->AsNameExpr());
z = ZInstI(op2, a0);
z.t = args[0]->GetType();
z.SetType(args[0]->GetType());
}
zam->AddInst(z);
@ -145,7 +144,7 @@ bool CatZBI::Build(ZAMCompiler* zam, const NameExpr* n, const ExprPList& args) c
if ( args.empty() ) {
// Weird, but easy enough to support.
z = ZInstI(OP_CAT1_VC, nslot);
z.t = n->GetType();
z.SetType(n->GetType());
z.c = ZVal(val_mgr->EmptyString());
}
@ -168,18 +167,18 @@ bool CatZBI::Build(ZAMCompiler* zam, const NameExpr* n, const ExprPList& args) c
else if ( a0->GetType()->Tag() != TYPE_STRING ) {
if ( a0->Tag() == EXPR_NAME ) {
z = zam->GenInst(OP_CAT1FULL_VV, n, a0->AsNameExpr());
z.t = a0->GetType();
z.SetType(a0->GetType());
}
else {
z = ZInstI(OP_CAT1_VC, nslot);
z.t = n->GetType();
z.SetType(n->GetType());
z.c = ZVal(ZAM_val_cat(a0->AsConstExpr()->ValuePtr()));
}
}
else if ( a0->Tag() == EXPR_CONST ) {
z = zam->GenInst(OP_CAT1_VC, n, a0->AsConstExpr());
z.t = n->GetType();
z.SetType(n->GetType());
}
else
@ -388,12 +387,12 @@ bool MultiZBI::Build(ZAMCompiler* zam, const NameExpr* n, const ExprPList& args)
z.is_managed = ZVal::IsManagedType(n->GetType());
if ( ! consts.empty() ) {
z.t = consts[0]->GetType();
z.c = ZVal(consts[0], z.t);
z.SetType(consts[0]->GetType());
z.c = ZVal(consts[0], z.GetType());
}
if ( type_arg >= 0 && ! z.t )
z.t = args[type_arg]->GetType();
if ( type_arg >= 0 && ! z.GetType() )
z.SetType(args[type_arg]->GetType());
zam->AddInst(z);
@ -434,14 +433,14 @@ SimpleZBI sta_ZBI{"subnet_to_addr", OP_SUBNET_TO_ADDR_VV, 1};
SimpleZBI ttd_ZBI{"time_to_double", OP_TIME_TO_DOUBLE_VV, 1};
SimpleZBI tl_ZBI{"to_lower", OP_TO_LOWER_VV, 1};
CondZBI ce_ZBI{"connection_exists", OP_CONN_EXISTS_VV, OP_CONN_EXISTS_COND_VV, 1};
CondZBI iip_ZBI{"is_icmp_port", OP_IS_ICMP_PORT_VV, OP_IS_ICMP_PORT_COND_VV, 1};
CondZBI itp_ZBI{"is_tcp_port", OP_IS_TCP_PORT_VV, OP_IS_TCP_PORT_COND_VV, 1};
CondZBI iup_ZBI{"is_udp_port", OP_IS_UDP_PORT_VV, OP_IS_UDP_PORT_COND_VV, 1};
CondZBI iv4_ZBI{"is_v4_addr", OP_IS_V4_ADDR_VV, OP_IS_V4_ADDR_COND_VV, 1};
CondZBI iv6_ZBI{"is_v6_addr", OP_IS_V6_ADDR_VV, OP_IS_V6_ADDR_COND_VV, 1};
CondZBI rlt_ZBI{"reading_live_traffic", OP_READING_LIVE_TRAFFIC_V, OP_READING_LIVE_TRAFFIC_COND_V, 0};
CondZBI rt_ZBI{"reading_traces", OP_READING_TRACES_V, OP_READING_TRACES_COND_V, 0};
CondZBI ce_ZBI{"connection_exists", OP_CONN_EXISTS_VV, OP_CONN_EXISTS_COND_Vb, 1};
CondZBI iip_ZBI{"is_icmp_port", OP_IS_ICMP_PORT_VV, OP_IS_ICMP_PORT_COND_Vb, 1};
CondZBI itp_ZBI{"is_tcp_port", OP_IS_TCP_PORT_VV, OP_IS_TCP_PORT_COND_Vb, 1};
CondZBI iup_ZBI{"is_udp_port", OP_IS_UDP_PORT_VV, OP_IS_UDP_PORT_COND_Vb, 1};
CondZBI iv4_ZBI{"is_v4_addr", OP_IS_V4_ADDR_VV, OP_IS_V4_ADDR_COND_Vb, 1};
CondZBI iv6_ZBI{"is_v6_addr", OP_IS_V6_ADDR_VV, OP_IS_V6_ADDR_COND_Vb, 1};
CondZBI rlt_ZBI{"reading_live_traffic", OP_READING_LIVE_TRAFFIC_V, OP_READING_LIVE_TRAFFIC_COND_b, 0};
CondZBI rt_ZBI{"reading_traces", OP_READING_TRACES_V, OP_READING_TRACES_COND_b, 0};
// These have a different form to avoid invoking copy constructors.
auto cat_ZBI = CatZBI();
@ -468,48 +467,48 @@ OptAssignZBI rtc_ZBI{ "PacketAnalyzer::TEREDO::remove_teredo_connection",
MultiZBI faa_ZBI{ "Files::__add_analyzer",
{{{VVV}, {OP_FILES_ADD_ANALYZER_VVV, OP_VVV}},
{{VCV}, {OP_FILES_ADD_ANALYZER_ViV, OP_VVC}}},
{{VCV}, {OP_FILES_ADD_ANALYZER_VCV, OP_VVC}}},
{{{VVV}, {OP_FILES_ADD_ANALYZER_VVVV, OP_VVVV}},
{{VCV}, {OP_FILES_ADD_ANALYZER_VViV, OP_VVVC}}},
{{VCV}, {OP_FILES_ADD_ANALYZER_VVCV, OP_VVVC}}},
1
};
MultiZBI fra_ZBI{ "Files::__remove_analyzer",
{{{VVV}, {OP_FILES_REMOVE_ANALYZER_VVV, OP_VVV}},
{{VCV}, {OP_FILES_REMOVE_ANALYZER_ViV, OP_VVC}}},
{{VCV}, {OP_FILES_REMOVE_ANALYZER_VCV, OP_VVC}}},
{{{VVV}, {OP_FILES_REMOVE_ANALYZER_VVVV, OP_VVVV}},
{{VCV}, {OP_FILES_REMOVE_ANALYZER_VViV, OP_VVVC}}},
{{VCV}, {OP_FILES_REMOVE_ANALYZER_VVCV, OP_VVVC}}},
1
};
MultiZBI fsrb_ZBI{ "Files::__set_reassembly_buffer",
{{{VV}, {OP_FILES_SET_REASSEMBLY_BUFFER_VV, OP_VV}},
{{VC}, {OP_FILES_SET_REASSEMBLY_BUFFER_VC, OP_VV_I2}}},
{{VC}, {OP_FILES_SET_REASSEMBLY_BUFFER_Vi, OP_VV_I2}}},
{{{VV}, {OP_FILES_SET_REASSEMBLY_BUFFER_VVV, OP_VVV}},
{{VC}, {OP_FILES_SET_REASSEMBLY_BUFFER_VVC, OP_VVV_I3}}}
{{VC}, {OP_FILES_SET_REASSEMBLY_BUFFER_VVi, OP_VVV_I3}}}
};
MultiZBI lw_ZBI{ "Log::__write",
{{{VV}, {OP_LOG_WRITE_VV, OP_VV}},
{{CV}, {OP_LOG_WRITEC_V, OP_V}}},
{{CV}, {OP_LOG_WRITE_CV, OP_V}}},
{{{VV}, {OP_LOG_WRITE_VVV, OP_VVV}},
{{CV}, {OP_LOG_WRITEC_VV, OP_VV}}}
{{CV}, {OP_LOG_WRITEC_VCV, OP_VV}}}
};
MultiZBI gccbt_ZBI{ "get_current_conn_bytes_threshold", true,
{{{VV}, {OP_GET_BYTES_THRESH_VVV, OP_VVV}},
{{VC}, {OP_GET_BYTES_THRESH_VVi, OP_VVC}}}
{{VC}, {OP_GET_BYTES_THRESH_VVC, OP_VVC}}}
};
MultiZBI sccbt_ZBI{ "set_current_conn_bytes_threshold",
{{{VVV}, {OP_SET_BYTES_THRESH_VVV, OP_VVV}},
{{VVC}, {OP_SET_BYTES_THRESH_VVi, OP_VVC}},
{{VCV}, {OP_SET_BYTES_THRESH_ViV, OP_VVC}},
{{VCC}, {OP_SET_BYTES_THRESH_Vii, OP_VVC_I2}}},
{{VVC}, {OP_SET_BYTES_THRESH_VVC, OP_VVC}},
{{VCV}, {OP_SET_BYTES_THRESH_VCV, OP_VVC}},
{{VCC}, {OP_SET_BYTES_THRESH_VCi, OP_VVC_I2}}},
{{{VVV}, {OP_SET_BYTES_THRESH_VVVV, OP_VVVV}},
{{VVC}, {OP_SET_BYTES_THRESH_VVVi, OP_VVVC}},
{{VCV}, {OP_SET_BYTES_THRESH_VViV, OP_VVVC}},
{{VCC}, {OP_SET_BYTES_THRESH_VVii, OP_VVVC_I3}}}
{{VVC}, {OP_SET_BYTES_THRESH_VVVC, OP_VVVC}},
{{VCV}, {OP_SET_BYTES_THRESH_VVCV, OP_VVVC}},
{{VCC}, {OP_SET_BYTES_THRESH_VVCi, OP_VVVC_I3}}}
};
MultiZBI sw_ZBI{ "starts_with", true,
@ -532,12 +531,12 @@ MultiZBI strstr_ZBI{ "strstr", true,
MultiZBI sb_ZBI{ "sub_bytes", true,
{{{VVV}, {OP_SUB_BYTES_VVVV, OP_VVVV}},
{{VVC}, {OP_SUB_BYTES_VVVi, OP_VVVC}},
{{VCV}, {OP_SUB_BYTES_VViV, OP_VVVC}},
{{VCC}, {OP_SUB_BYTES_VVii, OP_VVVC_I3}},
{{CVV}, {OP_SUB_BYTES_VVVC, OP_VVVC}},
{{CVC}, {OP_SUB_BYTES_VViC, OP_VVVC_I3}},
{{CCV}, {OP_SUB_BYTES_ViVC, OP_VVVC_I3}}}
{{VVC}, {OP_SUB_BYTES_VVVC, OP_VVVC}},
{{VCV}, {OP_SUB_BYTES_VVCV, OP_VVVC}},
{{VCC}, {OP_SUB_BYTES_VVCi, OP_VVVC_I3}},
{{CVV}, {OP_SUB_BYTES_VCVV, OP_VVVC}},
{{CVC}, {OP_SUB_BYTES_VCVi, OP_VVVC_I3}},
{{CCV}, {OP_SUB_BYTES2_VCVi, OP_VVVC_I3}}}
};
// clang-format on

7
src/script_opt/ZAM/BuiltInSupport.cc
View file

@ -48,8 +48,7 @@ FixedCatArg::FixedCatArg(TypePtr _t) : t(std::move(_t)) {
}
}
void FixedCatArg::RenderInto(ZVal* zframe, int slot, char*& res) {
auto& z = zframe[slot];
void FixedCatArg::RenderInto(const ZVal& z, char*& res) {
int n;
const char* text;
std::string str;
@ -140,8 +139,8 @@ void FixedCatArg::RenderInto(ZVal* zframe, int slot, char*& res) {
}
}
size_t PatternCatArg::ComputeMaxSize(ZVal* zframe, int slot) {
text = zframe[slot].AsPattern()->AsPattern()->PatternText();
size_t PatternCatArg::ComputeMaxSize(const ZVal& zv) {
text = zv.AsPattern()->AsPattern()->PatternText();
n = strlen(text);
return n;
}

24
src/script_opt/ZAM/BuiltInSupport.h
View file

@ -13,9 +13,9 @@ public:
virtual ~CatArg() {}
size_t MaxSize(ZVal* zframe, int slot) { return max_size ? *max_size : ComputeMaxSize(zframe, slot); }
size_t MaxSize(const ZVal& zv) { return max_size ? *max_size : ComputeMaxSize(zv); }
virtual void RenderInto(ZVal* zframe, int slot, char*& res) {
virtual void RenderInto(const ZVal& zv, char*& res) {
auto n = *max_size;
memcpy(res, s->data(), n);
res += n;
@ -25,7 +25,7 @@ protected:
CatArg() {}
CatArg(size_t _max_size) : max_size(_max_size) {}
virtual size_t ComputeMaxSize(ZVal* zframe, int slot) { return 0; }
virtual size_t ComputeMaxSize(const ZVal& zv) { return 0; }
// Present if max size is known a priori.
std::optional<size_t> max_size;
@ -38,7 +38,7 @@ class FixedCatArg : public CatArg {
public:
FixedCatArg(TypePtr t);
void RenderInto(ZVal* zframe, int slot, char*& res) override;
void RenderInto(const ZVal& zv, char*& res) override;
protected:
TypePtr t;
@ -49,22 +49,22 @@ class StringCatArg : public CatArg {
public:
StringCatArg() : CatArg() {}
void RenderInto(ZVal* zframe, int slot, char*& res) override {
auto s = zframe[slot].AsString();
void RenderInto(const ZVal& zv, char*& res) override {
auto s = zv.AsString();
auto n = s->Len();
memcpy(res, s->Bytes(), n);
res += n;
}
protected:
size_t ComputeMaxSize(ZVal* zframe, int slot) override { return zframe[slot].AsString()->Len(); }
size_t ComputeMaxSize(const ZVal& zv) override { return zv.AsString()->Len(); }
};
class PatternCatArg : public CatArg {
public:
PatternCatArg() : CatArg() {}
void RenderInto(ZVal* zframe, int slot, char*& res) override {
void RenderInto(const ZVal& zv, char*& res) override {
*(res++) = '/';
strcpy(res, text);
res += n;
@ -72,7 +72,7 @@ public:
}
protected:
size_t ComputeMaxSize(ZVal* zframe, int slot) override;
size_t ComputeMaxSize(const ZVal& zv) override;
const char* text = nullptr;
size_t n = 0;
@ -82,7 +82,7 @@ class DescCatArg : public CatArg {
public:
DescCatArg(TypePtr _t) : CatArg(), t(std::move(_t)) { d.SetStyle(RAW_STYLE); }
void RenderInto(ZVal* zframe, int slot, char*& res) override {
void RenderInto(const ZVal& zv, char*& res) override {
auto n = d.Len();
memcpy(res, d.Bytes(), n);
res += n;
@ -90,8 +90,8 @@ public:
}
protected:
size_t ComputeMaxSize(ZVal* zframe, int slot) override {
zframe[slot].ToVal(t)->Describe(&d);
size_t ComputeMaxSize(const ZVal& zv) override {
zv.ToVal(t)->Describe(&d);
return d.Len();
}

16
src/script_opt/ZAM/Compile.h
View file

@ -159,16 +159,17 @@ private:
const ZAMStmt ValueSwitch(const SwitchStmt* sw, const NameExpr* v, const ConstExpr* c);
const ZAMStmt TypeSwitch(const SwitchStmt* sw, const NameExpr* v, const ConstExpr* c);
const ZAMStmt GenSwitch(const SwitchStmt* sw, int slot, InternalTypeTag it);
void PushNexts() { PushGoTos(nexts); }
void PushBreaks() { PushGoTos(breaks); }
void PushFallThroughs() { PushGoTos(fallthroughs); }
void PushCatchReturns() { PushGoTos(catches); }
void ResolveNexts(const InstLabel l) { ResolveGoTos(nexts, l); }
void ResolveBreaks(const InstLabel l) { ResolveGoTos(breaks, l); }
void ResolveNexts(const InstLabel l) { ResolveGoTos(nexts, l, CFT_NEXT); }
void ResolveBreaks(const InstLabel l) { ResolveGoTos(breaks, l, CFT_BREAK); }
void ResolveFallThroughs(const InstLabel l) { ResolveGoTos(fallthroughs, l); }
void ResolveCatchReturns(const InstLabel l) { ResolveGoTos(catches, l); }
void ResolveCatchReturns(const InstLabel l) { ResolveGoTos(catches, l, CFT_INLINED_RETURN); }
const ZAMStmt LoopOverTable(const ForStmt* f, const NameExpr* val);
const ZAMStmt LoopOverVector(const ForStmt* f, const NameExpr* val);
@ -229,8 +230,8 @@ private:
const ZAMStmt CompileIndex(const NameExpr* n1, int n2_slot, const TypePtr& n2_type, const ListExpr* l,
bool in_when);
const ZAMStmt BuildLambda(const NameExpr* n, LambdaExpr* le);
const ZAMStmt BuildLambda(int n_slot, LambdaExpr* le);
const ZAMStmt BuildLambda(const NameExpr* n, ExprPtr le);
const ZAMStmt BuildLambda(int n_slot, ExprPtr le);
// Second argument is which instruction slot holds the branch target.
const ZAMStmt GenCond(const Expr* e, int& branch_v);
@ -277,7 +278,7 @@ private:
using GoToSets = std::vector<GoToSet>;
void PushGoTos(GoToSets& gotos);
void ResolveGoTos(GoToSets& gotos, const InstLabel l);
void ResolveGoTos(GoToSets& gotos, const InstLabel l, ControlFlowType cft = CFT_NONE);
ZAMStmt GenGoTo(GoToSet& v);
ZAMStmt GoToStub();
@ -322,6 +323,9 @@ private:
const ZAMStmt ErrorStmt();
const ZAMStmt LastInst();
// Adds control flow information to an instruction.
void AddCFT(ZInstI* inst, ControlFlowType cft);
// Returns a handle to state associated with building
// up a list of values.
std::unique_ptr<OpaqueVals> BuildVals(const ListExprPtr&);

3
src/script_opt/ZAM/Driver.cc
View file

@ -203,6 +203,7 @@ StmtPtr ZAMCompiler::CompileBody() {
auto zb = make_intrusive<ZBody>(fname, this);
zb->SetInsts(insts2);
zb->SetLocationInfo(body->GetLocationInfo());
// Could erase insts1 here to recover memory, but it's handy
// for debugging.
@ -218,7 +219,9 @@ void ZAMCompiler::ResolveHookBreaks() {
// Rewrite the breaks.
for ( auto& b : breaks[0] ) {
auto& i = insts1[b.stmt_num];
auto aux = i->aux;
*i = ZInstI(OP_HOOK_BREAK_X);
i->aux = aux;
}
}

103
src/script_opt/ZAM/Expr.cc
View file

@ -246,7 +246,7 @@ const ZAMStmt ZAMCompiler::CompileAssignExpr(const AssignExpr* e) {
}
if ( rhs->Tag() == EXPR_LAMBDA )
return BuildLambda(lhs, rhs->AsLambdaExpr());
return BuildLambda(lhs, op2);
if ( rhs->Tag() == EXPR_COND && r1->GetType()->Tag() == TYPE_VECTOR )
return Bool_Vec_CondVVVV(lhs, r1->AsNameExpr(), r2->AsNameExpr(), r3->AsNameExpr());
@ -466,20 +466,23 @@ const ZAMStmt ZAMCompiler::CompileFieldLHSAssignExpr(const FieldLHSAssignExpr* e
auto field = e->Field();
if ( rhs->Tag() == EXPR_NAME )
return Field_LHS_AssignFV(e, rhs->AsNameExpr());
return Field_LHS_AssignFVi(e, rhs->AsNameExpr(), field);
if ( rhs->Tag() == EXPR_CONST )
return Field_LHS_AssignFC(e, rhs->AsConstExpr());
return Field_LHS_AssignFCi(e, rhs->AsConstExpr(), field);
auto r1 = rhs->GetOp1();
auto r2 = rhs->GetOp2();
if ( rhs->Tag() == EXPR_FIELD ) {
auto rhs_f = rhs->AsFieldExpr();
if ( r1->Tag() == EXPR_NAME )
return Field_LHS_AssignFVi(e, r1->AsNameExpr(), rhs_f->Field());
return Field_LHS_AssignFCi(e, r1->AsConstExpr(), rhs_f->Field());
// Note, the LHS field comes after the RHS field rather than before,
// to maintain layout symmetry close to that for non-field RHS's.
if ( r1->Tag() == EXPR_NAME )
return Field_LHS_AssignFVii(e, r1->AsNameExpr(), rhs_f->Field(), field);
return Field_LHS_AssignFCii(e, r1->AsConstExpr(), rhs_f->Field(), field);
}
if ( r1 && r1->IsConst() )
@ -564,7 +567,7 @@ const ZAMStmt ZAMCompiler::CompileEvent(EventHandler* h, const ListExpr* l) {
else {
auto n0 = exprs[0]->AsNameExpr();
z.v1 = FrameSlot(n0);
z.t = n0->GetType();
z.SetType(n0->GetType());
if ( n == 1 ) {
z.op = OP_EVENT1_V;
@ -574,7 +577,7 @@ const ZAMStmt ZAMCompiler::CompileEvent(EventHandler* h, const ListExpr* l) {
else {
auto n1 = exprs[1]->AsNameExpr();
z.v2 = FrameSlot(n1);
z.t2 = n1->GetType();
z.SetType2(n1->GetType());
if ( n == 2 ) {
z.op = OP_EVENT2_VV;
@ -630,7 +633,7 @@ const ZAMStmt ZAMCompiler::CompileInExpr(const NameExpr* n1, const NameExpr* n2,
if ( op3_t->AsTableType()->IsPatternIndex() && op2_t->Tag() == TYPE_STRING )
a = n2 ? OP_STR_IN_PAT_TBL_VVV : OP_STR_IN_PAT_TBL_VCV;
else
a = n2 ? OP_VAL_IS_IN_TABLE_VVV : OP_CONST_IS_IN_TABLE_VCV;
a = n2 ? OP_VAL_IS_IN_TABLE_VVV : OP_CONST_IS_IN_TABLE_VVC;
}
else if ( op2->GetType()->Tag() == TYPE_PATTERN )
a = n2 ? (n3 ? OP_P_IN_S_VVV : OP_P_IN_S_VVC) : OP_P_IN_S_VCV;
@ -692,11 +695,11 @@ const ZAMStmt ZAMCompiler::CompileInExpr(const NameExpr* n1, const ListExpr* l,
else {
auto l_e0_c = l_e[0]->AsConstExpr();
ZOp op = is_vec ? OP_CONST_IS_IN_VECTOR_VCV : OP_CONST_IS_IN_TABLE_VCV;
ZOp op = is_vec ? OP_CONST_IS_IN_VECTOR_VCV : OP_CONST_IS_IN_TABLE_VVC;
z = GenInst(op, n1, l_e0_c, n2);
}
z.t = l_e[0]->GetType();
z.SetType(l_e[0]->GetType());
return AddInst(z);
}
@ -717,21 +720,21 @@ const ZAMStmt ZAMCompiler::CompileInExpr(const NameExpr* n1, const ListExpr* l,
if ( l_e0_n && l_e1_n ) {
z = GenInst(OP_VAL2_IS_IN_TABLE_VVVV, n1, l_e0_n, l_e1_n, n2);
z.t2 = l_e0_n->GetType();
z.SetType2(l_e0_n->GetType());
}
else if ( l_e0_n ) {
ASSERT(l_e1_c);
z = GenInst(OP_VAL2_IS_IN_TABLE_VVVC, n1, l_e0_n, n2, l_e1_c);
z.t2 = l_e0_n->GetType();
z.SetType2(l_e0_n->GetType());
}
else if ( l_e1_n ) {
ASSERT(l_e0_c);
z = GenInst(OP_VAL2_IS_IN_TABLE_VVCV, n1, l_e1_n, n2, l_e0_c);
z.t2 = l_e1_n->GetType();
z.SetType2(l_e1_n->GetType());
}
else {
@ -743,7 +746,7 @@ const ZAMStmt ZAMCompiler::CompileInExpr(const NameExpr* n1, const ListExpr* l,
auto slot = TempForConst(l_e0_c);
z = ZInstI(OP_VAL2_IS_IN_TABLE_VVVC, FrameSlot(n1), slot, FrameSlot(n2), l_e1_c);
z.op_type = OP_VVVC;
z.t2 = l_e0_c->GetType();
z.SetType2(l_e0_c->GetType());
}
return AddInst(z);
@ -817,7 +820,7 @@ const ZAMStmt ZAMCompiler::CompileIndex(const NameExpr* n1, int n2_slot, const T
z = ZInstI(zop, Frame1Slot(n1, zop), n2_slot, n3_slot);
}
else {
auto zop = OP_INDEX_STRINGC_VVV;
auto zop = OP_INDEX_STRINGC_VVi;
z = ZInstI(zop, Frame1Slot(n1, zop), n2_slot, c);
z.op_type = OP_VVV_I3;
}
@ -846,11 +849,11 @@ const ZAMStmt ZAMCompiler::CompileIndex(const NameExpr* n1, int n2_slot, const T
ZOp zop;
if ( in_when )
zop = OP_WHEN_INDEX_VECC_VVV;
zop = OP_WHEN_INDEX_VECC_VVi;
else if ( is_any )
zop = OP_INDEX_ANY_VECC_VVV;
zop = OP_INDEX_ANY_VECC_VVi;
else
zop = OP_INDEX_VECC_VVV;
zop = OP_INDEX_VECC_VVi;
z = ZInstI(zop, Frame1Slot(n1, zop), n2_slot, c);
z.op_type = OP_VVV_I3;
@ -932,18 +935,18 @@ const ZAMStmt ZAMCompiler::CompileIndex(const NameExpr* n1, int n2_slot, const T
return AddInst(z);
}
const ZAMStmt ZAMCompiler::BuildLambda(const NameExpr* n, LambdaExpr* le) {
return BuildLambda(Frame1Slot(n, OP1_WRITE), le);
const ZAMStmt ZAMCompiler::BuildLambda(const NameExpr* n, ExprPtr e) {
return BuildLambda(Frame1Slot(n, OP1_WRITE), std::move(e));
}
const ZAMStmt ZAMCompiler::BuildLambda(int n_slot, LambdaExpr* le) {
auto& captures = le->GetCaptures();
const ZAMStmt ZAMCompiler::BuildLambda(int n_slot, ExprPtr e) {
auto lambda = cast_intrusive<LambdaExpr>(e);
auto& captures = lambda->GetCaptures();
int ncaptures = captures ? captures->size() : 0;
auto aux = new ZInstAux(ncaptures);
aux->primary_func = le->PrimaryFunc();
aux->lambda_name = le->Name();
aux->id_val = le->Ingredients()->GetID();
aux->lambda = cast_intrusive<LambdaExpr>(std::move(e));
aux->id_val = lambda->Ingredients()->GetID();
for ( int i = 0; i < ncaptures; ++i ) {
auto& id_i = (*captures)[i].Id();
@ -954,7 +957,7 @@ const ZAMStmt ZAMCompiler::BuildLambda(int n_slot, LambdaExpr* le) {
aux->Add(i, FrameSlot(id_i), id_i->GetType());
}
auto z = ZInstI(OP_LAMBDA_VV, n_slot, le->PrimaryFunc()->FrameSize());
auto z = ZInstI(OP_LAMBDA_Vi, n_slot, lambda->PrimaryFunc()->FrameSize());
z.op_type = OP_VV_I2;
z.aux = aux;
@ -1031,7 +1034,7 @@ const ZAMStmt ZAMCompiler::AssignVecElems(const Expr* e) {
inst = Vector_Elem_AssignVVC(lhs, n2, c3);
}
TopMainInst()->t = t3;
TopMainInst()->SetType(t3);
return inst;
}
@ -1048,7 +1051,7 @@ const ZAMStmt ZAMCompiler::AssignVecElems(const Expr* e) {
else
inst = Vector_Elem_AssignVVi(lhs, n3, index);
TopMainInst()->t = t3;
TopMainInst()->SetType(t3);
return inst;
}
@ -1068,7 +1071,7 @@ const ZAMStmt ZAMCompiler::AssignTableElem(const Expr* e) {
z = GenInst(OP_TABLE_ELEM_ASSIGN_VC, op1, op3->AsConstExpr());
z.aux = InternalBuildVals(op2);
z.t = op3->GetType();
z.SetType(op3->GetType());
if ( pfs->HasSideEffects(SideEffectsOp::WRITE, op1->GetType()) )
z.aux->can_change_non_locals = true;
@ -1159,7 +1162,7 @@ const ZAMStmt ZAMCompiler::DoCall(const CallExpr* c, const NameExpr* n) {
}
}
z.t = arg0->GetType();
z.SetType(arg0->GetType());
}
else {
@ -1184,10 +1187,12 @@ const ZAMStmt ZAMCompiler::DoCall(const CallExpr* c, const NameExpr* n) {
default:
if ( in_when ) {
if ( indirect )
op = OP_WHENINDCALLN_VV;
else
if ( ! indirect )
op = OP_WHENCALLN_V;
else if ( func_id->IsGlobal() )
op = OP_WHEN_ID_INDCALLN_V;
else
op = OP_WHENINDCALLN_VV;
}
else if ( indirect ) {
@ -1279,9 +1284,9 @@ const ZAMStmt ZAMCompiler::ConstructTable(const NameExpr* n, const Expr* e) {
auto tt = cast_intrusive<TableType>(n->GetType());
auto width = tt->GetIndices()->GetTypes().size();
auto z = GenInst(OP_CONSTRUCT_TABLE_VV, n, width);
auto z = GenInst(OP_CONSTRUCT_TABLE_Vi, n, width);
z.aux = InternalBuildVals(con, width + 1);
z.t = tt;
z.SetType(tt);
ASSERT(e->Tag() == EXPR_TABLE_CONSTRUCTOR);
z.aux->attrs = static_cast<const TableConstructorExpr*>(e)->GetAttrs();
@ -1291,7 +1296,7 @@ const ZAMStmt ZAMCompiler::ConstructTable(const NameExpr* n, const Expr* e) {
if ( ! def_attr || def_attr->GetExpr()->Tag() != EXPR_LAMBDA )
return zstmt;
auto def_lambda = def_attr->GetExpr()->AsLambdaExpr();
auto def_lambda = cast_intrusive<LambdaExpr>(def_attr->GetExpr());
auto dl_t = def_lambda->GetType()->AsFuncType();
auto& captures = dl_t->GetCaptures();
@ -1308,7 +1313,7 @@ const ZAMStmt ZAMCompiler::ConstructTable(const NameExpr* n, const Expr* e) {
z = GenInst(OP_SET_TABLE_DEFAULT_LAMBDA_VV, n, slot);
z.op_type = OP_VV;
z.t = def_lambda->GetType();
z.SetType(def_lambda->GetType());
return AddInst(z);
}
@ -1318,9 +1323,9 @@ const ZAMStmt ZAMCompiler::ConstructSet(const NameExpr* n, const Expr* e) {
auto tt = n->GetType()->AsTableType();
auto width = tt->GetIndices()->GetTypes().size();
auto z = GenInst(OP_CONSTRUCT_SET_VV, n, width);
auto z = GenInst(OP_CONSTRUCT_SET_Vi, n, width);
z.aux = InternalBuildVals(con, width);
z.t = e->GetType();
z.SetType(e->GetType());
ASSERT(e->Tag() == EXPR_SET_CONSTRUCTOR);
z.aux->attrs = static_cast<const SetConstructorExpr*>(e)->GetAttrs();
@ -1391,13 +1396,13 @@ const ZAMStmt ZAMCompiler::ConstructRecord(const NameExpr* n, const Expr* e, boo
if ( fi->empty() ) {
if ( network_time_index >= 0 )
op = OP_CONSTRUCT_KNOWN_RECORD_WITH_NT_VV;
op = OP_CONSTRUCT_KNOWN_RECORD_WITH_NT_Vi;
else
op = OP_CONSTRUCT_KNOWN_RECORD_V;
}
else {
if ( network_time_index >= 0 )
op = OP_CONSTRUCT_KNOWN_RECORD_WITH_INITS_AND_NT_VV;
op = OP_CONSTRUCT_KNOWN_RECORD_WITH_INITS_AND_NT_Vi;
else
op = OP_CONSTRUCT_KNOWN_RECORD_WITH_INITS_V;
aux->field_inits = std::move(fi);
@ -1411,12 +1416,12 @@ const ZAMStmt ZAMCompiler::ConstructRecord(const NameExpr* n, const Expr* e, boo
if ( is_from_rec ) {
// Map non-from-rec operand to the from-rec equivalent.
switch ( op ) {
case OP_CONSTRUCT_KNOWN_RECORD_WITH_NT_VV: op = OP_CONSTRUCT_KNOWN_RECORD_WITH_NT_FROM_VVV; break;
case OP_CONSTRUCT_KNOWN_RECORD_WITH_NT_Vi: op = OP_CONSTRUCT_KNOWN_RECORD_WITH_NT_FROM_VVi; break;
case OP_CONSTRUCT_KNOWN_RECORD_V: op = OP_CONSTRUCT_KNOWN_RECORD_FROM_VV; break;
case OP_CONSTRUCT_KNOWN_RECORD_WITH_INITS_AND_NT_VV:
op = OP_CONSTRUCT_KNOWN_RECORD_WITH_INITS_AND_NT_FROM_VVV;
case OP_CONSTRUCT_KNOWN_RECORD_WITH_INITS_AND_NT_Vi:
op = OP_CONSTRUCT_KNOWN_RECORD_WITH_INITS_AND_NT_FROM_VVi;
break;
case OP_CONSTRUCT_KNOWN_RECORD_WITH_INITS_V:
@ -1448,7 +1453,7 @@ const ZAMStmt ZAMCompiler::ConstructRecord(const NameExpr* n, const Expr* e, boo
z = network_time_index >= 0 ? GenInst(op, n, network_time_index) : GenInst(op, n);
z.aux = aux;
z.t = rec_e->GetType();
z.SetType(rec_e->GetType());
auto inst = AddInst(z);
@ -1488,7 +1493,7 @@ const ZAMStmt ZAMCompiler::ConstructRecord(const NameExpr* n, const Expr* e, boo
// Need to add a separate instruction for concretizing the fields.
z = GenInst(OP_CONCRETIZE_VECTOR_FIELDS_V, n);
z.t = rec_e->GetType();
z.SetType(rec_e->GetType());
int nf = static_cast<int>(vector_fields.size());
z.aux = new ZInstAux(nf);
z.aux->elems_has_slots = false; // we're storing field offsets, not slots
@ -1503,7 +1508,7 @@ const ZAMStmt ZAMCompiler::ConstructVector(const NameExpr* n, const Expr* e) {
auto z = GenInst(OP_CONSTRUCT_VECTOR_V, n);
z.aux = InternalBuildVals(con);
z.t = e->GetType();
z.SetType(e->GetType());
return AddInst(z);
}
@ -1626,8 +1631,8 @@ const ZAMStmt ZAMCompiler::Is(const NameExpr* n, const Expr* e) {
int op_slot = FrameSlot(op);
ZInstI z(OP_IS_VV, Frame1Slot(n, OP_IS_VV), op_slot);
z.t2 = op->GetType();
z.SetType(is->TestType());
z.SetType2(op->GetType());
return AddInst(z);
}

74
src/script_opt/ZAM/IterInfo.h
View file

@ -15,21 +15,54 @@ namespace zeek::detail {
class TableIterInfo {
public:
// No constructor needed, as all of our member variables are
// instead instantiated via BeginLoop(). This allows us to
// reuse TableIterInfo objects to lower the overhead associated
// with executing ZBody::Exec for non-recursive functions.
// Empty constructor for a simple version that initializes all the
// member variables via BeginLoop(). Helpful for supporting recursive
// functions that include table iterations.
TableIterInfo() {}
// Version that populates the fixed fields up front, with the
// dynamic ones being done with SetLoopVars().
TableIterInfo(const std::vector<TypePtr>* _loop_var_types, const std::vector<bool>* _lvt_is_managed,
TypePtr _value_var_type) {
SetIterInfo(_loop_var_types, _lvt_is_managed, std::move(_value_var_type));
}
// Sets the fixed fields.
void SetIterInfo(const std::vector<TypePtr>* _loop_var_types, const std::vector<bool>* _lvt_is_managed,
TypePtr _value_var_type) {
loop_var_types = _loop_var_types;
lvt_is_managed = _lvt_is_managed;
value_var_type = std::move(_value_var_type);
}
// We do, however, want to make sure that when we go out of scope,
// if we have any pending iterators we clear them.
~TableIterInfo() { Clear(); }
// Start looping over the elements of the given table. "_aux"
// Start looping over the elements of the given table. "aux"
// provides information about the index variables, their types,
// and the type of the value variable (if any).
void BeginLoop(TableValPtr _tv, ZInstAux* _aux) {
tv = _tv;
aux = _aux;
void BeginLoop(TableValPtr _tv, ZVal* frame, ZInstAux* aux) {
tv = std::move(_tv);
for ( auto lv : aux->loop_vars )
if ( lv < 0 )
loop_vars.push_back(nullptr);
else
loop_vars.push_back(&frame[lv]);
SetIterInfo(&aux->types, &aux->is_managed, aux->value_var_type);
PrimeIter();
}
void BeginLoop(TableValPtr _tv, std::vector<ZVal*> _loop_vars) {
tv = std::move(_tv);
loop_vars = std::move(_loop_vars);
PrimeIter();
}
void PrimeIter() {
auto tvd = tv->AsTable();
tbl_iter = tvd->begin();
tbl_end = tvd->end();
@ -43,18 +76,17 @@ public:
// Performs the next iteration (assuming IsDoneIterating() returned
// false), assigning to the index variables.
void NextIter(ZVal* frame) {
void NextIter() {
auto ind_lv = tv->RecreateIndex(*(*tbl_iter)->GetHashKey());
for ( int i = 0; i < ind_lv->Length(); ++i ) {
ValPtr ind_lv_p = ind_lv->Idx(i);
auto lv = aux->loop_vars[i];
if ( lv < 0 )
auto lv = loop_vars[i];
if ( ! lv )
continue;
auto& var = frame[lv];
if ( aux->is_managed[i] )
ZVal::DeleteManagedType(var);
auto& t = aux->types[i];
var = ZVal(ind_lv_p, t);
ValPtr ind_lv_p = ind_lv->Idx(i);
if ( (*lvt_is_managed)[i] )
ZVal::DeleteManagedType(*lv);
*lv = ZVal(ind_lv_p, (*loop_var_types)[i]);
}
IterFinished();
@ -63,7 +95,7 @@ public:
// For the current iteration, returns the corresponding value.
ZVal IterValue() {
auto tev = (*tbl_iter)->value;
return ZVal(tev->GetVal(), aux->value_var_type);
return ZVal(tev->GetVal(), value_var_type);
}
// Called upon finishing the iteration.
@ -78,8 +110,10 @@ public:
private:
TableValPtr tv = nullptr;
// Associated auxiliary information.
ZInstAux* aux = nullptr;
std::vector<ZVal*> loop_vars;
const std::vector<TypePtr>* loop_var_types;
const std::vector<bool>* lvt_is_managed;
TypePtr value_var_type;
std::optional<DictIterator<TableEntryVal>> tbl_iter;
std::optional<DictIterator<TableEntryVal>> tbl_end;

28
src/script_opt/ZAM/Low-Level.cc
View file

@ -20,9 +20,25 @@ bool ZAMCompiler::NullStmtOK() const {
const ZAMStmt ZAMCompiler::EmptyStmt() { return ZAMStmt(insts1.size() - 1); }
const ZAMStmt ZAMCompiler::ErrorStmt() { return ZAMStmt(0); }
const ZAMStmt ZAMCompiler::LastInst() { return ZAMStmt(insts1.size() - 1); }
const ZAMStmt ZAMCompiler::ErrorStmt() { return ZAMStmt(0); }
void ZAMCompiler::AddCFT(ZInstI* inst, ControlFlowType cft) {
if ( cft == CFT_NONE )
return;
if ( ! inst->aux )
inst->aux = new ZInstAux(0);
auto cft_entry = inst->aux->cft.find(cft);
if ( cft_entry == inst->aux->cft.end() )
inst->aux->cft[cft] = 1;
else {
ASSERT(cft == CFT_BLOCK_END || cft == CFT_BREAK);
++cft_entry->second;
}
}
std::unique_ptr<OpaqueVals> ZAMCompiler::BuildVals(const ListExprPtr& l) {
return std::make_unique<OpaqueVals>(InternalBuildVals(l.get()));
@ -127,9 +143,9 @@ const ZAMStmt ZAMCompiler::AddInst(const ZInstI& inst, bool suppress_non_local)
auto gs = pending_global_store;
pending_global_store = -1;
auto store_inst = ZInstI(OP_STORE_GLOBAL_V, gs);
auto store_inst = ZInstI(OP_STORE_GLOBAL_g, gs);
store_inst.op_type = OP_V_I1;
store_inst.t = globalsI[gs].id->GetType();
store_inst.SetType(globalsI[gs].id->GetType());
return AddInst(store_inst);
}
@ -138,15 +154,15 @@ const ZAMStmt ZAMCompiler::AddInst(const ZInstI& inst, bool suppress_non_local)
auto cs = pending_capture_store;
pending_capture_store = -1;
auto& cv = *func->GetType()->AsFuncType()->GetCaptures();
auto& cv = *func->GetType()->GetCaptures();
auto& c_id = cv[cs].Id();
ZOp op;
if ( ZVal::IsManagedType(c_id->GetType()) )
op = OP_STORE_MANAGED_CAPTURE_VV;
op = OP_STORE_MANAGED_CAPTURE_Vi;
else
op = OP_STORE_CAPTURE_VV;
op = OP_STORE_CAPTURE_Vi;
auto store_inst = ZInstI(op, RawSlot(c_id.get()), cs);
store_inst.op_type = OP_VV_I2;

286
src/script_opt/ZAM/OPs/README.txt Normal file
View file

@ -0,0 +1,286 @@
# See the file "COPYING" in the main distribution directory for copyright.
# This directory contains templates used to generate virtual functions, opcodes,
# and evaluation code for compiled code. Each template describes a ZAM
# "operation", which generally corresponds to a set of concrete ZAM
# "instructions". (See ZInst.h for the layout of ZAM instructions.) Often
# a single ZAM operation gives rise to a family of instructions that differ
# in either the nature of the instruction's operands (typically, whether
# they are variables residing on the ZAM execution frame, or constants)
# and/or the Zeek type of the operands (e.g., "count" or "double" or "addr").
#
# The Gen-ZAM utility processes this file to generate numerous C++ inclusion
# files that are then compiled into Zeek. These files span the range of (1)
# hooks that enable run-time generation of ZAM code to execute ASTs (which
# have first been transformed to "reduced" form), (2) specifications of the
# properties of the different instructions, (3) code to evaluate (execute)
# each instruction, and (4) macros (C++ #define's) to aid in writing that
# code. See Gen-ZAM.h for a list of the different inclusion files.
#
# Operation templates are declarative, other than the imperative C++ snippets
# they include for instruction evaluation/execution. You specify a template
# using lines of text for which, for the most part, the first word on the
# line designates an "attribute" associated with the template, and the
# remainder of the line provides specifiers/arguments for that attribute.
# A blank line (or end of file) ends the template. By convention, for
# templates that include C++ evaluation snippets, those are specified as the
# last attribute. Comments begin with '#' at the start of the line (no
# leading whitespace allowed), and can be intermingled with a template's
# attributes.
#
# Each ZAM instruction includes up to 4 integer values and one constant
# (specified as a ZVal). Often, the integer values are interpreted as offsets
# ("slots") into the ZAM execution "frame", though sometimes they have other
# meanings, such as the offset of a particular field in a record, or an index
# into the ZAM code for a branch instruction. Most instructions compute
# some sort of result (expressed as a ZVal) that is stored into the frame
# slot specified by the instruction's first integer value. We refer to this
# target as the "assignment slot", and to the other 3 integer values as
# "operands". Thus, for example, an instruction with two operands used the
# first 3 integer values, the first as the assignment slot and the other two
# for computing the result to put in that slot.
#
# Instruction templates have one or more "type"s associated with them (as
# discussed below) specifying the types of operands (variables corresponding
# to slots, or constants) associated with the instruction. In the evaluation
# code for an instruction, these are referred to with $-parameters, such as
# $1 for the first operand. The special parameter $$ refers to the *assignment
# target* of the instruction, if applicable. These parameters always come
# first when specifying an instruction's type. For example, a type of "VVC"
# specifies an instruction with two variables and one constant associated
# with it. If the instruction assigns a value, then in the evaluation these
# will be specified as $$, $1 and $2, respectively. If it does not (usually
# reflected by the template having the "op1-read" attribute) then they
# are specified as $1, $2 and $3, respectively. See "eval" below.
#
# The first attribute of each template states the type of operation specified
# in the template, along with the name of the operation. The possible types
# are:
#
# op an operation that generally corresponds to a single ZAM
# instruction, and is fully specified
#
# expr-op an operation corresponding to an AST expression node
# (some sort of Expr object). Gen-ZAM generates code for
# automatically converting Expr objects to ZAM instructions.
# The name of the operation must match that used in the AST
# tag, so for example for "expr-op Foo" there must be a
# corresponding "EXPR_FOO" tag.
#
# unary-expr-op an expr-op for a unary Expr object
# binary-expr-op an expr-op for a binary Expr object
# rel-expr-op an expr-op for a (binary) Expr object that
# represents a relational operation
#
# assign-op directly assigning either a ZVal or a record field
# to either a frame slot or a record field
#
# unary-op an operation with one operand that requires special
# treatment that doesn't fit with how unary-expr-op's
# are expressed
#
# direct-unary-op an operation with one operand that corresponds to
# a specific ZAMCompiler method for generating its
# instruction
#
# internal-op similar to "op", but for ZAM instructions only used
# internally, and thus not having any AST counterpart
# internal-assignment-op the same, for operations that assign ZVals
# produced by loading interpreter variables
# or calling functions
#
# After specifying the type of operation, you list additional attributes to
# fill out the template, ending by convention with the C++ evaluation snippet
# (if appropriate). The most significant (and complex) of these are:
#
# class specifies how to interpret the operation in terms of ZAM
# instruction slots (and constant). The specification is
# in terms of single-letter mnemonics for the different
# possible classes:
#
# F special value designating a record field being
# assigned to
# H event handler
# L list of values
# O opaque value (here, "opaque" refers to ZAM
# internals, not OpaqueVal)
# R record
# V variable (frame slot)
# X used to indicate an empty specifier
# b branch target
# f iteration information associated with table "for" loop
# g access to a global
# i integer constant, often a record field offset
# s iteration information associated with stepping
# through a vector or string
#
# The full specification consists of concatenating mnemonics
# with the order left-to-right corresponding to each of the
# instruction's 4 integer values (stopping with the last one
# used). If the operation includes a constant, then it is
# listed at the point reflecting where the constant is used as
# an operand. For example, a class of "VVCV" means that the
# first integer is used as a frame variable (i.e., the usual
# "assignment slot"), the second integer (first "operand") is
# also a frame variable, the second operand is the instruction's
# constant, and the third operand is the instruction's third
# integer value, with the fourth integer value not being used.
#
# classes specifies a number of "class" values to instantiate over.
# Cannot be combined with "class", nor used for expressions.
#
# op-type for some form of expr-op, specifies to which Zeek scripting
# types the expression applies:
#
# A addr
# D double
# F file
# I int
# N subnet
# P pattern
# R record
# S string
# T table
# U count
# V vector
#
# along with two special types: 'X' indicates that Gen-ZAM
# should not iterate over any possible values, and '*'
# indicates that Gen-ZAM should additionally iterate over
# all of possible values not explicitly listed (used in
# conjunction with eval-type - see below)
#
# op-types similar to op-type, but lists a type for each operand
# (including assignment target), so for example "A N A"
# would correspond to a 3-operand instruction for which
# the first operand (or assignment target) is an "addr",
# the second a "subnet", and the third another "addr".
#
# Note that these types collectively apply to each instance of
# an operation, whereas listing multiple "op-type" types
# iterates through those one-at-a-time in turn (and generally
# the point is that the each type applies to *all* operands,
# rather than a per-operand list). Given that, the two are
# incompatible.
#
# For operands corresponding to 'i' or any of the internal types,
# such as 'b', 'f', 'g', and 's', the corresponding type to
# list is 'I', used for integer access.
#
# eval specifies a block of C++ code used to evaluation the
# execution of the instruction. The block begins with the
# remainder of the "eval" line and continues until either a
# blank line or a line that starts with non-whitespace.
#
# Blocks can include special '$' parameters that Gen-ZAM
# automatically expands. "$1" refers to an operation's first
# operand, "$2" to its second, etc. "$$" refers to the
# operation's assignment target.
#
# For simple expr-op's you can express the block as simply
# the C++ expression to compute. For example, for multiplication
# (named "Times"), the "eval" block is simply "$1 * $2",
# rather than "$$ = $1 * $2"; Gen-ZAM knows to expand it
# accordingly.
#
# Finally, to help with avoiding duplicate code, you can
# define macros that expand to code snippets you want to use
# in multiple places. You specify these using a "macro"
# keyword followed by the name of the macro and an evaluation
# block. Macros behave identically to C++ #define's, except
# you don't use "\" to continue them across line breaks, but
# instead just indent the lines you want included, ending
# (as with "eval" blocks) with an empty line or a line that
# starts with non-whitespace.
#
# We list the remaining types of attributes alphabetically. Note that some
# only apply to certain types of operations.
#
# assign-val for an assignment operation, the name of the
# C++ variable that holds the value to assign
#
# custom-method a ZAMCompiler method that Gen-ZAM should use for
# this operation, rather than generating one
#
# eval-mixed an expression "eval" block that applies to two
# different op-type's
#
# eval-type evaluation code associated with one specific op-type
#
# explicit-result-type the operation's evaluation yields a ZVal
# rather than a low-level C++ type
#
# field-op the operation is a direct assignment to a record field
#
# includes-field-op the operation should include a version
# that assigns to a record field as well as a
# version for assigning to a frame variable
#
# indirect-call the operation represents an indirect call (through
# a global variable, rather than directly). Only
# meaningful if num-call-args is also specified.
#
# indirect-local-call same, but via a local variable rather than
# global
#
# method-post C++ code to add to the end of the method that
# dynamically generates ZAM code
#
# no-const do not generate a version of the unary-expr-op
# where the operand is a constant
#
# no-eval this operation does not have an "eval" block
# (because it will be translated instead into internal
# operations)
#
# num-call-args indicates that the operation is a function call,
# and specifies how many arguments the call takes.
# A specification of 'n' means "build a ZAM instruction
# for calling with an arbitrary number of arguments".
#
# op1-internal states that the operation's treatment of the
# instruction's first integer value is for internal
# purposes; the value does not correspond to a frame
# variable
#
# op1-read the operation treats the instruction's first integer
# value as a frame variable, but only reads the value.
# (The default is that the frame variable is written
# to but not read.)
#
# op1-read-write the operation treats the instruction's first integer
# value as a frame variable, and both reads and
# writes the value.
#
# precheck a test conducted before evaluating an expression,
# which is skipped if the test is true. Must be used
# in conjunction with precheck-action.
#
# precheck-action code to execute if a precheck is true, instead
# of evaluating the expression. Must be used in
# conjunction with precheck.
#
# set-type the instruction's primary type comes from either the
# assignment target ("$$"), the first operand ("$1"),
# or the second operand ("$2")
#
# set-type2 the same as set-type but for the instruction's
# secondary type
#
# side-effects the operation has side-effects, so even if its
# assignment target winds up being "dead" (the value is
# no longer used), the operation should still occur.
# Optionally, this attribute can include two arguments
# specifying the ZAM opcode to use if the assignment
# is dead, and the internal "type" of that opcode.
#
# For example, "side-effects OP_CALL1_V OP_V" means
# "this operation has side-effects; if eliminating
# its assignment, change the ZAM op-code to OP_CALL1_V,
# which has an internal type of OP_V".
#
# vector generate a version of the operation that takes
# vectors as operands
#
# Finally, a note concernning comments: due to internal use of C++ #define
# macros, comments in C++ code should use /* ... */ rather than // delimiters.

3368
src/script_opt/ZAM/OPs/ZAM.op
View file

File diff suppressed because it is too large Load diff

627
src/script_opt/ZAM/OPs/ZBI.op Normal file
View file

@ -0,0 +1,627 @@
# Operations corresponding to ZAM BuiltIn Functions.
internal-op Remove-Teredo
op1-read
class V
op-types R
eval auto teredo = zeek::packet_mgr->GetAnalyzer("Teredo");
if ( teredo )
{
zeek::detail::ConnKey conn_key($1);
static_cast<zeek::packet_analysis::teredo::TeredoAnalyzer*>(teredo.get())->RemoveConnection(conn_key);
}
internal-op Remove-Teredo
side-effects OP_REMOVE_TEREDO_V OP_V
class VV
op-types I R
eval auto teredo = zeek::packet_mgr->GetAnalyzer("Teredo");
if ( teredo )
{
zeek::detail::ConnKey conn_key($1);
static_cast<zeek::packet_analysis::teredo::TeredoAnalyzer*>(teredo.get())->RemoveConnection(conn_key);
}
$$ = 1;
internal-op Remove-GTPv1
op1-read
class V
op-types R
eval auto gtpv1 = zeek::packet_mgr->GetAnalyzer("GTPv1");
if ( gtpv1 )
{
zeek::detail::ConnKey conn_key($1);
static_cast<zeek::packet_analysis::gtpv1::GTPv1_Analyzer*>(gtpv1.get())->RemoveConnection(conn_key);
}
internal-op Remove-GTPv1
side-effects OP_REMOVE_GTPV1_V OP_V
class VV
op-types I R
eval auto gtpv1 = zeek::packet_mgr->GetAnalyzer("GTPv1");
if ( gtpv1 )
{
zeek::detail::ConnKey conn_key($1);
static_cast<zeek::packet_analysis::gtpv1::GTPv1_Analyzer*>(gtpv1.get())->RemoveConnection(conn_key);
}
$$ = 1;
internal-op Set-File-Handle
op1-read
class V
op-types S
eval auto handle = $1;
auto bytes = reinterpret_cast<const char*>(handle->Bytes());
auto h = std::string(bytes, handle->Len());
zeek::file_mgr->SetHandle(h);
internal-op Subnet-To-Addr
class VV
op-types X N
eval auto addr_v = make_intrusive<AddrVal>($1->Prefix());
ZVal::DeleteManagedType($$);
$$ = ZVal(std::move(addr_v));
macro EvalSubBytes(lhs, arg1, arg2, arg3)
{
auto sv = ZAM_sub_bytes(arg1, arg2, arg3);
Unref(lhs);
lhs = sv;
}
internal-op Sub-Bytes
classes VVVV VVVC VVCV VCVV VVCi VCVi
op-types S S U I
eval EvalSubBytes($$, $1, $2, $3)
# Use a distinct name because due to the convention when constructing
# instructions, frame slots are always positioned earlier than non-frame
# slots, i.e. we can't construct "VCiV", which is why the arguments are
# in a different order than above.
internal-op Sub-Bytes2
class VCVi
op-types S S I U
eval EvalSubBytes($$, $1, $3, $2)
internal-op Time-To-Double
class VV
op-types D D
eval $$ = $1;
internal-op To-Lower
class VV
op-types S S
eval auto sv = ZAM_to_lower($1);
Unref($$);
$$ = sv;
# A ZAM version of Log::__write. In calls to it, the first argument
# is generally a constant (enum) *if we inlined*, but otherwise a
# parameter, so we support both VVV ad VVC.
#
# It's actually the case that the return value is pretty much always
# ignored ... plus optimization can elide it away. See the second
# pair of built-ins for versions that discard the return value.
#
# Could speed things up further by modifying the Write method to just
# take the raw enum value, as it appears that that's all that's ever
# actually used.
macro LogWritePre(id_val, columns_val)
auto id = id_val;
auto columns = columns_val;
macro LogWriteResPost(lhs)
bool result = log_mgr->Write(id->AsEnumVal(), columns->AsRecordVal());
lhs = result;
internal-op Log-Write
side-effects OP_LOG_WRITE_VV OP_VV
class VVV
op-types I X R
eval LogWritePre(LogEnum($1), $2)
LogWriteResPost($$)
### Check that invoked correctly
internal-op Log-WriteC
side-effects OP_LOG_WRITE_CV OP_VC
class VCV
op-types I X R
eval LogWritePre(LogEnum($1), $2)
LogWriteResPost($$)
# Versions that discard the return value.
internal-op Log-Write
side-effects
op1-read
classes VV CV
op-types X R
eval LogWritePre(LogEnum($1), $2)
(void) log_mgr->Write(id->AsEnumVal(), columns->AsRecordVal());
internal-op Broker-Flush-Logs
side-effects OP_BROKER_FLUSH_LOGS_X OP_X
class V
op-types U
eval $$ = broker_mgr->FlushLogBuffers();
internal-op Broker-Flush-Logs
side-effects
class X
eval (void) broker_mgr->FlushLogBuffers();
internal-op Get-Port-Transport-Proto
class VV
op-types U U
eval auto mask = $1 & PORT_SPACE_MASK;
auto v = 0; /* TRANSPORT_UNKNOWN */
if ( mask == TCP_PORT_MASK )
v = 1;
else if ( mask == UDP_PORT_MASK )
v = 2;
else if ( mask == ICMP_PORT_MASK )
v = 3;
$$ = v;
predicate-op Conn-Exists
class V
op-types R
eval session_mgr->FindConnection($1) != nullptr
internal-op Lookup-Conn
class VV
op-types X R
eval auto cid = $1;
Connection* conn = session_mgr->FindConnection(cid);
ValPtr res;
if ( conn )
res = conn->GetVal();
else
{
ERROR2("connection ID not a known connection", cid);
res = build_dummy_conn_record();
}
AssignTarget($$, ZVal(res, res->GetType()));
predicate-op Is-ICMP-Port
class V
op-types U
eval ($1 & PORT_SPACE_MASK) == ICMP_PORT_MASK
predicate-op Is-TCP-Port
class V
op-types U
eval ($1 & PORT_SPACE_MASK) == TCP_PORT_MASK
predicate-op Is-UDP-Port
class V
op-types U
eval ($1 & PORT_SPACE_MASK) == UDP_PORT_MASK
predicate-op Is-V4-Addr
class V
op-types A
eval $1->AsAddr().GetFamily() == IPv4
predicate-op Is-V6-Addr
class V
op-types A
eval $1->AsAddr().GetFamily() == IPv6
internal-op Network-Time
class V
op-types D
eval $$ = run_state::network_time;
internal-op Current-Time
class V
op-types D
eval $$ = util::current_time();
predicate-op Reading-Live-Traffic
class X
eval run_state::reading_live
predicate-op Reading-Traces
class X
eval run_state::reading_traces
internal-op Sort
op1-read
class V
op-types V
eval if ( $1->Size() > 1 )
$1->Sort();
internal-op Sort
class VV
op-types V V
eval auto vv = $1;
if ( vv->Size() > 1 )
vv->Sort();
zeek::Ref(vv);
Unref($$);
$$ = vv;
internal-op Sort-With-Cmp
op1-read
class VV
op-types V F
eval if ( $1->Size() > 1 )
$1->Sort($2);
internal-op Sort-With-Cmp
class VVV
op-types V V F
eval auto vv = $1;
if ( vv->Size() > 1 )
vv->Sort($2);
zeek::Ref(vv);
Unref($$);
$$ = vv;
internal-op Starts-With
classes VVV VCV VVC
op-types I S S
eval auto str = $1;
auto sub = $2;
auto str_n = str->Len();
auto sub_n = sub->Len();
if ( str_n < sub_n )
$$ = 0;
else
{
auto str_b = str->Bytes();
auto sub_b = sub->Bytes();
int i;
for ( i = 0; i < sub_n; ++i )
if ( str_b[i] != sub_b[i] )
break;
$$ = i == sub_n;
}
internal-op StrCmp
classes VVV VCV VVC
op-types I S S
eval auto s1 = $1;
auto s2 = $2;
$$ = Bstr_cmp(s1->AsString(), s2->AsString());
internal-op StrStr
classes VVV VCV VVC
op-types I S S
eval auto big = $1;
auto little = $2;
$$ = 1 + big->AsString()->FindSubstring(little->AsString());
macro Cat1Op(lhs, val)
auto& v1 = lhs;
ZVal::DeleteManagedType(v1);
v1 = val;
internal-op Cat1
classes VV VC
eval Cat1Op($$, $1)
zeek::Ref(v1.AsString());
internal-op Cat1Full
classes VV VC
eval auto formatted_val = ZVal(ZAM_val_cat($1.ToVal(Z_TYPE)));
Cat1Op($$, formatted_val)
internal-op CatN
class V
eval CatNPre()
int n = aux->n;
size_t max_size = 0;
for ( int i = 0; i < n; ++i )
max_size += ca[i]->MaxSize(aux->elems[i].ToDirectZVal(frame));
auto res = new char[max_size + /* slop */ n + 1];
auto res_p = res;
for ( int i = 0; i < n; ++i )
ca[i]->RenderInto(aux->elems[i].ToDirectZVal(frame), res_p);
*res_p = '\0';
auto s = new String(true, reinterpret_cast<byte_vec>(res), res_p - res);
Cat1Op($$, ZVal(new StringVal(s)))
macro CatNPre()
auto aux = Z_AUX;
auto& ca = aux->cat_args;
macro CatNMid()
auto res = new char[max_size + /* slop */ 10];
auto res_p = res;
macro CatNPost(lhs)
*res_p = '\0';
auto s = new String(true, reinterpret_cast<byte_vec>(res), res_p - res);
Cat1Op(lhs, ZVal(new StringVal(s)))
internal-op Cat2
class V
eval CatNPre()
size_t max_size = ca[0]->MaxSize(aux->elems[0].ToDirectZVal(frame));
max_size += ca[1]->MaxSize(aux->elems[1].ToDirectZVal(frame));
CatNMid()
ca[0]->RenderInto(aux->elems[0].ToDirectZVal(frame), res_p);
ca[1]->RenderInto(aux->elems[1].ToDirectZVal(frame), res_p);
CatNPost($$)
internal-op Cat3
class V
eval CatNPre()
size_t max_size = ca[0]->MaxSize(aux->elems[0].ToDirectZVal(frame));
max_size += ca[1]->MaxSize(aux->elems[1].ToDirectZVal(frame));
max_size += ca[2]->MaxSize(aux->elems[2].ToDirectZVal(frame));
CatNMid()
ca[0]->RenderInto(aux->elems[0].ToDirectZVal(frame), res_p);
ca[1]->RenderInto(aux->elems[1].ToDirectZVal(frame), res_p);
ca[2]->RenderInto(aux->elems[2].ToDirectZVal(frame), res_p);
CatNPost($$)
internal-op Cat4
class V
eval CatNPre()
size_t max_size = ca[0]->MaxSize(aux->elems[0].ToDirectZVal(frame));
max_size += ca[1]->MaxSize(aux->elems[1].ToDirectZVal(frame));
max_size += ca[2]->MaxSize(aux->elems[2].ToDirectZVal(frame));
max_size += ca[3]->MaxSize(aux->elems[3].ToDirectZVal(frame));
CatNMid()
ca[0]->RenderInto(aux->elems[0].ToDirectZVal(frame), res_p);
ca[1]->RenderInto(aux->elems[1].ToDirectZVal(frame), res_p);
ca[2]->RenderInto(aux->elems[2].ToDirectZVal(frame), res_p);
ca[3]->RenderInto(aux->elems[3].ToDirectZVal(frame), res_p);
CatNPost($$)
internal-op Cat5
class V
eval CatNPre()
size_t max_size = ca[0]->MaxSize(aux->elems[0].ToDirectZVal(frame));
max_size += ca[1]->MaxSize(aux->elems[1].ToDirectZVal(frame));
max_size += ca[2]->MaxSize(aux->elems[2].ToDirectZVal(frame));
max_size += ca[3]->MaxSize(aux->elems[3].ToDirectZVal(frame));
max_size += ca[4]->MaxSize(aux->elems[4].ToDirectZVal(frame));
CatNMid()
ca[0]->RenderInto(aux->elems[0].ToDirectZVal(frame), res_p);
ca[1]->RenderInto(aux->elems[1].ToDirectZVal(frame), res_p);
ca[2]->RenderInto(aux->elems[2].ToDirectZVal(frame), res_p);
ca[3]->RenderInto(aux->elems[3].ToDirectZVal(frame), res_p);
ca[4]->RenderInto(aux->elems[4].ToDirectZVal(frame), res_p);
CatNPost($$)
internal-op Cat6
class V
eval CatNPre()
size_t max_size = ca[0]->MaxSize(aux->elems[0].ToDirectZVal(frame));
max_size += ca[1]->MaxSize(aux->elems[1].ToDirectZVal(frame));
max_size += ca[2]->MaxSize(aux->elems[2].ToDirectZVal(frame));
max_size += ca[3]->MaxSize(aux->elems[3].ToDirectZVal(frame));
max_size += ca[4]->MaxSize(aux->elems[4].ToDirectZVal(frame));
max_size += ca[5]->MaxSize(aux->elems[5].ToDirectZVal(frame));
CatNMid()
ca[0]->RenderInto(aux->elems[0].ToDirectZVal(frame), res_p);
ca[1]->RenderInto(aux->elems[1].ToDirectZVal(frame), res_p);
ca[2]->RenderInto(aux->elems[2].ToDirectZVal(frame), res_p);
ca[3]->RenderInto(aux->elems[3].ToDirectZVal(frame), res_p);
ca[4]->RenderInto(aux->elems[4].ToDirectZVal(frame), res_p);
ca[5]->RenderInto(aux->elems[5].ToDirectZVal(frame), res_p);
CatNPost($$)
internal-op Cat7
class V
eval CatNPre()
size_t max_size = ca[0]->MaxSize(aux->elems[0].ToDirectZVal(frame));
max_size += ca[1]->MaxSize(aux->elems[1].ToDirectZVal(frame));
max_size += ca[2]->MaxSize(aux->elems[2].ToDirectZVal(frame));
max_size += ca[3]->MaxSize(aux->elems[3].ToDirectZVal(frame));
max_size += ca[4]->MaxSize(aux->elems[4].ToDirectZVal(frame));
max_size += ca[5]->MaxSize(aux->elems[5].ToDirectZVal(frame));
max_size += ca[6]->MaxSize(aux->elems[6].ToDirectZVal(frame));
CatNMid()
ca[0]->RenderInto(aux->elems[0].ToDirectZVal(frame), res_p);
ca[1]->RenderInto(aux->elems[1].ToDirectZVal(frame), res_p);
ca[2]->RenderInto(aux->elems[2].ToDirectZVal(frame), res_p);
ca[3]->RenderInto(aux->elems[3].ToDirectZVal(frame), res_p);
ca[4]->RenderInto(aux->elems[4].ToDirectZVal(frame), res_p);
ca[5]->RenderInto(aux->elems[5].ToDirectZVal(frame), res_p);
ca[6]->RenderInto(aux->elems[6].ToDirectZVal(frame), res_p);
CatNPost($$)
internal-op Cat8
class V
eval CatNPre()
size_t max_size = ca[0]->MaxSize(aux->elems[0].ToDirectZVal(frame));
max_size += ca[1]->MaxSize(aux->elems[1].ToDirectZVal(frame));
max_size += ca[2]->MaxSize(aux->elems[2].ToDirectZVal(frame));
max_size += ca[3]->MaxSize(aux->elems[3].ToDirectZVal(frame));
max_size += ca[4]->MaxSize(aux->elems[4].ToDirectZVal(frame));
max_size += ca[5]->MaxSize(aux->elems[5].ToDirectZVal(frame));
max_size += ca[6]->MaxSize(aux->elems[6].ToDirectZVal(frame));
max_size += ca[7]->MaxSize(aux->elems[7].ToDirectZVal(frame));
CatNMid()
ca[0]->RenderInto(aux->elems[0].ToDirectZVal(frame), res_p);
ca[1]->RenderInto(aux->elems[1].ToDirectZVal(frame), res_p);
ca[2]->RenderInto(aux->elems[2].ToDirectZVal(frame), res_p);
ca[3]->RenderInto(aux->elems[3].ToDirectZVal(frame), res_p);
ca[4]->RenderInto(aux->elems[4].ToDirectZVal(frame), res_p);
ca[5]->RenderInto(aux->elems[5].ToDirectZVal(frame), res_p);
ca[6]->RenderInto(aux->elems[6].ToDirectZVal(frame), res_p);
ca[7]->RenderInto(aux->elems[7].ToDirectZVal(frame), res_p);
CatNPost($$)
internal-op Analyzer-Name
classes VV VC
op-types S X
eval auto atype = $1.ToVal(Z_TYPE);
auto val = atype->AsEnumVal();
Unref($$);
plugin::Component* component = zeek::analyzer_mgr->Lookup(val);
if ( ! component )
component = zeek::packet_mgr->Lookup(val);
if ( ! component )
component = zeek::file_mgr->Lookup(val);
if ( component )
$$ = new StringVal(component->CanonicalName());
else
$$ = new StringVal("<error>");
macro FilesAddOrRemoveAnalyzer(file_id_val, tag, args_val, METHOD)
auto file_id = file_id_val;
using zeek::BifType::Record::Files::AnalyzerArgs;
auto rv = args_val->CoerceTo(AnalyzerArgs);
bool result = zeek::file_mgr->METHOD(
file_id->CheckString(),
zeek::file_mgr->GetComponentTag(tag.ToVal(Z_TYPE).get()),
std::move(rv));
macro FilesAddAnalyzer(file_id_val, tag, args_val)
FilesAddOrRemoveAnalyzer(file_id_val, tag, args_val, AddAnalyzer)
internal-op Files-Add-Analyzer
op1-read
classes VVV VCV
op-types S X R
eval FilesAddAnalyzer($1, $2, $3)
internal-op Files-Add-Analyzer
class VVVV
side-effects OP_FILES_ADD_ANALYZER_VVV OP_VVV
op-types I S X R
eval FilesAddAnalyzer($1, $2, $3)
$$ = result;
internal-op Files-Add-Analyzer
class VVCV
op-types I S X R
side-effects OP_FILES_ADD_ANALYZER_VCV OP_VVC
eval FilesAddAnalyzer($1, $2, $3)
$$ = result;
macro FilesRemoveAnalyzer(file_id_val, tag, args_slot)
FilesAddOrRemoveAnalyzer(file_id_val, tag, args_slot, RemoveAnalyzer)
internal-op Files-Remove-Analyzer
op1-read
classes VVV VCV
op-types S X R
eval FilesRemoveAnalyzer($1, $2, $3)
internal-op Files-Remove-Analyzer
class VVVV
op-types I S X R
side-effects OP_FILES_REMOVE_ANALYZER_VVV OP_VVV
eval FilesRemoveAnalyzer($1, $2, $3)
$$ = result;
internal-op Files-Remove-Analyzer
class VVCV
op-types I S X R
side-effects OP_FILES_REMOVE_ANALYZER_VCV OP_VVC
eval FilesRemoveAnalyzer($1, $2, $3)
$$ = result;
internal-op Analyzer-Enabled
classes VV VC
op-types I X
eval auto atype = $1.ToVal(Z_TYPE);
auto c = zeek::file_mgr->Lookup(atype->AsEnumVal());
$$ = c && c->Enabled();
internal-op File-Analyzer-Name
classes VV VC
eval auto atype = $1.ToVal(Z_TYPE);
Unref($$.AsString());
$$ = ZVal(file_mgr->GetComponentNameVal({NewRef{}, atype->AsEnumVal()}));
internal-op Is-Protocol-Analyzer
classes VV VC
op-types I X
eval auto atype = $1.ToVal(Z_TYPE);
$$ = analyzer_mgr->Lookup(atype->AsEnumVal()) != nullptr;
internal-op Clear-Table
op1-read
class V
op-types T
eval $1->RemoveAll();
internal-op Files-Enable-Reassembly
op1-read
class V
op-types S
eval auto f = $1->CheckString();
file_mgr->EnableReassembly(f);
internal-op Files-Set-Reassembly-Buffer
op1-read
classes VV Vi
op-types S U
eval auto f = $1->CheckString();
file_mgr->SetReassemblyBuffer(f, $2);
internal-op Files-Set-Reassembly-Buffer
class VVV
op-types I S U
side-effects OP_FILES_SET_REASSEMBLY_BUFFER_VV OP_VV
eval auto f = $1->CheckString();
$$ = file_mgr->SetReassemblyBuffer(f, $2);
internal-op Files-Set-Reassembly-Buffer
class VVi
op-types I S U
side-effects OP_FILES_SET_REASSEMBLY_BUFFER_Vi OP_VV_I2
eval auto f = $1->CheckString();
$$ = file_mgr->SetReassemblyBuffer(f, $2);
internal-op Get-Bytes-Thresh
classes VVV VVC
op-types U R I
eval auto a = analyzer::conn_size::GetConnsizeAnalyzer($1);
auto res = 0U;
if ( a )
res = static_cast<analyzer::conn_size::ConnSize_Analyzer*>(a)->GetByteAndPacketThreshold(true, $2);
$$ = res;
macro SetBytesThresh(cid, threshold, is_orig)
bool res = false;
auto a = analyzer::conn_size::GetConnsizeAnalyzer(cid);
if ( a )
{
static_cast<analyzer::conn_size::ConnSize_Analyzer*>(a)->SetByteAndPacketThreshold(threshold, true, is_orig);
res = true;
}
internal-op Set-Bytes-Thresh
op1-read
classes VVV VVC VCV VCi
op-types R U I
eval SetBytesThresh($1, $2, $3)
internal-op Set-Bytes-Thresh
class VVVV
op-types I R U I
side-effects OP_SET_BYTES_THRESH_VVV OP_VVV
eval SetBytesThresh($1, $2, $3)
$$ = res;
internal-op Set-Bytes-Thresh
class VVVC
op-types I R U I
side-effects OP_SET_BYTES_THRESH_VVC OP_VVC
eval SetBytesThresh($1, $2, $3)
$$ = res;
internal-op Set-Bytes-Thresh
class VVCV
op-types I R U I
side-effects OP_SET_BYTES_THRESH_VCV OP_VVC
eval SetBytesThresh($1, $2, $3)
$$ = res;
internal-op Set-Bytes-Thresh
class VVCi
op-types I R U I
side-effects OP_SET_BYTES_THRESH_VCi OP_VVC_I2
eval SetBytesThresh($1, $2, $3)
$$ = res;

89
src/script_opt/ZAM/OPs/aggr-assignments.op Normal file
View file

@ -0,0 +1,89 @@
# Operations corresponding to assigning to elements of aggregates.
macro VectorElemAssignPre(vec, index)
auto ind = index.AsCount();
auto vv = vec.AsVector();
macro EvalVectorElemAssign(vec, index, val_setup, assign_op)
VectorElemAssignPre(vec, index)
val_setup
if ( ! assign_op )
ERROR("value used but not set");
op Vector-Elem-Assign
op1-read
set-type $1
class VVV
eval EvalVectorElemAssign($1, $2,, copy_vec_elem(vv, ind, $3, Z_TYPE))
op Any-Vector-Elem-Assign
op1-read
set-type $1
classes VVV VVC
eval EvalVectorElemAssign($1, $2,, vv->Assign(ind, $3.ToVal(Z_TYPE)))
op Vector-Elem-Assign-Any
op1-read
class VVV
op-types X X a
eval EvalVectorElemAssign($1, $2, auto any_v = $3;, vv->Assign(ind, {NewRef{}, any_v}))
op Vector-Elem-Assign
op1-read
set-type $2
class VVC
eval VectorElemAssignPre($1, $2)
(void) copy_vec_elem(vv, ind, $3, Z_TYPE);
# These versions are used when the constant is the index, not the new value.
op Vector-Elem-Assign
op1-read
set-type $1
class VVi
op-types V X U
eval auto vv = $1;
if ( ! copy_vec_elem(vv, $3, $2, Z_TYPE) )
ERROR("value used but not set");
op Any-Vector-Elem-Assign
op1-read
set-type $1
class VVi
op-types V X I
eval auto vv = $1;
if ( ! vv->Assign($3, $2.ToVal(Z_TYPE)) )
ERROR("value used but not set");
op Vector-Elem-Assign-Any
op1-read
class VVi
op-types V a I
eval auto vv = $1;
auto any_v = $2;
vv->Assign($3, {NewRef{}, any_v});
internal-op Vector-Slice-Assign
op1-read
class VV
op-types V V
eval ValPtr vec = {NewRef{}, $1};
auto indices = Z_AUX->ToListVal(frame);
ValPtr vals = {NewRef{}, $2};
bool iterators_invalidated;
auto error = assign_to_index(std::move(vec), std::move(indices), std::move(vals), iterators_invalidated);
if ( error )
ERROR(error);
if ( iterators_invalidated )
WARN("possible loop/iterator invalidation");
internal-op Table-Elem-Assign
op1-read
classes VV VC
op-types T X
eval auto indices = Z_AUX->ToListVal(frame);
auto val = $2.ToVal(Z_TYPE);
bool iterators_invalidated = false;
$1->Assign(std::move(indices), std::move(val), true, &iterators_invalidated);
if ( iterators_invalidated )
WARN("possible loop/iterator invalidation");

104
src/script_opt/ZAM/OPs/binary-exprs.op Normal file
View file

@ -0,0 +1,104 @@
# Operations corresponding to binary expressions.
binary-expr-op Add
op-type I U D S
vector
eval $1 + $2
eval-type S vector<const String*> strings;
strings.push_back($1->AsString());
strings.push_back($2->AsString());
auto res = new StringVal(concatenate(strings));
$$ = res;
binary-expr-op Sub
op-type I U D T
vector
eval $1 - $2
#
eval-type T auto v = $1->Clone();
auto s = v.release()->AsTableVal();
$2->RemoveFrom(s);
$$ = s;
binary-expr-op Times
op-type I U D
vector
eval $1 * $2
binary-expr-op Divide
op-type I U D
vector
#
precheck $2 == 0
precheck-action ERROR("division by zero");
eval $1 / $2
binary-expr-op Mask
# Signal that this expression only has mixed-type evaluation.
op-type X
explicit-result-type
eval-mixed A I auto mask = static_cast<uint32_t>($2);
auto a = $1->AsAddr();
if ( a.GetFamily() == IPv4 && mask > 32 )
ERROR(util::fmt("bad IPv4 subnet prefix length: %" PRIu32, mask));
if ( a.GetFamily() == IPv6 && mask > 128 )
ERROR(util::fmt("bad IPv6 subnet prefix length: %" PRIu32, mask));
auto v = make_intrusive<SubNetVal>(a, mask);
Unref($$.AsSubNet());
$$.AsSubNetRef() = v.release();
binary-expr-op Mod
op-type I U
vector
precheck $2 == 0
precheck-action ERROR("modulo by zero");
eval $1 % $2
binary-expr-op And-And
op-type I
vector
eval zeek_int_t($1 && $2)
binary-expr-op Or-Or
op-type I
vector
eval zeek_int_t($1 || $2)
binary-expr-op And
op-type U P T
vector
eval $1 & $2
#
eval-type P $$ = new PatternVal(RE_Matcher_conjunction($1->AsPattern(), $2->AsPattern()));
#
eval-type T $$ = $1->Intersection(*$2).release();
binary-expr-op Or
op-type U P T
vector
eval $1 | $2
#
eval-type P $$ = new PatternVal(RE_Matcher_disjunction($1->AsPattern(), $2->AsPattern()));
#
eval-type T auto v = $1->Clone();
auto s = v.release()->AsTableVal();
(void) $2->AddTo(s, false, false);
$$ = s;
binary-expr-op Xor
op-type U
vector
eval $1 ^ $2
binary-expr-op Lshift
op-type I U
vector
eval-type I if ( $1 < 0 )
ERROR("left shifting a negative number is undefined");
$$ = $1 << $2;
eval $1 << $2
binary-expr-op Rshift
op-type I U
vector
eval $1 >> $2

180
src/script_opt/ZAM/OPs/calls.op Normal file
View file

@ -0,0 +1,180 @@
# Operations corresponding to function calls.
# A call with no arguments and no return value.
internal-op Call0
op1-read
class X
side-effects
num-call-args 0
# A call with no arguments and a return value.
internal-assignment-op Call0
class V
side-effects OP_CALL0_X OP_X
assign-val v
num-call-args 0
# Calls with 1 argument and no return value.
internal-op Call1
op1-read
classes V C
side-effects
num-call-args 1
# Same but with a return value.
internal-assignment-op Call1
class VV
side-effects OP_CALL1_V OP_V
assign-val v
num-call-args 1
internal-assignment-op Call1
class VC
side-effects OP_CALL1_C OP_C
assign-val v
num-call-args 1
# Calls with 2-5 arguments and no return value.
internal-op Call2
class X
side-effects
num-call-args 2
# Same with a return value.
internal-assignment-op Call2
class V
side-effects OP_CALL2_X OP_X
assign-val v
num-call-args 2
internal-op Call3
class X
side-effects
num-call-args 3
# Same with a return value.
internal-assignment-op Call3
class V
side-effects OP_CALL3_X OP_X
assign-val v
num-call-args 3
internal-op Call4
class X
side-effects
num-call-args 4
# Same with a return value.
internal-assignment-op Call4
class V
side-effects OP_CALL4_X OP_X
assign-val v
num-call-args 4
internal-op Call5
class X
side-effects
num-call-args 5
# Same with a return value.
internal-assignment-op Call5
class V
side-effects OP_CALL5_X OP_X
assign-val v
num-call-args 5
# ... and with an arbitrary number of arguments.
internal-op CallN
class X
side-effects
num-call-args n
# Same with a return value.
internal-assignment-op CallN
class V
side-effects OP_CALLN_X OP_X
assign-val v
num-call-args n
# Same, but for indirect calls via a global variable.
internal-op IndCallN
class X
side-effects
indirect-call
num-call-args n
# Same with a return value.
internal-assignment-op IndCallN
class V
side-effects OP_INDCALLN_X OP_X
assign-val v
indirect-call
num-call-args n
# And versions with a local variable rather than a global.
internal-op Local-IndCallN
op1-read
class V
side-effects
indirect-local-call
num-call-args n
internal-assignment-op Local-IndCallN
class VV
side-effects OP_LOCAL_INDCALLN_V OP_V
assign-val v
indirect-local-call
num-call-args n
# A call made in a "when" context. These always have assignment targets.
# To keep things simple, we just use one generic flavor (for N arguments,
# doing a less-streamlined-but-simpler Val-based assignment).
macro WhenCall(lhs, func)
if ( ! func )
throw ZAMDelayedCallException();
auto trigger = Z_FRAME->GetTrigger();
Val* v = trigger ? trigger->Lookup(Z_AUX->call_expr.get()) : nullptr;
ValPtr vp;
if ( v )
vp = {NewRef{}, v};
else
{
auto aux = Z_AUX;
auto current_assoc = Z_FRAME->GetTriggerAssoc();
auto n = aux->n;
std::vector<ValPtr> args;
for ( auto i = 0; i < n; ++i )
args.push_back(aux->ToVal(frame, i));
Z_FRAME->SetCall(Z_AUX->call_expr.get());
/* It's possible that this function will call another that
* itself returns null because *it* is the actual blocker.
* That will set ZAM_error, which we need to ignore.
*/
auto hold_ZAM_error = ZAM_error;
vp = func->Invoke(&args, Z_FRAME);
ZAM_error = hold_ZAM_error;
Z_FRAME->SetTriggerAssoc(current_assoc);
if ( ! vp )
throw ZAMDelayedCallException();
}
if ( Z_IS_MANAGED )
ZVal::DeleteManagedType(lhs);
lhs = ZVal(vp, Z_TYPE);
internal-op WhenCallN
class V
side-effects
eval WhenCall($$, Z_AUX_FUNC)
internal-op WhenIndCallN
class VV
op-types X F
side-effects
eval WhenCall($$, $1)
# Form for when we need to look up the function value at run-time.
internal-op When-ID-IndCallN
class V
side-effects
eval WhenCall($$, Z_AUX_ID->GetVal()->AsFunc())

151
src/script_opt/ZAM/OPs/coercions.op Normal file
View file

@ -0,0 +1,151 @@
# Operations corresponding to type coercions.
direct-unary-op Arith-Coerce ArithCoerce
internal-op Coerce-UI
class VV
op-types U I
eval auto v = $1;
if ( v < 0 )
ERROR("underflow converting int to count");
else
$$ = zeek_uint_t(v);
internal-op Coerce-UD
class VV
op-types U D
eval auto v = $1;
if ( v < 0.0 )
ERROR("underflow converting double to count");
else if ( v > static_cast<double>(UINT64_MAX) )
ERROR("overflow converting double to count");
else
$$ = zeek_uint_t(v);
internal-op Coerce-IU
class VV
op-types I U
eval auto v = $1;
if ( v > INT64_MAX )
ERROR("overflow converting count to int");
else
$$ = zeek_int_t(v);
internal-op Coerce-ID
class VV
op-types I D
eval auto v = $1;
if ( v < static_cast<double>(INT64_MIN) )
ERROR("underflow converting double to int");
else if ( v > static_cast<double>(INT64_MAX) )
ERROR("overflow converting double to int");
else
$$ = zeek_int_t(v);
internal-op Coerce-DI
class VV
op-types D I
eval $$ = double($1);
internal-op Coerce-DU
class VV
op-types D U
eval $$ = double($1);
macro EvalCoerceVec(lhs, rhs, coercer)
auto old_v1 = lhs.AsVector();
lhs.AsVectorRef() = coercer(rhs.AsVector(), Z_LOC);
Unref(old_v1); /* delayed to allow for same value on both sides */
internal-op Coerce-UI-Vec
class VV
eval EvalCoerceVec($$, $1, vec_coerce_UI)
internal-op Coerce-UD-Vec
class VV
eval EvalCoerceVec($$, $1, vec_coerce_UD)
internal-op Coerce-IU-Vec
class VV
eval EvalCoerceVec($$, $1, vec_coerce_IU)
internal-op Coerce-ID-Vec
class VV
eval EvalCoerceVec($$, $1, vec_coerce_ID)
internal-op Coerce-DI-Vec
class VV
eval EvalCoerceVec($$, $1, vec_coerce_DI)
internal-op Coerce-DU-Vec
class VV
eval EvalCoerceVec($$, $1, vec_coerce_DU)
direct-unary-op Record-Coerce RecordCoerce
internal-op Record-Coerce
class VV
op-types R R
eval auto rt = cast_intrusive<RecordType>(Z_TYPE);
auto v = $1;
auto to_r = coerce_to_record(std::move(rt), v, Z_AUX_MAP);
Unref($$);
$$ = to_r.release();
direct-unary-op Table-Coerce TableCoerce
internal-op Table-Coerce
class VV
op-types T T
eval auto tv = $1;
if ( tv->Size() > 0 )
ERROR("coercion of non-empty table/set");
else
{
auto tt = cast_intrusive<TableType>(Z_TYPE);
AttributesPtr attrs = tv->GetAttrs();
auto t = make_intrusive<TableVal>(tt, attrs);
Unref($$);
$$ = t.release();
}
direct-unary-op Vector-Coerce VectorCoerce
internal-op Vector-Coerce
class VV
op-types V V
eval if ( $1->Size() > 0 )
ERROR("coercion of non-empty vector");
else
{
auto vv = new VectorVal(cast_intrusive<VectorType>(Z_TYPE));
Unref($$);
$$ = vv;
}
unary-expr-op To-Any-Coerce
op-type X
set-type $1
eval AssignTarget($$, ZVal($1.ToVal(Z_TYPE), ZAM::any_base_type))
unary-expr-op From-Any-Coerce
no-const
op-type X
set-type $$
eval auto v = $1.AsAny();
AssignTarget($$, ZVal({NewRef{}, v}, Z_TYPE))
unary-expr-op From-Any-Vec-Coerce
no-const
op-type X
set-type $$
eval auto vv = $1.AsVector();
if ( ! vv->Concretize(Z_TYPE->Yield()) )
ERROR("incompatible vector-of-any");
else
{
zeek::Ref(vv);
AssignTarget($$, ZVal(vv))
}

263
src/script_opt/ZAM/OPs/constructors.op Normal file
View file

@ -0,0 +1,263 @@
# Operations corresponding to aggregated constructors.
# Table construction requires atypical evaluation of list elements
# using information from their expression specifics.
direct-unary-op Table-Constructor ConstructTable
macro ConstructTableOrSetPre(width)
auto tt = cast_intrusive<TableType>(Z_TYPE);
auto new_t = new TableVal(tt, Z_AUX_ATTRS);
auto aux = Z_AUX;
auto n = aux->n;
auto ind_width = width;
macro ConstructTableOrSetPost(lhs)
auto& t = lhs.AsTableRef();
Unref(t);
t = new_t;
internal-op Construct-Table
class Vi
eval ConstructTableOrSetPre($1)
for ( auto i = 0; i < n; ++i )
{
auto indices = aux->ToIndices(frame, i, ind_width);
auto v = aux->ToVal(frame, i + ind_width);
new_t->Assign(indices, v);
i += ind_width;
}
ConstructTableOrSetPost($$)
# When tables are constructed, if their &default is a lambda with captures
# then we need to explicitly set up the default.
internal-op Set-Table-Default-Lambda
op1-read
class VV
op-types T X
eval auto tbl = $1;
auto lambda = $2.ToVal(Z_TYPE);
tbl->InitDefaultVal(std::move(lambda));
direct-unary-op Set-Constructor ConstructSet
internal-op Construct-Set
class Vi
eval ConstructTableOrSetPre($1)
for ( auto i = 0; i < n; i += ind_width )
{
auto indices = aux->ToIndices(frame, i, ind_width);
new_t->Assign(indices, nullptr);
}
ConstructTableOrSetPost($$)
direct-unary-op Record-Constructor ConstructRecord
direct-unary-op Rec-Construct-With-Rec ConstructRecordFromRecord
macro ConstructRecordPost(lhs)
Unref(lhs);
lhs = new RecordVal(cast_intrusive<RecordType>(Z_TYPE), std::move(init_vals));
op Construct-Direct-Record
class V
op-types R
eval auto init_vals = Z_AUX->ToZValVec(frame);
ConstructRecordPost($$)
op Construct-Known-Record
class V
op-types R
eval auto init_vals = Z_AUX->ToZValVecWithMap(frame);
ConstructRecordPost($$)
macro AssignFromRec(rhs)
/* The following is defined below, for use by Rec-Assign-Fields */
SetUpRecFieldOps(lhs_map)
auto is_managed = Z_AUX->is_managed;
for ( size_t i = 0U; i < n; ++i )
{
auto rhs_i = rhs->RawField(rhs_map[i]);
if ( is_managed[i] )
zeek::Ref(rhs_i.ManagedVal());
init_vals[lhs_map[i]] = rhs_i;
}
op Construct-Known-Record-From
class VV
op-types R R
eval auto init_vals = Z_AUX->ToZValVecWithMap(frame);
AssignFromRec($1)
ConstructRecordPost($$)
macro DoNetworkTimeInit(slot)
init_vals[slot] = ZVal(run_state::network_time);
op Construct-Known-Record-With-NT
class Vi
op-types R I
eval auto init_vals = Z_AUX->ToZValVecWithMap(frame);
DoNetworkTimeInit($1)
ConstructRecordPost($$)
op Construct-Known-Record-With-NT-From
class VVi
op-types R R I
eval auto init_vals = Z_AUX->ToZValVecWithMap(frame);
DoNetworkTimeInit($2)
AssignFromRec($1)
ConstructRecordPost($$)
macro GenInits()
auto init_vals = Z_AUX->ToZValVecWithMap(frame);
for ( auto& fi : *Z_AUX->field_inits )
init_vals[fi.first] = fi.second->Generate();
op Construct-Known-Record-With-Inits
class V
op-types R
eval GenInits()
ConstructRecordPost($$)
op Construct-Known-Record-With-Inits-From
class VV
op-types R R
eval GenInits()
AssignFromRec($1)
ConstructRecordPost($$)
op Construct-Known-Record-With-Inits-And-NT
class Vi
op-types R I
eval GenInits()
DoNetworkTimeInit($1)
ConstructRecordPost($$)
op Construct-Known-Record-With-Inits-And-NT-From
class VVi
op-types R R I
eval GenInits()
DoNetworkTimeInit($2)
AssignFromRec($1)
ConstructRecordPost($$)
macro SetUpRecFieldOps(which_lhs_map)
auto& lhs_map = Z_AUX->which_lhs_map;
auto& rhs_map = Z_AUX->rhs_map;
auto n = rhs_map.size();
op Rec-Assign-Fields
op1-read
class VV
op-types R R
eval SetUpRecFieldOps(map)
for ( size_t i = 0U; i < n; ++i )
$1->RawOptField(lhs_map[i]) = $2->RawField(rhs_map[i]);
macro DoManagedRecAssign(lhs, rhs)
auto is_managed = Z_AUX->is_managed;
for ( size_t i = 0U; i < n; ++i )
if ( is_managed[i] )
{
auto& lhs_i = lhs->RawOptField(lhs_map[i]);
auto rhs_i = rhs->RawField(rhs_map[i]);
zeek::Ref(rhs_i.ManagedVal());
if ( lhs_i )
ZVal::DeleteManagedType(*lhs_i);
lhs_i = rhs_i;
}
else
lhs->RawOptField(lhs_map[i]) = rhs->RawField(rhs_map[i]);
op Rec-Assign-Fields-Managed
op1-read
class VV
op-types R R
eval SetUpRecFieldOps(map)
DoManagedRecAssign($1, $2)
op Rec-Assign-Fields-All-Managed
op1-read
class VV
op-types R R
eval SetUpRecFieldOps(map)
for ( size_t i = 0U; i < n; ++i )
{
auto& lhs_i = $1->RawOptField(lhs_map[i]);
auto rhs_i = $2->RawField(rhs_map[i]);
zeek::Ref(rhs_i.ManagedVal());
if ( lhs_i )
ZVal::DeleteManagedType(*lhs_i);
lhs_i = rhs_i;
}
op Rec-Add-Int-Fields
op1-read
class VV
op-types R R
eval SetUpRecFieldOps(map)
for ( size_t i = 0U; i < n; ++i )
$1->RawField(lhs_map[i]).AsIntRef() += $2->RawField(rhs_map[i]).AsInt();
op Rec-Add-Double-Fields
op1-read
class VV
op-types R R
eval SetUpRecFieldOps(map)
for ( size_t i = 0U; i < n; ++i )
$1->RawField(lhs_map[i]).AsDoubleRef() += $2->RawField(rhs_map[i]).AsDouble();
op Rec-Add-Fields
op1-read
class VV
op-types R R
eval SetUpRecFieldOps(map)
auto& types = Z_AUX->types;
for ( size_t i = 0U; i < n; ++i )
{
auto& lhs_i = $1->RawField(lhs_map[i]);
auto rhs_i = $2->RawField(rhs_map[i]);
auto tag = types[i]->Tag();
if ( tag == TYPE_INT )
lhs_i.AsIntRef() += rhs_i.AsInt();
else if ( tag == TYPE_COUNT )
lhs_i.AsCountRef() += rhs_i.AsCount();
else
lhs_i.AsDoubleRef() += rhs_i.AsDouble();
}
# Special instruction for concretizing vectors that are fields in a
# newly-constructed record. "aux" holds which fields in the record to
# inspect.
op Concretize-Vector-Fields
op1-read
class V
op-types R
eval auto rt = cast_intrusive<RecordType>(Z_TYPE);
auto r = $1;
auto aux = Z_AUX;
auto n = aux->n;
for ( auto i = 0; i < n; ++i )
{
auto ind = aux->elems[i].IntVal();
auto v_i = r->GetField(ind);
ASSERT(v_i);
if ( v_i->GetType<VectorType>()->IsUnspecifiedVector() )
{
const auto& t_i = rt->GetFieldType(ind);
v_i->AsVectorVal()->Concretize(t_i->Yield());
}
}
direct-unary-op Vector-Constructor ConstructVector
internal-op Construct-Vector
class V
op-types V
eval auto new_vv = new VectorVal(cast_intrusive<VectorType>(Z_TYPE));
auto aux = Z_AUX;
auto n = aux->n;
for ( auto i = 0; i < n; ++i )
new_vv->Assign(i, aux->ToVal(frame, i));
auto& vv = $$;
Unref(vv);
vv = new_vv;

212
src/script_opt/ZAM/OPs/indexing.op Normal file
View file

@ -0,0 +1,212 @@
# Operations corresponding to indexing of tables, vectors, strings,
# and "any" values.
op IndexVecBoolSelect
classes VVV VCV
op-types V V V
set-type $$
eval if ( $1->Size() != $2->Size() )
ERROR("size mismatch, boolean index and vector");
else
{
auto vt = cast_intrusive<VectorType>(Z_TYPE);
auto v2 = $1;
auto v3 = $2;
auto v = vector_bool_select(std::move(vt), v2, v3);
Unref($$);
$$ = v.release();
}
op IndexVecIntSelect
classes VVV VCV
op-types V V V
set-type $$
eval auto vt = cast_intrusive<VectorType>(Z_TYPE);
auto v2 = $1;
auto v3 = $2;
auto v = vector_int_select(std::move(vt), v2, v3);
Unref($$);
$$ = v.release();
op Index
class VVL
custom-method return CompileIndex(n1, n2, l, false);
no-eval
op Index
class VCL
custom-method return CompileIndex(n, c, l, false);
no-eval
op WhenIndex
class VVL
custom-method return CompileIndex(n1, n2, l, true);
no-eval
op WhenIndex
class VCL
custom-method return CompileIndex(n, c, l, true);
no-eval
internal-op Index-Vec
class VVV
op-types X V I
eval EvalIndexVec($$, $1, $2)
macro EvalIndexVec(lhs, rhs_vec, index)
auto& vv = rhs_vec->RawVec();
zeek_int_t ind = index;
if ( ind < 0 )
ind += vv.size();
if ( ind < 0 || ind >= int(vv.size()) )
ERROR("no such index");
AssignTarget(lhs, CopyVal(*(vv[ind])))
internal-op Index-VecC
class VVi
op-types X V I
eval EvalIndexVec($$, $1, $2)
internal-op Index-Any-Vec
class VVV
op-types X V I
eval EvalIndexAnyVec($$, $1, $2)
macro EvalIndexAnyVec(lhs, vec, index)
auto vv = vec;
zeek_int_t ind = index;
if ( ind < 0 )
ind += vv->Size();
if ( ind < 0 || ind >= int(vv->Size()) )
ERROR("no such index");
AssignTarget(lhs, ZVal(vv->ValAt(ind).release()))
internal-op Index-Any-VecC
class VVi
op-types X V I
eval EvalIndexAnyVec($$, $1, $2)
macro WhenIndexResCheck(vec)
if ( vec && IndexExprWhen::evaluating > 0 )
IndexExprWhen::results.push_back({NewRef{}, vec});
internal-op When-Index-Vec
class VVV
op-types X V I
eval EvalIndexAnyVec($$, $1, $2)
WhenIndexResCheck($$.AsVector())
internal-op When-Index-VecC
class VVi
op-types X V I
eval EvalIndexAnyVec($$, $1, $2)
WhenIndexResCheck($$.AsVector())
macro EvalVecSlice(lhs, vv)
auto vec = vv;
auto v = index_slice(vec, indices.get());
Unref(lhs);
lhs = v.release();
internal-op Index-Vec-Slice
class VV
op-types V V
eval auto indices = Z_AUX->ToListVal(frame);
EvalVecSlice($$, $1)
internal-op When-Index-Vec-Slice
class VV
op-types V V
eval auto indices = Z_AUX->ToListVal(frame);
EvalVecSlice($$, $1)
WhenIndexResCheck($$)
internal-op Table-Index
class VV
eval auto indices = Z_AUX->ToListVal(frame);
EvalTableIndex($1, indices)
if ( v )
AssignTarget($$, BuildVal(v, Z_TYPE))
internal-op Table-PatStr-Index
classes VVV VVC
op-types X T S
eval auto vec = ZVal($1->LookupPattern({NewRef{}, $2}));
ZVal::DeleteManagedType($$);
$$ = vec;
internal-op When-Table-Index
class VV
eval auto indices = Z_AUX->ToListVal(frame);
EvalTableIndex($1, indices)
if ( v )
{
if ( IndexExprWhen::evaluating > 0 )
IndexExprWhen::results.emplace_back(v);
AssignTarget($$, BuildVal(v, Z_TYPE))
}
macro EvalTableIndex(tbl, index)
auto v = tbl.AsTable()->FindOrDefault(index);
if ( ! v )
ERROR("no such index");
internal-op When-PatStr-Index
class VV
op-types X T
eval auto indices = Z_AUX->ToListVal(frame);
auto arg0 = indices->Idx(0);
auto v = $1->LookupPattern({NewRef{}, arg0->AsStringVal()});
if ( IndexExprWhen::evaluating > 0 )
IndexExprWhen::results.emplace_back(v);
AssignTarget($$, BuildVal(v, Z_TYPE))
internal-assignment-op Table-Index1
classes VVV VVC
assign-val v
eval EvalTableIndex($1, $2.ToVal(Z_TYPE))
# No AssignTarget needed, as this is an assignment-op
# This version is for a variable v3.
internal-op Index-String
class VVV
op-types S S I
eval EvalIndexString($$, $1, $2)
macro EvalIndexString(lhs, s, index)
auto str = s->AsString();
auto len = str->Len();
auto idx = index;
if ( idx < 0 )
idx += len;
auto v = str->GetSubstring(idx, 1);
Unref(lhs);
lhs = new StringVal(v ? v : new String(""));
# This version is for a constant v3.
internal-op Index-StringC
class VVi
op-types S S I
eval EvalIndexString($$, $1, $2)
internal-op Index-String-Slice
class VV
op-types S S
eval auto str = $1->AsString();
auto indices = Z_AUX->ToListVal(frame);
auto slice = index_string(str, indices.get());
Unref($$);
$$ = new StringVal(slice->ToStdString());
op AnyIndex
class VVi
op-types X a I
set-type $$
eval auto lv = $1->AsListVal();
if ( $2 < 0 || $2 >= lv->Length() )
reporter->InternalError("bad \"any\" element index");
ValPtr elem = lv->Idx($2);
if ( CheckAnyType(elem->GetType(), Z_TYPE, Z_LOC) )
AssignTarget($$, BuildVal(elem, Z_TYPE))
else
ZAM_error = true;

124
src/script_opt/ZAM/OPs/internal.op Normal file
View file

@ -0,0 +1,124 @@
# Internal operations not directly driven off of AST elements.
# These two are only needed for type-based switch statements. Could think
# about replacing them using CoerceFromAnyExpr.
op Assign-Any
classes VV VC
set-type $1
op-types a X
eval auto v = $1.ToVal(Z_TYPE);
$$ = v.release();
# Lazy way to assign without having to track the specific type of
# a constant.
internal-op Assign-Const
class VC
eval AssignTarget($$, BuildVal($1.ToVal(Z_TYPE), Z_TYPE))
internal-assignment-op Load-Val
class Vi
assign-val v
eval auto& v = Z_FRAME->GetElement($1);
internal-assignment-op Load-Global
# We don't use GlobalVal() for the assignment because we want to leverage
# the bookkeeping that assign-val gives us in terms of memory management.
class Vg
assign-val v
eval auto& v = GlobalID($1)->GetVal();
if ( ! v )
ERROR2("value used but not set", Z_AUX_ID.get());
# We need a special form here for loading global types, as they don't
# fit the usual template.
internal-op Load-Global-Type
class Vg
op-types t I
eval auto& v = $$;
Unref(v);
auto& t = GlobalID($1)->GetType();
v = new TypeVal(t, true);
internal-op Load-Capture
class Vi
eval $$ = Z_FRAME->GetFunction()->GetCapturesVec()[$1];
internal-op Load-Managed-Capture
class Vi
eval auto& lhs = $$;
auto& rhs = Z_FRAME->GetFunction()->GetCapturesVec()[$1];
zeek::Ref(rhs.ManagedVal());
ZVal::DeleteManagedType(lhs);
lhs = rhs;
internal-op Store-Global
op1-internal
class g
eval GlobalID($1)->SetVal(GlobalVal($1).ToVal(Z_TYPE));
# Both of these have the LHS as v2 not v1, to keep with existing
# conventions of OP_VV_I2 op type (as opposed to OP_VV_I1_V2, which doesn't
# currently exist, and would be a pain to add).
internal-op Store-Capture
op1-read
class Vi
eval Z_FRAME->GetFunction()->GetCapturesVec()[$2] = $1;
internal-op Store-Managed-Capture
op1-read
class Vi
eval auto& lhs = Z_FRAME->GetFunction()->GetCapturesVec()[$2];
auto& rhs = $1;
zeek::Ref(rhs.ManagedVal());
ZVal::DeleteManagedType(lhs);
lhs = rhs;
internal-op Copy-To
class VC
set-type $1
eval AssignTarget($$, CopyVal($1))
internal-op GoTo
class b
eval $1
internal-op Hook-Break
class X
eval flow = FLOW_BREAK;
pc = end_pc;
DO_ZAM_PROFILE
continue;
# Slot 2 gives frame size.
internal-op Lambda
class Vi
op-types F I
eval auto& primary_func = Z_AUX_PRIMARY_FUNC;
auto& body = primary_func->GetBodies()[0].stmts;
ASSERT(body->Tag() == STMT_ZAM);
auto lamb = make_intrusive<ScriptFunc>(Z_AUX_ID);
lamb->AddBody(body, $1);
lamb->SetName(Z_AUX_LAMBDA_NAME.c_str());
auto& aux = Z_AUX;
if ( aux->n > 0 )
{
auto captures = std::make_unique<std::vector<ZVal>>();
for ( auto i = 0; i < aux->n; ++i )
{
auto slot = aux->elems[i].Slot();
if ( slot >= 0 )
{
auto& cp = frame[slot];
if ( aux->elems[i].IsManaged() )
zeek::Ref(cp.ManagedVal());
captures->push_back(cp);
}
else
/* Used for when-locals. */
captures->push_back(ZVal());
}
lamb->CreateCaptures(std::move(captures));
}
Unref($$);
$$ = lamb.release();

124
src/script_opt/ZAM/OPs/iterations.op Normal file
View file

@ -0,0 +1,124 @@
# Operations corresponding to iterations.
internal-op Init-Table-Loop
op1-read
class Vf
op-types T I
eval $2.BeginLoop({NewRef{}, $1}, frame, Z_AUX);
internal-op Next-Table-Iter
op1-read
class fb
eval NextTableIterPre($1, $2)
$1.NextIter();
macro NextTableIterPre(iter, BRANCH)
if ( iter.IsDoneIterating() )
BRANCH
internal-op Next-Table-Iter-No-Vars
op1-read
class fb
eval NextTableIterPre($1, $2)
$1.IterFinished();
internal-op Next-Table-Iter-Val-Var
# v1 = slot of the "ValueVar"
class Vfb
eval NextTableIterPre($1, $2)
AssignTarget($$, $1.IterValue());
$1.NextIter();
internal-op Next-Table-Iter-Val-Var-No-Vars
# v1 = slot of the "ValueVar"
class Vfb
eval NextTableIterPre($1, $2)
AssignTarget($$, $1.IterValue());
$1.IterFinished();
internal-op Init-Vector-Loop
op1-read
class Vs
op-types V I
eval auto& vv = $1->RawVec();
$2.InitLoop(&vv);
macro NextVectorIterCore(info, BRANCH)
if ( info.IsDoneIterating() )
BRANCH
const auto& vv = *info.vv;
if ( ! vv[info.iter] )
{ /* Account for vector hole. Re-execute for next position. */
info.IterFinished();
REDO
}
internal-op Next-Vector-Iter
# v1 = iteration variable
class Vsb
op-types U I I
eval NextVectorIterCore($1, $2)
$$ = $1.iter;
$1.IterFinished();
internal-op Next-Vector-Blank-Iter
op1-internal
class sb
eval NextVectorIterCore($1, $2)
$1.IterFinished();
internal-op Next-Vector-Iter-Val-Var
# v1 = iteration variable
# v2 = value variable
op1-read-write
class VVsb
op-types U X I I
eval NextVectorIterCore($2, $3)
$$ = $2.iter;
if ( Z_IS_MANAGED )
$1 = BuildVal(vv[$2.iter]->ToVal(Z_TYPE), Z_TYPE);
else
$1 = *vv[$2.iter];
$2.IterFinished();
internal-op Next-Vector-Blank-Iter-Val-Var
# v1 = value variable
class Vsb
eval NextVectorIterCore($1, $2)
if ( Z_IS_MANAGED )
$$ = BuildVal(vv[$1.iter]->ToVal(Z_TYPE), Z_TYPE);
else
$$ = *vv[$1.iter];
$1.IterFinished();
internal-op Init-String-Loop
op1-read
classes Vs Cs
op-types S I
eval $2.InitLoop($1->AsString());
internal-op Next-String-Iter
# v1 = iteration variable
class Vsb
op-types S I I
eval if ( $1.IsDoneIterating() )
$2
auto bytes = (const char*) $1.s->Bytes() + $1.iter;
auto sv = new StringVal(1, bytes);
Unref($$);
$$ = sv;
$1.IterFinished();
internal-op Next-String-Blank-Iter
op1-internal
class sb
eval if ( $1.IsDoneIterating() )
$2
$1.IterFinished();
internal-op End-Table-Loop
op1-internal
class f
eval $1.Clear();

74
src/script_opt/ZAM/OPs/macros.op Normal file
View file

@ -0,0 +1,74 @@
# General-purpose macros. Those that are specific to a group of instructions
# are defined with those templates rather than appearing here.
# Macros for information associated with the current instruction.
# The Val frame used to pass in arguments.
macro Z_FRAME f
# The main type.
macro Z_TYPE z.GetType()
# Whether it's managed.
macro Z_IS_MANAGED *(z.is_managed)
# Secondary type.
macro Z_TYPE2 z.GetType2()
# Auxiliary information.
macro Z_AUX z.aux
macro Z_AUX_ID z.aux->id_val
macro Z_AUX_FUNC z.aux->func
macro Z_AUX_MAP z.aux->map
macro Z_AUX_ATTRS z.aux->attrs
macro Z_AUX_WHEN_INFO z.aux->wi
macro Z_AUX_EVENT_HANDLER z.aux->event_handler
macro Z_AUX_PRIMARY_FUNC z.aux->lambda->PrimaryFunc()
macro Z_AUX_LAMBDA_NAME z.aux->lambda->Name()
# Location in the original script.
macro Z_LOC z.loc
macro SET_RET_TYPE(type) ret_type = type;
macro INDEX_LIST zam_index_val_list
macro ERROR(msg) ZAM_run_time_error(Z_LOC, msg)
macro ERROR2(msg, obj) ZAM_run_time_error(Z_LOC, msg, obj)
macro WARN(msg) ZAM_run_time_warning(Z_LOC, msg)
# The following abstracts the process of creating a frame-assignable value.
macro BuildVal(v, t) ZVal(v, t)
# Returns a memory-managed-if-necessary copy of an existing value.
macro CopyVal(v) (Z_IS_MANAGED ? BuildVal((v).ToVal(Z_TYPE), Z_TYPE) : (v))
# Managed assignments to the given target.
macro AssignTarget(target, v) {
if ( Z_IS_MANAGED )
{
/* It's important to hold a reference to v here prior
to the deletion in case target points to v. */
auto v2 = v;
ZVal::DeleteManagedType(target);
target = v2;
}
else
target = v;
}
macro Branch(target) { DO_ZAM_PROFILE; pc = target; continue; }
macro REDO { --pc; /* so we then increment to here again */ break; }
macro GlobalID(g) globals[g].id
macro GlobalVal(g) frame[globals[g].slot]
macro StepIter(slot) step_iters[slot]
macro TableIter(slot) (*tiv_ptr)[slot]
macro DirectField(r, f) r->RawField(f)
macro DirectOptField(r, f) r->RawOptField(f)
macro LogEnum(v) v.ToVal(ZAM::log_ID_enum_type)

267
src/script_opt/ZAM/OPs/non-uniform.op Normal file
View file

@ -0,0 +1,267 @@
# Operations corresponding to non-uniform expressions.
assign-op Field
class R
field-op
assign-val v
eval auto r = $1.AsRecord();
auto& rv = DirectOptField(r, $2);
ZVal v;
if ( ! rv )
{
auto def = r->GetType<RecordType>()->FieldDefault($2);
if ( def )
v = ZVal(def, Z_TYPE);
else
ERROR(util::fmt("field value missing: $%s", r->GetType()->AsRecordType()->FieldName($2)));
}
else
v = *rv;
expr-op Has-Field
class VRi
includes-field-op
no-eval
internal-op Has-Field
class VRi
op-types I R I
eval $$ = $1->HasField($2);
internal-op Has-Field
class VRii
op-types R R I I
eval DirectOptField($$, $2) = ZVal(zeek_int_t($1->HasField($3)));
# The following generates an assignment version of Has-Field that we
# don't use (because we need the one above that uses "includes-field-op")
# but lets us compress the two conditionals.
predicate-op Has-Field
class Vi
op-types R I
eval $1->HasField($2)
predicate-op Table-Has-Elements
class V
op-types T
eval $1->Size() > 0
predicate-op Vector-Has-Elements
class V
op-types V
eval $1->Size() > 0
expr-op In
class VVV
custom-method return CompileInExpr(n1, n2, n3);
no-eval
expr-op In
class VCV
custom-method return CompileInExpr(n1, c, n2);
no-eval
expr-op In
class VVC
custom-method return CompileInExpr(n1, n2, c);
no-eval
internal-op P-In-S
classes VVV VCV VVC
op-types I P S
eval $$ = $1->MatchAnywhere($2->AsString()) != 0;
internal-op Str-In-Pat-Tbl
classes VVV VCV
op-types I S T
eval $$ = $2->MatchPattern({NewRef{}, $1});
internal-op S-In-S
classes VVV VCV VVC
op-types I S S
eval auto sc = reinterpret_cast<const unsigned char*>($1->CheckString());
auto cmp = util::strstr_n($2->Len(), $2->Bytes(), $1->Len(), sc);
$$ = cmp != -1;
internal-op A-In-S
classes VVV VCV VVC
op-types I A N
eval $$ = $2->Contains($1->AsAddr());
# Handled differently because of the unusual middle argument.
op L-In-T
class VLV
custom-method return CompileInExpr(n1, l, n2);
no-eval
op L-In-T
class VLC
custom-method return CompileInExpr(n, l, c);
no-eval
op L-In-Vec
class VLV
custom-method return CompileInExpr(n1, l, n2);
no-eval
op L-In-Vec
class VLC
custom-method return CompileInExpr(n, l, c);
no-eval
predicate-op Val-Is-In-Table
class VV
op-types X T
eval $2->Find($1.ToVal(Z_TYPE)) != nullptr
# Variants for indexing two values, one of which might be a constant.
# We set the instructions's *second* type to be that of the first variable
# index. We get the type of the second variable (if any) by digging it
# out of the table's type. For a constant in either position, we use
# the main instruction type, as always.
macro EvalVal2InTableCore(op1, op2)
INDEX_LIST->Clear();
INDEX_LIST->Append(op1);
INDEX_LIST->Append(op2);
macro EvalVal2InTableAssignCore(lhs, tbl)
lhs.AsIntRef() = tbl.AsTable()->Find(INDEX_LIST) != nullptr;
macro EvalVal2InTablePre(op1, op2, tbl)
auto& tt_ind = tbl.AsTable()->GetType()->AsTableType()->GetIndexTypes();
EvalVal2InTableCore(op1.ToVal(Z_TYPE2), op2.ToVal(tt_ind[1]))
internal-op Val2-Is-In-Table
class VVVV
eval EvalVal2InTablePre($1,$2,$3)
EvalVal2InTableAssignCore($$, $3)
internal-op Val2-Is-In-Table-Cond
op1-read
class VVVb
eval EvalVal2InTablePre($1,$2,$3)
EvalVal2InTableCond($3, INDEX_LIST, $4, !)
macro EvalVal2InTableCond(tbl, op, BRANCH, negate)
if ( negate tbl.AsTable()->Find(op) )
BRANCH
internal-op Val2-Is-Not-In-Table-Cond
op1-read
class VVVb
eval EvalVal2InTablePre($1,$2,$3)
EvalVal2InTableCond($3, INDEX_LIST, $4,)
internal-op Val2-Is-In-Table
class VVVC
eval EvalVal2InTableCore($1.ToVal(Z_TYPE2), $3.ToVal(Z_TYPE))
EvalVal2InTableAssignCore($$, $2)
internal-op Val2-Is-In-Table
class VVCV
eval EvalVal2InTableCore($2.ToVal(Z_TYPE), $1.ToVal(Z_TYPE2))
EvalVal2InTableAssignCore($$, $3)
internal-op Val2-Is-In-Table-Cond
op1-read
class VVbC
eval EvalVal2InTableCore($1.ToVal(Z_TYPE2), $4.ToVal(Z_TYPE))
EvalVal2InTableCond($2, INDEX_LIST, $3, !)
internal-op Val2-Is-In-Table-Cond
op1-read
class VVCb
eval EvalVal2InTableCore($3.ToVal(Z_TYPE), $1.ToVal(Z_TYPE2))
EvalVal2InTableCond($2, INDEX_LIST, $4, !)
internal-op Val2-Is-Not-In-Table-Cond
op1-read
class VVbC
eval EvalVal2InTableCore($1.ToVal(Z_TYPE2), $4.ToVal(Z_TYPE))
EvalVal2InTableCond($2, INDEX_LIST, $3, )
internal-op Val2-Is-Not-In-Table-Cond
op1-read
class VVCb
eval EvalVal2InTableCore($3.ToVal(Z_TYPE), $1.ToVal(Z_TYPE2))
EvalVal2InTableCond($2, INDEX_LIST, $4, )
predicate-op Const-Is-In-Table
class VC
op-types T X
eval $1->Find($2.ToVal(Z_TYPE)) != nullptr
internal-op List-Is-In-Table
classes VV VC
op-types I T
eval auto indices = Z_AUX->ToListVal(frame);
$$ = $1->Find(std::move(indices)) != nullptr;
internal-op Val-Is-In-Vector
class VVV
op-types I I V
eval auto vec = $2;
auto ind = $1;
$$ = vec->Has(ind);
internal-op Const-Is-In-Vector
class VCV
op-types I I V
eval auto vec = $2;
auto ind = $1;
$$ = vec->Has(ind);
expr-op Cond
class VVVV
op-types X I X X
set-type $2
eval AssignTarget($$, $1 ? CopyVal($2) : CopyVal($3))
expr-op Cond
class VVVC
op-types X I X X
set-type $2
eval AssignTarget($$, $1 ? CopyVal($2) : CopyVal($3))
expr-op Cond
class VVCV
op-types X I X X
set-type $2
eval AssignTarget($$, $1 ? CopyVal($2) : CopyVal($3))
op Bool-Vec-Cond
class VVVV
op-types V V V V
set-type $2
eval auto& vsel = $1->RawVec();
auto& v1 = $2->RawVec();
auto& v2 = $3->RawVec();
auto n = v1.size();
auto res = new vector<std::optional<ZVal>>(n);
for ( auto i = 0U; i < n; ++i )
if ( vsel[i] )
(*res)[i] = vsel[i]->AsInt() ? v1[i] : v2[i];
auto& full_res = $$;
Unref(full_res);
full_res = new VectorVal(cast_intrusive<VectorType>(Z_TYPE), res);
# Our instruction format doesn't accommodate two constants, so for
# the singular case of a V ? C1 : C2 conditional, we split it into
# two operations, V ? C1 and !V ? C2.
op CondC1
class VVC
op-types X I X
set-type $$
eval if ( $1 )
AssignTarget($$, CopyVal($2))
op CondC2
class VVC
op-types X I X
set-type $$
eval if ( ! $1 )
AssignTarget($$, CopyVal($2))

55
src/script_opt/ZAM/OPs/rel-exprs.op Normal file
View file

@ -0,0 +1,55 @@
# Operations corresponding to relational expressions.
rel-expr-op LT
op-type I U D S T A
vector
eval $1 < $2
eval-type S Bstr_cmp($1->AsString(), $2->AsString()) < 0
eval-type T $1->IsSubsetOf(*$2) && $1->Size() < $2->Size()
eval-type A $1->AsAddr() < $2->AsAddr()
rel-expr-op LE
op-type I U D S T A
vector
eval $1 <= $2
eval-type S Bstr_cmp($1->AsString(), $2->AsString()) <= 0
eval-type T $1->IsSubsetOf(*$2)
eval-type A $1->AsAddr() < $2->AsAddr() || $1->AsAddr() == $2->AsAddr()
rel-expr-op EQ
op-type I U D S T A N F
vector
eval $1 == $2
eval-type S Bstr_cmp($1->AsString(), $2->AsString()) == 0
eval-type T $1->EqualTo(*$2)
eval-type A $1->AsAddr() == $2->AsAddr()
eval-type N $1->AsSubNet() == $2->AsSubNet()
eval-type F util::streq($1->Name(), $2->Name())
eval-mixed P S $1->MatchExactly($2->AsString())
rel-expr-op NE
op-type I U D S T A N F
vector
eval $1 != $2
eval-type S Bstr_cmp($1->AsString(), $2->AsString()) != 0
eval-type T ! $1->EqualTo(*$2)
eval-type A $1->AsAddr() != $2->AsAddr()
eval-type N $1->AsSubNet() != $2->AsSubNet()
eval-type F ! util::streq($1->Name(), $2->Name())
eval-mixed P S ! $1->MatchExactly($2->AsString())
# Note, canonicalization means that GE and GT shouldn't occur
# for Sets (type T).
rel-expr-op GE
op-type I U D S A
vector
eval $1 >= $2
eval-type S Bstr_cmp($1->AsString(), $2->AsString()) >= 0
eval-type A ! ($1->AsAddr() < $2->AsAddr())
rel-expr-op GT
op-type I U D S A
vector
eval $1 > $2
eval-type S Bstr_cmp($1->AsString(), $2->AsString()) > 0
eval-type A ! ($1->AsAddr() < $2->AsAddr()) && $1->AsAddr() != $2->AsAddr()

57
src/script_opt/ZAM/OPs/script-idioms.op Normal file
View file

@ -0,0 +1,57 @@
# Operations corresponding to scripting idioms / known script functions.
internal-op MinU
classes VVV VVC
op-types U U U
eval $$ = std::min($1, $2);
internal-op MinI
classes VVV VVC
op-types I I I
eval $$ = std::min($1, $2);
internal-op MinD
classes VVV VVC
op-types D D D
eval $$ = std::min($1, $2);
internal-op MaxU
classes VVV VVC
op-types U U U
eval $$ = std::max($1, $2);
internal-op MaxI
classes VVV VVC
op-types I I I
eval $$ = std::max($1, $2);
internal-op MaxD
classes VVV VVC
op-types D D D
eval $$ = std::max($1, $2);
internal-op Func-Id-String
class VV
op-types S R
eval auto id_rec = $1;
auto orig_h = DirectField(id_rec, 0).AsAddr()->AsAddr().AsString();
auto resp_h = DirectField(id_rec, 2).AsAddr()->AsAddr().AsString();
auto orig_p = static_cast<uint32_t>(DirectField(id_rec, 1).AsCount()) & ~PORT_SPACE_MASK;
auto resp_p = static_cast<uint32_t>(DirectField(id_rec, 3).AsCount()) & ~PORT_SPACE_MASK;
/* Maximum address size is for IPv6 with no compression. Each
* 8 16-bit hex elements plus 7 colons between them plus the two []'s
* = 8*4 + 7 + 2 = 41 characters.
*
* Maximum port size is 5.
*
* Two of these = 2*41 + 2*5 = 92.
* Other delimiters: two ':', one ' < ' for 5 more.
*
* TOTAL: 97 characters.
*
* We use considerably more for safety.
*/
char buf[128];
snprintf(buf, sizeof buf, "%s:%u > %s:%u", orig_h.c_str(), orig_p, resp_h.c_str(), resp_p);
Unref($$);
$$ = new StringVal(buf);

339
src/script_opt/ZAM/OPs/stmts.op Normal file
View file

@ -0,0 +1,339 @@
# Operations corresponding to statements, other than iterations.
macro EvalScheduleArgs(time, is_delta, build_args)
if ( ! run_state::terminating )
{
double dt = time;
if ( is_delta )
dt += run_state::network_time;
auto handler = EventHandlerPtr(Z_AUX_EVENT_HANDLER);
ValVec args;
build_args
auto timer = new ScheduleTimer(handler, std::move(args), dt);
timer_mgr->Add(timer);
}
macro EvalSchedule(time, is_delta)
EvalScheduleArgs(time, is_delta, Z_AUX->FillValVec(args, frame);)
op Schedule
class ViHL
op-types D I X X
op1-read
custom-method return CompileSchedule(n, nullptr, i, h, l);
eval EvalSchedule($1, $2)
op Schedule
class CiHL
op-types D I X X
op1-read
custom-method return CompileSchedule(nullptr, c, i, h, l);
eval EvalSchedule($1, $2)
internal-op Schedule0
classes ViH CiH
op-types D I X
op1-read
eval EvalScheduleArgs($1, $2,)
macro QueueEvent(eh, args)
if ( *eh )
event_mgr.Enqueue(eh, std::move(args));
op Event
class HL
op1-read
custom-method return CompileEvent(h, l);
eval ValVec args;
Z_AUX->FillValVec(args, frame);
QueueEvent(Z_AUX_EVENT_HANDLER, args);
internal-op Event0
class X
eval ValVec args(0);
QueueEvent(Z_AUX_EVENT_HANDLER, args);
internal-op Event1
class V
op1-read
eval ValVec args(1);
args[0] = $1.ToVal(Z_TYPE);
QueueEvent(Z_AUX_EVENT_HANDLER, args);
internal-op Event2
class VV
op1-read
eval ValVec args(2);
args[0] = $1.ToVal(Z_TYPE);
args[1] = $2.ToVal(Z_TYPE2);
QueueEvent(Z_AUX_EVENT_HANDLER, args);
internal-op Event3
class VVV
op1-read
eval ValVec args(3);
auto& aux = Z_AUX;
args[0] = $1.ToVal(Z_TYPE);
args[1] = $2.ToVal(Z_TYPE2);
args[2] = $3.ToVal(aux->elems[2].GetType());
QueueEvent(Z_AUX_EVENT_HANDLER, args);
internal-op Event4
class VVVV
op1-read
eval ValVec args(4);
auto& aux = Z_AUX;
args[0] = $1.ToVal(Z_TYPE);
args[1] = $2.ToVal(Z_TYPE2);
args[2] = $3.ToVal(aux->elems[2].GetType());
args[3] = $4.ToVal(aux->elems[3].GetType());
QueueEvent(Z_AUX_EVENT_HANDLER, args);
op Return
class X
eval EvalReturn(nullptr,)
macro EvalReturn(val, type)
ret_u = val;
type
DO_ZAM_PROFILE
pc = end_pc;
continue;
op Return
op1-read
classes V C
set-type $$
eval EvalReturn(&$$, SET_RET_TYPE(Z_TYPE))
op When-Return
class X
eval static auto any_val = ZVal();
EvalReturn(&any_val,);
# Branch on the value of v1 using switch table v2, with default branch to v3
macro EvalSwitchBody(index, branch, cases, postscript)
{
auto t = cases[index];
if ( t.find(v) == t.end() )
pc = branch;
else
pc = t[v];
postscript
DO_ZAM_PROFILE
continue;
}
internal-op SwitchI
op1-read
class Vii
op-types I I I
eval auto v = $1;
EvalSwitchBody($2, $3, int_cases,)
internal-op SwitchU
op1-read
class Vii
op-types U I I
eval auto v = $1;
EvalSwitchBody($2, $3, uint_cases,)
internal-op SwitchD
op1-read
class Vii
op-types D I I
eval auto v = $1;
EvalSwitchBody($2, $3, double_cases,)
internal-op SwitchS
op1-read
class Vii
op-types S I I
eval auto vs = $1->AsString()->Render();
std::string v(vs);
EvalSwitchBody($2, $3, str_cases,delete[] vs;)
internal-op SwitchA
op1-read
class Vii
op-types A I I
eval auto v = $1->AsAddr().AsString();
EvalSwitchBody($2, $3, str_cases,)
internal-op SwitchN
op1-read
class Vii
op-types N I I
eval auto v = $1->AsSubNet().AsString();
EvalSwitchBody($2, $3, str_cases,)
internal-op Determine-Type-Match
class VV
op-types I a
eval auto& aux = Z_AUX;
int match = -1;
for ( int i = 0; i < aux->n; ++i )
{
auto& el = aux->elems[i];
auto& et = el.GetType();
if ( can_cast_value_to_type($1, et.get()) )
{
match = i;
if ( el.Slot() >= 0 )
{
auto& tv = frame[el.Slot()];
if ( el.IsManaged() )
Unref(tv.ManagedVal());
tv = ZVal(cast_value_to_type($1, et.get()), et);
}
break;
}
}
$$ = match;
op CheckAnyLen
op1-read
class Vi
op-types L U
eval auto v = $1;
if ( v->Vals().size() != $2 )
ERROR("mismatch in list lengths");
op Print
class O
eval do_print_stmt(Z_AUX->ToValVec(frame));
method-post z.aux = v->aux;
op Print1
op1-read
classes V C
set-type $$
eval std::vector<ValPtr> vals;
vals.push_back($$.ToVal(Z_TYPE));
do_print_stmt(vals);
internal-op If-Else
op1-read
class Vb
op-types I I
eval if ( ! $1 ) $2
internal-op If
op1-read
class Vb
op-types I I
eval if ( ! $1 ) $2
internal-op If-Not
op1-read
class Vb
op-types I I
eval if ( $1 ) $2
op AddStmt
op1-read
class VO
eval auto indices = Z_AUX->ToListVal(frame);
EvalAddStmt($1, indices)
method-post z.aux = v->aux;
macro EvalAddStmt(lhs, ind)
auto index = ind;
bool iterators_invalidated = false;
lhs.AsTable()->Assign(std::move(index), nullptr, true, &iterators_invalidated);
if ( iterators_invalidated )
WARN("possible loop/iterator invalidation");
op AddStmt1
op1-read
set-type $1
classes VV VC
eval EvalAddStmt($1, $2.ToVal(Z_TYPE))
op ClearTable
op1-read
class V
op-types T
eval $1->RemoveAll();
op ClearVector
op1-read
class V
op-types V
eval $1->Resize(0);
op DelTable
op1-read
class VO
op-types T X
eval auto indices = Z_AUX->ToListVal(frame);
bool iterators_invalidated = false;
$1->Remove(*indices, true, &iterators_invalidated);
if ( iterators_invalidated )
WARN("possible loop/iterator invalidation");
method-post z.aux = v->aux;
op DelField
op1-read
class Vi
op-types R I
eval $1->Remove($2);
internal-op Init-Record
class V
op-types R
eval auto r = new RecordVal(cast_intrusive<RecordType>(Z_TYPE));
Unref($$);
$$ = r;
internal-op Init-Vector
class V
op-types V
eval auto vt = cast_intrusive<VectorType>(Z_TYPE);
auto vec = new VectorVal(std::move(vt));
Unref($$);
$$ = vec;
internal-op Init-Table
class V
op-types T
eval auto tt = cast_intrusive<TableType>(Z_TYPE);
auto t = new TableVal(tt, Z_AUX_ATTRS);
Unref($$);
$$ = t;
op When
class V
op1-read
op-types F
eval BuildWhen($1, -1.0)
op When-Timeout
classes VV VC
op1-read
op-types F D
eval BuildWhen($1, $2)
macro BuildWhen(zf, timeout)
auto& aux = Z_AUX;
auto wi = Z_AUX_WHEN_INFO;
FuncPtr func{NewRef{}, zf};
auto lambda = make_intrusive<FuncVal>(func);
wi->Instantiate(std::move(lambda));
std::vector<ValPtr> local_aggrs;
for ( int i = 0; i < aux->n; ++i )
{
auto v = aux->ToVal(frame, i);
if ( v )
local_aggrs.push_back(v);
}
(void)make_intrusive<trigger::Trigger>(wi, wi->WhenExprGlobals(), local_aggrs, timeout, Z_FRAME, Z_LOC->Loc());

181
src/script_opt/ZAM/OPs/unary-exprs.op Normal file
View file

@ -0,0 +1,181 @@
# Operations corresponding to unary expressions.
# Direct assignment of an existing value.
assign-op Assign
class V
# The same, but where the assignment target (LHS) is a record field.
assign-op Field-LHS-Assign
op1-read
class F
unary-expr-op Clone
no-const
op-type X
set-type $$
set-type2 $1
eval auto v = $1.ToVal(Z_TYPE2)->Clone();
AssignTarget($$, BuildVal(v, Z_TYPE))
unary-expr-op Size
no-const
op-type I U D A N S T V *
explicit-result-type
set-type $$
set-type2 $1
eval-type I $$ = ZVal(zeek_int_t($1 < 0 ? -$1 : $1));
eval-type U $$ = ZVal($1);
eval-type D $$ = ZVal($1 < 0 ? -$1 : $1);
eval-type A $$ = ZVal(zeek_uint_t($1->AsAddr().GetFamily() == IPv4 ? 32 : 128));
eval-type N $$ = ZVal(pow(2.0, double(128 - $1->AsSubNet().LengthIPv6())));
eval-type S $$ = ZVal(zeek_uint_t($1->Len()));
eval-type T $$ = ZVal(zeek_uint_t($1->Size()));
eval-type V $$ = ZVal(zeek_uint_t($1->Size()));
eval auto v = $1.ToVal(Z_TYPE2)->SizeVal();
$$ = BuildVal(v, Z_TYPE);
unary-expr-op Not
op-type I
eval ! $1
unary-expr-op Complement
op-type U
eval ~ $1
unary-expr-op Positive
op-type I U D
vector
eval $1
unary-expr-op Negate
op-type I U D
vector
eval -$1
op IncrI
op1-read-write
class V
op-types I
eval ++$$;
op IncrU
op1-read-write
class V
op-types U
eval ++$$;
op DecrI
op1-read-write
class V
op-types I
eval --$$;
op DecrU
op1-read-write
class V
op-types U
eval auto& u = $$;
if ( u == 0 )
WARN("count underflow");
--u;
unary-op AppendTo
# Note, even though it feels like appending both reads and modifies
# its first operand, for our purposes it just reads it (to get the
# aggregate), and then modifies its *content* but not the operand's
# value itself.
op1-read
set-type $1
eval auto vv = $1.AsVector();
if ( vv->Size() == 0 )
/* Use the slightly more expensive Assign(), since it
* knows how to deal with empty vectors that do not yet
* have concrete types.
*/
vv->Assign(0, $2.ToVal(Z_TYPE));
else
{
vv->RawVec().push_back(CopyVal($2));
vv->Modified();
}
# For vectors-of-any, we always go through the Assign() interface because
# it's needed for tracking the potentially differing types.
unary-op AppendToAnyVec
op1-read
set-type $1
eval auto vv = $1.AsVector();
vv->Assign(vv->Size(), $2.ToVal(Z_TYPE));
internal-op AddPatternToField
classes VVi VCi
op1-read
op-types R P I
eval auto r = $$;
auto fpat = r->GetField($2)->AsPatternVal();
if ( fpat )
{
$1->AddTo(fpat, false);
r->Modified();
}
else
ERROR(util::fmt("field value missing: $%s", r->GetType()->AsRecordType()->FieldName($2)));
unary-op ExtendPattern
op1-read
eval $1.AsPattern()->AddTo($$.AsPattern(), false);
unary-op AddVecToVec
op1-read
eval if ( ! $1.AsVector()->AddTo($$.AsVector(), false) )
ERROR("incompatible vector element assignment");
unary-op AddTableToTable
op1-read
eval auto t = $$.AsTable();
auto v = $1.AsTable();
if ( v->Size() > 0 )
{
v->AddTo(t, false);
t->Modified();
}
unary-op RemoveTableFromTable
op1-read
eval auto t = $$.AsTable();
auto v = $1.AsTable();
if ( v->Size() > 0 )
{
v->RemoveFrom(t);
t->Modified();
}
unary-expr-op Cast
op-type X
set-type $$
set-type2 $1
eval EvalCast($$, $1.ToVal(Z_TYPE2))
macro EvalCast(lhs, rhs)
std::string error;
auto res = cast_value(rhs, Z_TYPE, error);
if ( res )
AssignTarget(lhs, BuildVal(res, Z_TYPE))
else
ERROR(error.c_str());
# Cast an "any" type to the given type. Only needed for type-based switch
# statements.
internal-op Cast-Any
class VV
op-types X a
eval ValPtr rhs = {NewRef{}, $1};
EvalCast($$, rhs)
direct-unary-op Is Is
internal-op Is
class VV
op-types I X
eval auto rhs = $1.ToVal(Z_TYPE2).get();
$$ = can_cast_value_to_type(rhs, Z_TYPE.get());

1
src/script_opt/ZAM/README.md
View file

@ -100,6 +100,7 @@ issues:
|`profile-ZAM` | Generate to "zprof.out" a ZAM execution profile. (Requires configuring with `--enable-ZAM-profiling` or `--enable-debug`.)|
|`report-recursive` | Report on recursive functions and exit.|
|`report-uncompilable` | Report on uncompilable functions and exit. For ZAM, all functions should be compilable.|
|`validate-ZAM` | Perform internal validation of ZAM instructions and exit.|
|`xform` | Transform scripts to "reduced" form.|
<a name="ZAM-profiling"></a>

382
src/script_opt/ZAM/Stmt.cc
View file

@ -132,7 +132,7 @@ const ZAMStmt ZAMCompiler::IfElse(const Expr* e, const Stmt* s1, const Stmt* s2)
if ( e->Tag() == EXPR_NAME ) {
auto n = e->AsNameExpr();
ZOp op = (s1 && s2) ? OP_IF_ELSE_VV : (s1 ? OP_IF_VV : OP_IF_NOT_VV);
ZOp op = (s1 && s2) ? OP_IF_ELSE_Vb : (s1 ? OP_IF_Vb : OP_IF_NOT_Vb);
ZInstI cond(op, FrameSlot(n), 0);
cond_stmt = AddInst(cond);
@ -141,13 +141,21 @@ const ZAMStmt ZAMCompiler::IfElse(const Expr* e, const Stmt* s1, const Stmt* s2)
else
cond_stmt = GenCond(e, branch_v);
AddCFT(insts1.back(), CFT_IF);
if ( s1 ) {
auto s1_end = CompileStmt(s1);
AddCFT(insts1.back(), CFT_BLOCK_END);
if ( s2 ) {
auto branch_after_s1 = GoToStub();
auto else_start = insts1.size();
auto s2_end = CompileStmt(s2);
SetV(cond_stmt, GoToTargetBeyond(branch_after_s1), branch_v);
SetGoTo(branch_after_s1, GoToTargetBeyond(s2_end));
AddCFT(insts1[else_start], CFT_ELSE);
AddCFT(insts1.back(), CFT_BLOCK_END);
return s2_end;
}
@ -160,66 +168,67 @@ const ZAMStmt ZAMCompiler::IfElse(const Expr* e, const Stmt* s1, const Stmt* s2)
// Only the else clause is non-empty.
auto s2_end = CompileStmt(s2);
AddCFT(insts1.back(), CFT_BLOCK_END);
// For complex conditionals, we need to invert their sense since
// we're switching to "if ( ! cond ) s2".
auto z = insts1[cond_stmt.stmt_num];
switch ( z->op ) {
case OP_IF_ELSE_VV:
case OP_IF_VV:
case OP_IF_NOT_VV:
case OP_IF_ELSE_Vb:
case OP_IF_Vb:
case OP_IF_NOT_Vb:
// These are generated correctly above, no need
// to fix up.
break;
case OP_HAS_FIELD_COND_VVV: z->op = OP_NOT_HAS_FIELD_COND_VVV; break;
case OP_NOT_HAS_FIELD_COND_VVV: z->op = OP_HAS_FIELD_COND_VVV; break;
case OP_HAS_FIELD_COND_Vib: z->op = OP_NOT_HAS_FIELD_COND_Vib; break;
case OP_NOT_HAS_FIELD_COND_Vib: z->op = OP_HAS_FIELD_COND_Vib; break;
case OP_CONN_EXISTS_COND_VV: z->op = OP_NOT_CONN_EXISTS_COND_VV; break;
case OP_NOT_CONN_EXISTS_COND_VV: z->op = OP_CONN_EXISTS_COND_VV; break;
case OP_CONN_EXISTS_COND_Vb: z->op = OP_NOT_CONN_EXISTS_COND_Vb; break;
case OP_NOT_CONN_EXISTS_COND_Vb: z->op = OP_CONN_EXISTS_COND_Vb; break;
case OP_IS_ICMP_PORT_COND_VV: z->op = OP_NOT_IS_ICMP_PORT_COND_VV; break;
case OP_NOT_IS_ICMP_PORT_COND_VV: z->op = OP_IS_ICMP_PORT_COND_VV; break;
case OP_IS_ICMP_PORT_COND_Vb: z->op = OP_NOT_IS_ICMP_PORT_COND_Vb; break;
case OP_NOT_IS_ICMP_PORT_COND_Vb: z->op = OP_IS_ICMP_PORT_COND_Vb; break;
case OP_IS_TCP_PORT_COND_VV: z->op = OP_NOT_IS_TCP_PORT_COND_VV; break;
case OP_NOT_IS_TCP_PORT_COND_VV: z->op = OP_IS_TCP_PORT_COND_VV; break;
case OP_IS_TCP_PORT_COND_Vb: z->op = OP_NOT_IS_TCP_PORT_COND_Vb; break;
case OP_NOT_IS_TCP_PORT_COND_Vb: z->op = OP_IS_TCP_PORT_COND_Vb; break;
case OP_IS_UDP_PORT_COND_VV: z->op = OP_NOT_IS_UDP_PORT_COND_VV; break;
case OP_NOT_IS_UDP_PORT_COND_VV: z->op = OP_IS_UDP_PORT_COND_VV; break;
case OP_IS_UDP_PORT_COND_Vb: z->op = OP_NOT_IS_UDP_PORT_COND_Vb; break;
case OP_NOT_IS_UDP_PORT_COND_Vb: z->op = OP_IS_UDP_PORT_COND_Vb; break;
case OP_IS_V4_ADDR_COND_VV: z->op = OP_NOT_IS_V4_ADDR_COND_VV; break;
case OP_NOT_IS_V4_ADDR_COND_VV: z->op = OP_IS_V4_ADDR_COND_VV; break;
case OP_IS_V4_ADDR_COND_Vb: z->op = OP_NOT_IS_V4_ADDR_COND_Vb; break;
case OP_NOT_IS_V4_ADDR_COND_Vb: z->op = OP_IS_V4_ADDR_COND_Vb; break;
case OP_IS_V6_ADDR_COND_VV: z->op = OP_NOT_IS_V6_ADDR_COND_VV; break;
case OP_NOT_IS_V6_ADDR_COND_VV: z->op = OP_IS_V6_ADDR_COND_VV; break;
case OP_IS_V6_ADDR_COND_Vb: z->op = OP_NOT_IS_V6_ADDR_COND_Vb; break;
case OP_NOT_IS_V6_ADDR_COND_Vb: z->op = OP_IS_V6_ADDR_COND_Vb; break;
case OP_READING_LIVE_TRAFFIC_COND_V: z->op = OP_NOT_READING_LIVE_TRAFFIC_COND_V; break;
case OP_NOT_READING_LIVE_TRAFFIC_COND_V: z->op = OP_READING_LIVE_TRAFFIC_COND_V; break;
case OP_READING_LIVE_TRAFFIC_COND_b: z->op = OP_NOT_READING_LIVE_TRAFFIC_COND_b; break;
case OP_NOT_READING_LIVE_TRAFFIC_COND_b: z->op = OP_READING_LIVE_TRAFFIC_COND_b; break;
case OP_READING_TRACES_COND_V: z->op = OP_NOT_READING_TRACES_COND_V; break;
case OP_NOT_READING_TRACES_COND_V: z->op = OP_READING_TRACES_COND_V; break;
case OP_READING_TRACES_COND_b: z->op = OP_NOT_READING_TRACES_COND_b; break;
case OP_NOT_READING_TRACES_COND_b: z->op = OP_READING_TRACES_COND_b; break;
case OP_TABLE_HAS_ELEMENTS_COND_VV: z->op = OP_NOT_TABLE_HAS_ELEMENTS_COND_VV; break;
case OP_NOT_TABLE_HAS_ELEMENTS_COND_VV: z->op = OP_TABLE_HAS_ELEMENTS_COND_VV; break;
case OP_TABLE_HAS_ELEMENTS_COND_Vb: z->op = OP_NOT_TABLE_HAS_ELEMENTS_COND_Vb; break;
case OP_NOT_TABLE_HAS_ELEMENTS_COND_Vb: z->op = OP_TABLE_HAS_ELEMENTS_COND_Vb; break;
case OP_VECTOR_HAS_ELEMENTS_COND_VV: z->op = OP_NOT_VECTOR_HAS_ELEMENTS_COND_VV; break;
case OP_NOT_VECTOR_HAS_ELEMENTS_COND_VV: z->op = OP_VECTOR_HAS_ELEMENTS_COND_VV; break;
case OP_VECTOR_HAS_ELEMENTS_COND_Vb: z->op = OP_NOT_VECTOR_HAS_ELEMENTS_COND_Vb; break;
case OP_NOT_VECTOR_HAS_ELEMENTS_COND_Vb: z->op = OP_VECTOR_HAS_ELEMENTS_COND_Vb; break;
case OP_VAL_IS_IN_TABLE_COND_VVV: z->op = OP_VAL_IS_NOT_IN_TABLE_COND_VVV; break;
case OP_VAL_IS_NOT_IN_TABLE_COND_VVV: z->op = OP_VAL_IS_IN_TABLE_COND_VVV; break;
case OP_VAL_IS_IN_TABLE_COND_VVb: z->op = OP_NOT_VAL_IS_IN_TABLE_COND_VVb; break;
case OP_NOT_VAL_IS_IN_TABLE_COND_VVb: z->op = OP_VAL_IS_IN_TABLE_COND_VVb; break;
case OP_CONST_IS_IN_TABLE_COND_VVC: z->op = OP_CONST_IS_NOT_IN_TABLE_COND_VVC; break;
case OP_CONST_IS_NOT_IN_TABLE_COND_VVC: z->op = OP_CONST_IS_IN_TABLE_COND_VVC; break;
case OP_CONST_IS_IN_TABLE_COND_VCb: z->op = OP_NOT_CONST_IS_IN_TABLE_COND_VCb; break;
case OP_NOT_CONST_IS_IN_TABLE_COND_VCb: z->op = OP_CONST_IS_IN_TABLE_COND_VCb; break;
case OP_VAL2_IS_IN_TABLE_COND_VVVV: z->op = OP_VAL2_IS_NOT_IN_TABLE_COND_VVVV; break;
case OP_VAL2_IS_NOT_IN_TABLE_COND_VVVV: z->op = OP_VAL2_IS_IN_TABLE_COND_VVVV; break;
case OP_VAL2_IS_IN_TABLE_COND_VVVb: z->op = OP_VAL2_IS_NOT_IN_TABLE_COND_VVVb; break;
case OP_VAL2_IS_NOT_IN_TABLE_COND_VVVb: z->op = OP_VAL2_IS_IN_TABLE_COND_VVVb; break;
case OP_VAL2_IS_IN_TABLE_COND_VVVC: z->op = OP_VAL2_IS_NOT_IN_TABLE_COND_VVVC; break;
case OP_VAL2_IS_NOT_IN_TABLE_COND_VVVC: z->op = OP_VAL2_IS_IN_TABLE_COND_VVVC; break;
case OP_VAL2_IS_IN_TABLE_COND_VVbC: z->op = OP_VAL2_IS_NOT_IN_TABLE_COND_VVbC; break;
case OP_VAL2_IS_NOT_IN_TABLE_COND_VVbC: z->op = OP_VAL2_IS_IN_TABLE_COND_VVbC; break;
case OP_VAL2_IS_IN_TABLE_COND_VVCV: z->op = OP_VAL2_IS_NOT_IN_TABLE_COND_VVCV; break;
case OP_VAL2_IS_NOT_IN_TABLE_COND_VVCV: z->op = OP_VAL2_IS_IN_TABLE_COND_VVCV; break;
case OP_VAL2_IS_IN_TABLE_COND_VVCb: z->op = OP_VAL2_IS_NOT_IN_TABLE_COND_VVCb; break;
case OP_VAL2_IS_NOT_IN_TABLE_COND_VVCb: z->op = OP_VAL2_IS_IN_TABLE_COND_VVCb; break;
default: reporter->InternalError("inconsistency in ZAMCompiler::IfElse");
}
@ -234,7 +243,7 @@ const ZAMStmt ZAMCompiler::GenCond(const Expr* e, int& branch_v) {
if ( e->Tag() == EXPR_HAS_FIELD ) {
auto hf = e->AsHasFieldExpr();
auto z = GenInst(OP_HAS_FIELD_COND_VVV, op1->AsNameExpr(), hf->Field());
auto z = GenInst(OP_HAS_FIELD_COND_Vib, op1->AsNameExpr(), hf->Field());
z.op_type = OP_VVV_I2_I3;
branch_v = 3;
return AddInst(z);
@ -251,15 +260,15 @@ const ZAMStmt ZAMCompiler::GenCond(const Expr* e, int& branch_v) {
}
if ( op1->Tag() == EXPR_NAME ) {
auto z = GenInst(OP_VAL_IS_IN_TABLE_COND_VVV, op1->AsNameExpr(), op2, 0);
z.t = op1->GetType();
auto z = GenInst(OP_VAL_IS_IN_TABLE_COND_VVb, op1->AsNameExpr(), op2, 0);
z.SetType(op1->GetType());
branch_v = 3;
return AddInst(z);
}
if ( op1->Tag() == EXPR_CONST ) {
auto z = GenInst(OP_CONST_IS_IN_TABLE_COND_VVC, op2, op1->AsConstExpr(), 0);
z.t = op1->GetType();
auto z = GenInst(OP_CONST_IS_IN_TABLE_COND_VCb, op2, op1->AsConstExpr(), 0);
z.SetType(op1->GetType());
branch_v = 2;
return AddInst(z);
}
@ -286,30 +295,30 @@ const ZAMStmt ZAMCompiler::GenCond(const Expr* e, int& branch_v) {
ZInstI z;
if ( name0 && name1 ) {
z = GenInst(OP_VAL2_IS_IN_TABLE_COND_VVVV, n0, n1, op2, 0);
z = GenInst(OP_VAL2_IS_IN_TABLE_COND_VVVb, n0, n1, op2, 0);
branch_v = 4;
z.t2 = n0->GetType();
z.SetType2(n0->GetType());
}
else if ( name0 ) {
z = GenInst(OP_VAL2_IS_IN_TABLE_COND_VVVC, n0, op2, c1, 0);
z = GenInst(OP_VAL2_IS_IN_TABLE_COND_VVbC, n0, op2, c1, 0);
branch_v = 3;
z.t2 = n0->GetType();
z.SetType2(n0->GetType());
}
else if ( name1 ) {
z = GenInst(OP_VAL2_IS_IN_TABLE_COND_VVCV, n1, op2, c0, 0);
z = GenInst(OP_VAL2_IS_IN_TABLE_COND_VVCb, n1, op2, c0, 0);
branch_v = 3;
z.t2 = n1->GetType();
z.SetType2(n1->GetType());
}
else { // Both are constants, assign first to temporary.
auto slot = TempForConst(c0);
z = ZInstI(OP_VAL2_IS_IN_TABLE_COND_VVVC, slot, FrameSlot(op2), 0, c1);
z = ZInstI(OP_VAL2_IS_IN_TABLE_COND_VVbC, slot, FrameSlot(op2), 0, c1);
z.op_type = OP_VVVC_I3;
branch_v = 3;
z.t2 = c0->GetType();
z.SetType2(c0->GetType());
}
return AddInst(z);
@ -328,9 +337,9 @@ const ZAMStmt ZAMCompiler::GenCond(const Expr* e, int& branch_v) {
ZOp op;
if ( aggr->GetType()->Tag() == TYPE_TABLE )
op = OP_TABLE_HAS_ELEMENTS_COND_VV;
op = OP_TABLE_HAS_ELEMENTS_COND_Vb;
else
op = OP_VECTOR_HAS_ELEMENTS_COND_VV;
op = OP_VECTOR_HAS_ELEMENTS_COND_Vb;
branch_v = 2;
return AddInst(GenInst(op, aggr, +0));
@ -409,45 +418,32 @@ const ZAMStmt ZAMCompiler::ValueSwitch(const SwitchStmt* sw, const NameExpr* v,
// Figure out which jump table we're using.
auto t = v ? v->GetType() : c->GetType();
int tbl = 0;
return GenSwitch(sw, slot, t->InternalType());
}
const ZAMStmt ZAMCompiler::GenSwitch(const SwitchStmt* sw, int slot, InternalTypeTag it) {
ZOp op;
switch ( t->InternalType() ) {
case TYPE_INTERNAL_INT:
op = OP_SWITCHI_VVV;
tbl = int_casesI.size();
break;
switch ( it ) {
case TYPE_INTERNAL_INT: op = OP_SWITCHI_Vii; break;
case TYPE_INTERNAL_UNSIGNED:
op = OP_SWITCHU_VVV;
tbl = uint_casesI.size();
break;
case TYPE_INTERNAL_UNSIGNED: op = OP_SWITCHU_Vii; break;
case TYPE_INTERNAL_DOUBLE:
op = OP_SWITCHD_VVV;
tbl = double_casesI.size();
break;
case TYPE_INTERNAL_DOUBLE: op = OP_SWITCHD_Vii; break;
case TYPE_INTERNAL_STRING:
op = OP_SWITCHS_VVV;
tbl = str_casesI.size();
break;
case TYPE_INTERNAL_STRING: op = OP_SWITCHS_Vii; break;
case TYPE_INTERNAL_ADDR:
op = OP_SWITCHA_VVV;
tbl = str_casesI.size();
break;
case TYPE_INTERNAL_ADDR: op = OP_SWITCHA_Vii; break;
case TYPE_INTERNAL_SUBNET:
op = OP_SWITCHN_VVV;
tbl = str_casesI.size();
break;
case TYPE_INTERNAL_SUBNET: op = OP_SWITCHN_Vii; break;
default: reporter->InternalError("bad switch type");
}
// Add the "head", i.e., the execution of the jump table.
auto sw_head_op = ZInstI(op, slot, tbl, 0);
// Add the "head", i.e., the execution of the jump table. At this point,
// we leave the table (v2) and default (v3) TBD.
auto sw_head_op = ZInstI(op, slot, 0, 0);
sw_head_op.op_type = OP_VVV_I2_I3;
auto sw_head = AddInst(sw_head_op);
@ -456,6 +452,7 @@ const ZAMStmt ZAMCompiler::ValueSwitch(const SwitchStmt* sw, const NameExpr* v,
// Generate each of the cases.
auto cases = sw->Cases();
std::vector<InstLabel> case_start;
int case_index = 0;
PushFallThroughs();
for ( auto sw_case : *cases ) {
@ -471,8 +468,11 @@ const ZAMStmt ZAMCompiler::ValueSwitch(const SwitchStmt* sw, const NameExpr* v,
ResolveBreaks(sw_end);
int def_ind = sw->DefaultCaseIndex();
if ( def_ind >= 0 )
SetV3(sw_head, case_start[def_ind]);
if ( def_ind >= 0 ) {
auto def = case_start[def_ind];
SetV3(sw_head, def);
AddCFT(def, CFT_DEFAULT);
}
else
SetV3(sw_head, sw_end);
@ -520,103 +520,93 @@ const ZAMStmt ZAMCompiler::ValueSwitch(const SwitchStmt* sw, const NameExpr* v,
}
}
// For type switches, we map them to consecutive numbers, and then use
// a integer-valued switch on those.
int tm_ctr = 0;
for ( auto [_, index] : *sw->TypeMap() ) {
auto case_body_start = case_start[index];
new_int_cases[tm_ctr++] = case_body_start;
}
// Now add the jump table to the set we're keeping for the
// corresponding type.
switch ( t->InternalType() ) {
case TYPE_INTERNAL_INT: int_casesI.push_back(new_int_cases); break;
size_t tbl;
case TYPE_INTERNAL_UNSIGNED: uint_casesI.push_back(new_uint_cases); break;
switch ( it ) {
case TYPE_INTERNAL_INT:
tbl = int_casesI.size();
int_casesI.push_back(new_int_cases);
break;
case TYPE_INTERNAL_DOUBLE: double_casesI.push_back(new_double_cases); break;
case TYPE_INTERNAL_UNSIGNED:
tbl = uint_casesI.size();
uint_casesI.push_back(new_uint_cases);
break;
case TYPE_INTERNAL_DOUBLE:
tbl = double_casesI.size();
double_casesI.push_back(new_double_cases);
break;
case TYPE_INTERNAL_STRING:
case TYPE_INTERNAL_ADDR:
case TYPE_INTERNAL_SUBNET: str_casesI.push_back(new_str_cases); break;
case TYPE_INTERNAL_SUBNET:
tbl = str_casesI.size();
str_casesI.push_back(new_str_cases);
break;
default: reporter->InternalError("bad switch type");
}
insts1[sw_head.stmt_num]->v2 = int(tbl);
AddCFT(insts1[body_end.stmt_num], CFT_BLOCK_END);
return body_end;
}
const ZAMStmt ZAMCompiler::TypeSwitch(const SwitchStmt* sw, const NameExpr* v, const ConstExpr* c) {
auto cases = sw->Cases();
auto type_map = sw->TypeMap();
auto body_end = EmptyStmt();
auto tmp = NewSlot(true); // true since we know "any" is managed
int slot = v ? FrameSlot(v) : 0;
if ( v && v->GetType()->Tag() != TYPE_ANY ) {
auto z = ZInstI(OP_ASSIGN_ANY_VV, tmp, slot);
body_end = AddInst(z);
slot = tmp;
if ( v ) {
if ( v->GetType()->Tag() != TYPE_ANY ) {
auto z = ZInstI(OP_ASSIGN_ANY_VV, tmp, slot);
AddInst(z);
slot = tmp;
}
}
if ( c ) {
else {
ASSERT(c);
auto z = ZInstI(OP_ASSIGN_ANY_VC, tmp, c);
body_end = AddInst(z);
AddInst(z);
slot = tmp;
}
int def_ind = sw->DefaultCaseIndex();
ZAMStmt def_succ(0); // successor to default, if any
bool saw_def_succ = false; // whether def_succ is meaningful
int ntypes = type_map->size();
auto aux = new ZInstAux(ntypes);
PushFallThroughs();
for ( auto& i : *type_map ) {
auto id = i.first;
auto type = id->GetType();
for ( size_t i = 0; i < type_map->size(); ++i ) {
auto& tm = (*type_map)[i];
auto id_i = tm.first;
auto id_case = tm.second;
ZInstI z;
z = ZInstI(OP_BRANCH_IF_NOT_TYPE_VV, slot, 0);
z.SetType(type);
auto case_test = AddInst(z);
// Type cases that don't use "as" create a placeholder
// ID with a null name.
if ( id->Name() ) {
int id_slot = Frame1Slot(id, OP_CAST_ANY_VV);
z = ZInstI(OP_CAST_ANY_VV, id_slot, slot);
z.SetType(type);
body_end = AddInst(z);
}
else
body_end = case_test;
ResolveFallThroughs(GoToTargetBeyond(body_end));
body_end = CompileStmt((*cases)[i.second]->Body());
SetV2(case_test, GoToTargetBeyond(body_end));
if ( def_ind >= 0 && i.second == def_ind + 1 ) {
def_succ = case_test;
saw_def_succ = true;
}
PushFallThroughs();
auto slot = id_i->Name() ? FrameSlot(id_i) : -1;
aux->Add(i, slot, id_i->GetType());
}
ResolveFallThroughs(GoToTargetBeyond(body_end));
auto match_tmp = NewSlot(false);
auto z = ZInstI(OP_DETERMINE_TYPE_MATCH_VV, match_tmp, slot);
z.op_type = OP_VV;
z.aux = aux;
AddInst(z);
if ( def_ind >= 0 ) {
PushFallThroughs();
body_end = CompileStmt((*sw->Cases())[def_ind]->Body());
// Now resolve any fallthrough's in the default.
if ( saw_def_succ )
ResolveFallThroughs(GoToTargetBeyond(def_succ));
else
ResolveFallThroughs(GoToTargetBeyond(body_end));
}
ResolveBreaks(GoToTargetBeyond(body_end));
return body_end;
return GenSwitch(sw, match_tmp, TYPE_INTERNAL_INT);
}
const ZAMStmt ZAMCompiler::CompileWhile(const WhileStmt* ws) {
@ -645,18 +635,23 @@ const ZAMStmt ZAMCompiler::While(const Stmt* cond_stmt, const Expr* cond, const
if ( cond->Tag() == EXPR_NAME ) {
auto n = cond->AsNameExpr();
cond_IF = AddInst(ZInstI(OP_IF_VV, FrameSlot(n), 0));
cond_IF = AddInst(ZInstI(OP_IF_Vb, FrameSlot(n), 0));
branch_v = 2;
}
else
cond_IF = GenCond(cond, branch_v);
AddCFT(insts1[head.stmt_num], CFT_LOOP);
AddCFT(insts1[cond_IF.stmt_num], CFT_LOOP_COND);
PushNexts();
PushBreaks();
if ( body && body->Tag() != STMT_NULL )
(void)CompileStmt(body);
AddCFT(insts1.back(), CFT_BLOCK_END);
auto tail = GoTo(GoToTarget(head));
auto beyond_tail = GoToTargetBeyond(tail);
@ -676,17 +671,21 @@ const ZAMStmt ZAMCompiler::CompileFor(const ForStmt* f) {
PushNexts();
PushBreaks();
ZAMStmt z;
if ( et == TYPE_TABLE )
return LoopOverTable(f, val);
z = LoopOverTable(f, val);
else if ( et == TYPE_VECTOR )
return LoopOverVector(f, val);
z = LoopOverVector(f, val);
else if ( et == TYPE_STRING )
return LoopOverString(f, e);
z = LoopOverString(f, e);
else
reporter->InternalError("bad \"for\" loop-over value when compiling");
return z;
}
const ZAMStmt ZAMCompiler::LoopOverTable(const ForStmt* f, const NameExpr* val) {
@ -723,29 +722,39 @@ const ZAMStmt ZAMCompiler::LoopOverTable(const ForStmt* f, const NameExpr* val)
auto iter_slot = table_iters.size();
table_iters.emplace_back();
auto z = ZInstI(OP_INIT_TABLE_LOOP_VV, FrameSlot(val), iter_slot);
z.op_type = OP_VV_I2;
z.SetType(value_var ? value_var->GetType() : nullptr);
z.aux = aux;
auto zi = ZInstI(OP_INIT_TABLE_LOOP_Vf, FrameSlot(val), iter_slot);
zi.op_type = OP_VV_I2;
if ( value_var )
zi.SetType(value_var->GetType());
zi.aux = aux;
auto init_end = AddInst(z);
(void)AddInst(zi);
ZInstI zn;
auto iter_head = StartingBlock();
if ( value_var ) {
ZOp op = no_loop_vars ? OP_NEXT_TABLE_ITER_VAL_VAR_NO_VARS_VVV : OP_NEXT_TABLE_ITER_VAL_VAR_VVV;
z = ZInstI(op, FrameSlot(value_var), iter_slot, 0);
z.CheckIfManaged(value_var->GetType());
z.op_type = OP_VVV_I2_I3;
ZOp op = no_loop_vars ? OP_NEXT_TABLE_ITER_VAL_VAR_NO_VARS_Vfb : OP_NEXT_TABLE_ITER_VAL_VAR_Vfb;
zn = ZInstI(op, FrameSlot(value_var), iter_slot, 0);
zn.CheckIfManaged(value_var->GetType());
zn.op_type = OP_VVV_I2_I3;
}
else {
ZOp op = no_loop_vars ? OP_NEXT_TABLE_ITER_NO_VARS_VV : OP_NEXT_TABLE_ITER_VV;
z = ZInstI(op, iter_slot, 0);
z.op_type = OP_VV_I1_I2;
ZOp op = no_loop_vars ? OP_NEXT_TABLE_ITER_NO_VARS_fb : OP_NEXT_TABLE_ITER_fb;
zn = ZInstI(op, iter_slot, 0);
zn.op_type = OP_VV_I1_I2;
}
z.aux = aux; // so ZOpt.cc can get to it
// Need a separate instance of aux so the CFT info doesn't get shared with
// the loop init. We populate it with the loop_vars (only) because the
// optimizer needs access to those for (1) tracking their lifetime, and
// (2) remapping them (not strictly needed, see the comment in ReMapFrame()).
zn.aux = new ZInstAux(0);
zn.aux->loop_vars = aux->loop_vars;
AddCFT(&zn, CFT_LOOP);
AddCFT(&zn, CFT_LOOP_COND);
return FinishLoop(iter_head, z, body, iter_slot, true);
return FinishLoop(iter_head, zn, body, iter_slot, true);
}
const ZAMStmt ZAMCompiler::LoopOverVector(const ForStmt* f, const NameExpr* val) {
@ -755,7 +764,7 @@ const ZAMStmt ZAMCompiler::LoopOverVector(const ForStmt* f, const NameExpr* val)
int iter_slot = num_step_iters++;
auto z = ZInstI(OP_INIT_VECTOR_LOOP_VV, FrameSlot(val), iter_slot);
auto z = ZInstI(OP_INIT_VECTOR_LOOP_Vs, FrameSlot(val), iter_slot);
z.op_type = OP_VV_I2;
auto init_end = AddInst(z);
@ -765,29 +774,31 @@ const ZAMStmt ZAMCompiler::LoopOverVector(const ForStmt* f, const NameExpr* val)
if ( value_var ) {
if ( slot >= 0 ) {
z = ZInstI(OP_NEXT_VECTOR_ITER_VAL_VAR_VVVV, slot, FrameSlot(value_var), iter_slot, 0);
z = ZInstI(OP_NEXT_VECTOR_ITER_VAL_VAR_VVsb, slot, FrameSlot(value_var), iter_slot, 0);
z.op_type = OP_VVVV_I3_I4;
}
else {
z = ZInstI(OP_NEXT_VECTOR_BLANK_ITER_VAL_VAR_VVV, FrameSlot(value_var), iter_slot, 0);
z = ZInstI(OP_NEXT_VECTOR_BLANK_ITER_VAL_VAR_Vsb, FrameSlot(value_var), iter_slot, 0);
z.op_type = OP_VVV_I2_I3;
}
z.t = value_var->GetType();
z.is_managed = ZVal::IsManagedType(z.t);
z.SetType(value_var->GetType());
}
else {
if ( slot >= 0 ) {
z = ZInstI(OP_NEXT_VECTOR_ITER_VVV, slot, iter_slot, 0);
z = ZInstI(OP_NEXT_VECTOR_ITER_Vsb, slot, iter_slot, 0);
z.op_type = OP_VVV_I2_I3;
}
else {
z = ZInstI(OP_NEXT_VECTOR_BLANK_ITER_VV, iter_slot, 0);
z = ZInstI(OP_NEXT_VECTOR_BLANK_ITER_sb, iter_slot, 0);
z.op_type = OP_VV_I1_I2;
}
}
AddCFT(&z, CFT_LOOP);
AddCFT(&z, CFT_LOOP_COND);
return FinishLoop(iter_head, z, f->LoopBody(), iter_slot, false);
}
@ -802,12 +813,12 @@ const ZAMStmt ZAMCompiler::LoopOverString(const ForStmt* f, const Expr* e) {
ZInstI z;
if ( n ) {
z = ZInstI(OP_INIT_STRING_LOOP_VV, FrameSlot(n), iter_slot);
z = ZInstI(OP_INIT_STRING_LOOP_Vs, FrameSlot(n), iter_slot);
z.op_type = OP_VV_I2;
}
else {
ASSERT(c);
z = ZInstI(OP_INIT_STRING_LOOP_VC, iter_slot, c);
z = ZInstI(OP_INIT_STRING_LOOP_Cs, iter_slot, c);
z.op_type = OP_VC_I1;
}
@ -815,15 +826,18 @@ const ZAMStmt ZAMCompiler::LoopOverString(const ForStmt* f, const Expr* e) {
auto iter_head = StartingBlock();
if ( loop_var->IsBlank() ) {
z = ZInstI(OP_NEXT_STRING_BLANK_ITER_VV, iter_slot, 0);
z = ZInstI(OP_NEXT_STRING_BLANK_ITER_sb, iter_slot, 0);
z.op_type = OP_VV_I1_I2;
}
else {
z = ZInstI(OP_NEXT_STRING_ITER_VVV, FrameSlot(loop_var), iter_slot, 0);
z = ZInstI(OP_NEXT_STRING_ITER_Vsb, FrameSlot(loop_var), iter_slot, 0);
z.op_type = OP_VVV_I2_I3;
z.is_managed = true;
}
AddCFT(&z, CFT_LOOP);
AddCFT(&z, CFT_LOOP_COND);
return FinishLoop(iter_head, z, f->LoopBody(), iter_slot, false);
}
@ -832,7 +846,11 @@ const ZAMStmt ZAMCompiler::Loop(const Stmt* body) {
PushBreaks();
auto head = StartingBlock();
(void)CompileStmt(body);
auto b = CompileStmt(body);
AddCFT(insts1[head.stmt_num], CFT_LOOP);
AddCFT(insts1[b.stmt_num], CFT_BLOCK_END);
auto tail = GoTo(GoToTarget(head));
ResolveNexts(GoToTarget(head));
@ -845,11 +863,12 @@ const ZAMStmt ZAMCompiler::FinishLoop(const ZAMStmt iter_head, ZInstI& iter_stmt
bool is_table) {
auto loop_iter = AddInst(iter_stmt);
auto body_end = CompileStmt(body);
AddCFT(insts1[body_end.stmt_num], CFT_BLOCK_END);
// We only need cleanup for looping over tables, but for now we
// need some sort of placeholder instruction (until the optimizer
// can elide it) to resolve loop exits.
ZOp op = is_table ? OP_END_TABLE_LOOP_V : OP_NOP;
ZOp op = is_table ? OP_END_TABLE_LOOP_f : OP_NOP;
auto loop_end = GoTo(GoToTarget(iter_head));
auto z = ZInstI(op, iter_slot);
@ -875,6 +894,12 @@ const ZAMStmt ZAMCompiler::CompileReturn(const ReturnStmt* r) {
if ( retvars.empty() ) { // a "true" return
if ( e ) {
if ( pf->ProfiledFunc()->Flavor() == FUNC_FLAVOR_HOOK ) {
ASSERT(e->GetType()->Tag() == TYPE_BOOL);
auto true_c = make_intrusive<ConstExpr>(val_mgr->True());
return ReturnC(true_c.get());
}
if ( e->Tag() == EXPR_NAME )
return ReturnV(e->AsNameExpr());
else
@ -970,7 +995,7 @@ const ZAMStmt ZAMCompiler::CompileWhen(const WhenStmt* ws) {
auto timeout = wi->TimeoutExpr();
auto lambda = NewSlot(true);
(void)BuildLambda(lambda, wi->Lambda().get());
(void)BuildLambda(lambda, wi->Lambda());
std::vector<IDPtr> local_aggr_slots;
for ( auto& l : wi->WhenExprLocals() )
@ -1006,8 +1031,7 @@ const ZAMStmt ZAMCompiler::CompileWhen(const WhenStmt* ws) {
if ( ws->IsReturn() ) {
(void)AddInst(z);
z = ZInstI(OP_RETURN_C);
z.c = ZVal();
z = ZInstI(OP_WHEN_RETURN_X);
}
return AddInst(z);

38
src/script_opt/ZAM/Support.cc
View file

@ -16,7 +16,7 @@ namespace ZAM {
std::string curr_func;
std::shared_ptr<ZAMLocInfo> curr_loc;
TypePtr log_ID_enum_type;
TypePtr any_base_type;
TypePtr any_base_type = base_type(TYPE_ANY);
} // namespace ZAM
bool ZAM_error = false;
@ -35,6 +35,32 @@ bool is_ZAM_compilable(const ProfileFunc* pf, const char** reason) {
bool IsAny(const Type* t) { return t->Tag() == TYPE_ANY; }
bool CheckAnyType(const TypePtr& any_type, const TypePtr& expected_type, const std::shared_ptr<ZAMLocInfo>& loc) {
if ( IsAny(expected_type) )
return true;
if ( ! same_type(any_type, expected_type, false, false) ) {
auto at = any_type->Tag();
auto et = expected_type->Tag();
if ( at == TYPE_RECORD && et == TYPE_RECORD ) {
auto at_r = any_type->AsRecordType();
auto et_r = expected_type->AsRecordType();
if ( record_promotion_compatible(et_r, at_r) )
return true;
}
char buf[8192];
snprintf(buf, sizeof buf, "run-time type clash (%s/%s)", type_name(at), type_name(et));
reporter->RuntimeError(loc->Loc(), "%s", buf);
return false;
}
return true;
}
StringVal* ZAM_to_lower(const StringVal* sv) {
auto bs = sv->AsString();
const u_char* s = bs->Bytes();
@ -82,7 +108,10 @@ void ZAM_run_time_error(const char* msg) {
}
void ZAM_run_time_error(std::shared_ptr<ZAMLocInfo> loc, const char* msg) {
reporter->RuntimeError(loc->Loc(), "%s", msg);
if ( loc )
reporter->RuntimeError(loc->Loc(), "%s", msg);
else
fprintf(stderr, "<no location in optimized code>: %s\n", msg);
ZAM_error = true;
}
@ -92,7 +121,10 @@ void ZAM_run_time_error(const char* msg, const Obj* o) {
}
void ZAM_run_time_error(std::shared_ptr<ZAMLocInfo> loc, const char* msg, const Obj* o) {
reporter->RuntimeError(loc->Loc(), "%s (%s)", msg, obj_desc(o).c_str());
if ( loc )
reporter->RuntimeError(loc->Loc(), "%s (%s)", msg, obj_desc(o).c_str());
else
ZAM_run_time_error(msg, o);
ZAM_error = true;
}

4
src/script_opt/ZAM/Support.h
View file

@ -42,6 +42,10 @@ extern bool IsAny(const Type* t);
inline bool IsAny(const TypePtr& t) { return IsAny(t.get()); }
inline bool IsAny(const Expr* e) { return IsAny(e->GetType()); }
// Run-time checking for "any" type being consistent with
// expected typed. Returns true if the type match is okay.
extern bool CheckAnyType(const TypePtr& any_type, const TypePtr& expected_type, const std::shared_ptr<ZAMLocInfo>& loc);
extern void ZAM_run_time_error(const char* msg);
extern void ZAM_run_time_error(std::shared_ptr<ZAMLocInfo> loc, const char* msg);
extern void ZAM_run_time_error(std::shared_ptr<ZAMLocInfo> loc, const char* msg, const Obj* o);

110
src/script_opt/ZAM/Validate.cc Normal file
View file

@ -0,0 +1,110 @@
// See the file "COPYING" in the main distribution directory for copyright.
#include <string>
#include "zeek/script_opt/ZAM/ZBody.h"
#include "zeek/script_opt/ZAM/ZOp.h"
using std::string;
namespace zeek::detail {
std::unordered_map<ZOp, ZAMInstDesc> zam_inst_desc = {
#include "ZAM-Desc.h"
};
// While the following has commonalities that could be factored out,
// for now we keep this form because it provides flexibility for
// accommodating other forms of accessors.
static std::map<char, string> type_pats = {
{'A', "Addr"}, {'a', "Any"}, {'D', "Double"}, {'F', "Func"}, {'I', "Int"}, {'L', "List"}, {'N', "SubNet"},
{'P', "Pattern"}, {'R', "Record"}, {'S', "String"}, {'T', "Table"}, {'t', "Type"}, {'U', "Count"}, {'V', "Vector"},
};
static int num_valid = 0;
static int num_tested = 0;
static int num_skipped = 0;
void analyze_ZAM_inst(const char* op_name, const ZAMInstDesc& zid) {
auto& oc = zid.op_class;
auto& ot = zid.op_types;
auto& eval = zid.op_eval;
bool have_ot = ! ot.empty();
if ( have_ot && oc.size() != ot.size() )
reporter->InternalError("%s: instruction class/types mismatch (%s/%s)", op_name, oc.c_str(), ot.c_str());
int nslot = 0;
for ( size_t i = 0; i < oc.size(); ++i ) {
auto oc_i = oc[i];
string op;
switch ( oc_i ) {
case 'V':
case 'R': op = "frame\\[z\\.v" + std::to_string(++nslot) + "\\]"; break;
case 'b':
case 'f':
case 'g':
case 's':
case 'i': op = "z\\.v" + std::to_string(++nslot); break;
case 'C': op = "z\\.c"; break;
default:
if ( have_ot && ot[i] != 'X' )
reporter->InternalError("instruction types mismatch: %s (%c)", op_name, oc_i);
}
auto match_pat = op;
if ( have_ot ) {
auto ot_i = ot[i];
bool bare_int = std::string("bfgis").find(oc_i) != std::string::npos;
if ( ot_i == 'X' || bare_int ) {
if ( ot_i == 'X' && bare_int )
reporter->InternalError("empty instruction type for '%c' class element: %s", oc_i, op_name);
if ( ! std::regex_search(eval, std::regex(op)) )
reporter->InternalError("%s: operand %s not found", op_name, op.c_str());
++num_skipped;
continue;
}
auto tp = type_pats.find(ot_i);
if ( tp == type_pats.end() )
reporter->InternalError("%s: instruction type %c not found", op_name, ot_i);
match_pat += ".As" + tp->second + "(Ref)?\\(\\)";
++num_tested;
}
if ( ! std::regex_search(eval, std::regex(match_pat)) )
reporter->InternalError("%s: did not find /%s/ in %s", op_name, match_pat.c_str(), eval.c_str());
}
++num_valid;
}
void validate_ZAM_insts() {
// The following primes a data structure we access.
(void)AssignmentFlavor(OP_NOP, TYPE_VOID, false);
for ( int i = 0; i < int(OP_NOP); ++i ) {
auto zop = ZOp(i);
if ( zam_inst_desc.find(zop) == zam_inst_desc.end() && assignment_flavor.find(zop) == assignment_flavor.end() )
reporter->InternalError("op %s missing from description", ZOP_name(zop));
}
for ( auto& zid : zam_inst_desc )
analyze_ZAM_inst(ZOP_name(zid.first), zid.second);
printf("%d valid, %d tested, %d skipped\n", num_valid, num_tested, num_skipped);
}
} // namespace zeek::detail

11
src/script_opt/ZAM/Vars.cc
View file

@ -40,7 +40,7 @@ void ZAMCompiler::LoadParam(const ID* id) {
ZOp op;
op = AssignmentFlavor(OP_LOAD_VAL_VV, id->GetType()->Tag());
op = AssignmentFlavor(OP_LOAD_VAL_Vi, id->GetType()->Tag());
int slot = AddToFrame(id);
@ -57,9 +57,9 @@ const ZAMStmt ZAMCompiler::LoadGlobal(const ID* id) {
if ( id->IsType() )
// Need a special load for these, as they don't fit
// with the usual template.
op = OP_LOAD_GLOBAL_TYPE_VV;
op = OP_LOAD_GLOBAL_TYPE_Vg;
else
op = AssignmentFlavor(OP_LOAD_GLOBAL_VV, id->GetType()->Tag());
op = AssignmentFlavor(OP_LOAD_GLOBAL_Vg, id->GetType()->Tag());
auto slot = RawSlot(id);
@ -78,13 +78,14 @@ const ZAMStmt ZAMCompiler::LoadCapture(const ID* id) {
ZOp op;
if ( ZVal::IsManagedType(id->GetType()) )
op = OP_LOAD_MANAGED_CAPTURE_VV;
op = OP_LOAD_MANAGED_CAPTURE_Vi;
else
op = OP_LOAD_CAPTURE_VV;
op = OP_LOAD_CAPTURE_Vi;
auto slot = RawSlot(id);
ZInstI z(op, slot, CaptureOffset(id));
z.SetType(id->GetType());
z.op_type = OP_VV_I2;
return AddInst(z, true);

70
src/script_opt/ZAM/ZBody.cc
View file

@ -14,27 +14,6 @@
#include "zeek/script_opt/ZAM/Compile.h"
#include "zeek/session/Manager.h"
// Needed for managing the corresponding values.
#include "zeek/File.h"
#include "zeek/Func.h"
#include "zeek/OpaqueVal.h"
// Just needed for BiFs.
#include "zeek/analyzer/Manager.h"
#include "zeek/analyzer/protocol/conn-size/ConnSize.h"
#include "zeek/broker/Manager.h"
#include "zeek/file_analysis/Manager.h"
#include "zeek/file_analysis/file_analysis.bif.h"
#include "zeek/logging/Manager.h"
#include "zeek/packet_analysis/Manager.h"
#include "zeek/packet_analysis/protocol/gtpv1/GTPv1.h"
#include "zeek/packet_analysis/protocol/teredo/Teredo.h"
#include "zeek.bif.func_h"
// For reading_live and reading_traces
#include "zeek/RunState.h"
namespace zeek::detail {
static double CPU_prof_overhead = 0.0;
@ -165,7 +144,7 @@ void report_ZOP_profile() {
// assigned value was missing (which we can only tell for managed types),
// true otherwise.
static bool copy_vec_elem(VectorVal* vv, zeek_uint_t ind, ZVal zv, const TypePtr& t) {
bool copy_vec_elem(VectorVal* vv, zeek_uint_t ind, ZVal zv, const TypePtr& t) {
if ( vv->Size() <= ind )
vv->Resize(ind + 1);
@ -200,7 +179,7 @@ static void vec_exec(ZOp op, TypePtr t, VectorVal*& v1, const VectorVal* v2, con
// Vector coercion.
#define VEC_COERCE(tag, lhs_type, cast, rhs_accessor, ov_check, ov_err) \
static VectorVal* vec_coerce_##tag(VectorVal* vec, const ZInst& z) { \
VectorVal* vec_coerce_##tag(VectorVal* vec, std::shared_ptr<ZAMLocInfo> z_loc) { \
auto& v = vec->RawVec(); \
auto yt = make_intrusive<VectorType>(base_type(lhs_type)); \
auto res_zv = new VectorVal(yt); \
@ -214,7 +193,7 @@ static void vec_exec(ZOp op, TypePtr t, VectorVal*& v1, const VectorVal* v2, con
std::string err = "overflow promoting from "; \
err += ov_err; \
err += " arithmetic value"; \
ZAM_run_time_error(z.loc, err.c_str()); \
ZAM_run_time_error(z_loc, err.c_str()); \
res[i] = std::nullopt; \
} \
else \
@ -272,7 +251,6 @@ ZBody::ZBody(std::string _func_name, const ZAMCompiler* zc) : Stmt(STMT_ZAM) {
auto log_ID_type = lookup_ID("ID", "Log");
ASSERT(log_ID_type);
ZAM::log_ID_enum_type = log_ID_type->GetType<EnumType>();
ZAM::any_base_type = base_type(TYPE_ANY);
ZVal::SetZValNilStatusAddr(&ZAM_error);
did_init = false;
}
@ -335,6 +313,9 @@ ValPtr ZBody::Exec(Frame* f, StmtFlowType& flow) {
// Type of the return value. If nil, then we don't have a value.
TypePtr ret_type;
// ListVal corresponding to INDEX_LIST.
static auto zam_index_val_list = make_intrusive<ListVal>(TYPE_ANY);
#ifdef ENABLE_ZAM_PROFILE
static bool profiling_active = analysis_options.profile_ZAM;
static int sampling_rate = analysis_options.profile_sampling_rate;
@ -529,33 +510,6 @@ void ZBody::ReportProfile(ProfMap& pm, const ProfVec& pv, const std::string& pre
}
}
bool ZBody::CheckAnyType(const TypePtr& any_type, const TypePtr& expected_type,
const std::shared_ptr<ZAMLocInfo>& loc) const {
if ( IsAny(expected_type) )
return true;
if ( ! same_type(any_type, expected_type, false, false) ) {
auto at = any_type->Tag();
auto et = expected_type->Tag();
if ( at == TYPE_RECORD && et == TYPE_RECORD ) {
auto at_r = any_type->AsRecordType();
auto et_r = expected_type->AsRecordType();
if ( record_promotion_compatible(et_r, at_r) )
return true;
}
char buf[8192];
snprintf(buf, sizeof buf, "run-time type clash (%s/%s)", type_name(at), type_name(et));
reporter->RuntimeError(loc->Loc(), "%s", buf);
return false;
}
return true;
}
void ZBody::Dump() const {
printf("Frame:\n");
@ -591,12 +545,22 @@ TraversalCode ZBody::Traverse(TraversalCallback* cb) const {
TraversalCode tc = cb->PreStmt(this);
HANDLE_TC_STMT_PRE(tc);
for ( auto& gi : globals ) {
tc = gi.id->Traverse(cb);
HANDLE_TC_STMT_PRE(tc);
}
for ( size_t i = 0; i < NumInsts(); ++i ) {
tc = insts[i].Traverse(cb);
HANDLE_TC_STMT_PRE(tc);
}
tc = cb->PostStmt(this);
HANDLE_TC_STMT_POST(tc);
}
// Unary vector operation of v1 <vec-op> v2.
static void vec_exec(ZOp op, TypePtr t, VectorVal*& v1, const VectorVal* v2, const ZInst& z) {
static void vec_exec(ZOp op, TypePtr t, VectorVal*& v1, const VectorVal* v2, const ZInst& /* z */) {
// We could speed this up further still by gen'ing up an instance
// of the loop inside each switch case (in which case we might as
// well move the whole kit-and-caboodle into the Exec method). But

44
src/script_opt/ZAM/ZBody.h
View file

@ -8,6 +8,31 @@
#include "zeek/script_opt/ZAM/Profile.h"
#include "zeek/script_opt/ZAM/Support.h"
////////////////////////////////////////////////////////////////////////
// Headers needed for run-time execution:
// Needed for managing the corresponding values.
#include "zeek/File.h"
#include "zeek/Func.h"
#include "zeek/OpaqueVal.h"
// Just needed for BiFs.
#include "zeek/analyzer/Manager.h"
#include "zeek/analyzer/protocol/conn-size/ConnSize.h"
#include "zeek/broker/Manager.h"
#include "zeek/file_analysis/Manager.h"
#include "zeek/file_analysis/file_analysis.bif.h"
#include "zeek/logging/Manager.h"
#include "zeek/packet_analysis/Manager.h"
#include "zeek/packet_analysis/protocol/gtpv1/GTPv1.h"
#include "zeek/packet_analysis/protocol/teredo/Teredo.h"
#include "zeek.bif.func_h"
// For reading_live and reading_traces
#include "zeek/RunState.h"
////////////////////////////////////////////////////////////////////////
namespace zeek::detail {
// Static information about globals used in a function.
@ -63,6 +88,11 @@ public:
const std::string& FuncName() const { return func_name; }
private:
friend class CPPCompile;
auto Instructions() const { return insts; }
auto NumInsts() const { return end_pc; }
// Initializes profiling information, if needed.
void InitProfile();
std::shared_ptr<ProfVec> BuildProfVec() const;
@ -70,11 +100,6 @@ private:
void ReportProfile(ProfMap& pm, const ProfVec& pv, const std::string& prefix,
std::set<std::string> caller_modules) const;
// Run-time checking for "any" type being consistent with
// expected typed. Returns true if the type match is okay.
bool CheckAnyType(const TypePtr& any_type, const TypePtr& expected_type,
const std::shared_ptr<ZAMLocInfo>& loc) const;
StmtPtr Duplicate() override { return {NewRef{}, this}; }
void StmtDescribe(ODesc* d) const override;
@ -128,4 +153,13 @@ private:
ProfVec* curr_prof_vec;
};
extern bool copy_vec_elem(VectorVal* vv, zeek_uint_t ind, ZVal zv, const TypePtr& t);
extern VectorVal* vec_coerce_DI(VectorVal* vec, std::shared_ptr<ZAMLocInfo> z_loc);
extern VectorVal* vec_coerce_DU(VectorVal* vec, std::shared_ptr<ZAMLocInfo> z_loc);
extern VectorVal* vec_coerce_ID(VectorVal* vec, std::shared_ptr<ZAMLocInfo> z_loc);
extern VectorVal* vec_coerce_IU(VectorVal* vec, std::shared_ptr<ZAMLocInfo> z_loc);
extern VectorVal* vec_coerce_UD(VectorVal* vec, std::shared_ptr<ZAMLocInfo> z_loc);
extern VectorVal* vec_coerce_UI(VectorVal* vec, std::shared_ptr<ZAMLocInfo> z_loc);
} // namespace zeek::detail

114
src/script_opt/ZAM/ZInst.cc
View file

@ -5,6 +5,7 @@
#include "zeek/Desc.h"
#include "zeek/Func.h"
#include "zeek/Reporter.h"
#include "zeek/module_util.h"
using std::string;
@ -229,16 +230,16 @@ ValPtr ZInst::ConstVal() const {
bool ZInst::IsLoopIterationAdvancement() const {
switch ( op ) {
case OP_NEXT_TABLE_ITER_VV:
case OP_NEXT_TABLE_ITER_NO_VARS_VV:
case OP_NEXT_TABLE_ITER_VAL_VAR_VVV:
case OP_NEXT_TABLE_ITER_VAL_VAR_NO_VARS_VVV:
case OP_NEXT_VECTOR_ITER_VVV:
case OP_NEXT_VECTOR_BLANK_ITER_VV:
case OP_NEXT_VECTOR_ITER_VAL_VAR_VVVV:
case OP_NEXT_VECTOR_BLANK_ITER_VAL_VAR_VVV:
case OP_NEXT_STRING_ITER_VVV:
case OP_NEXT_STRING_BLANK_ITER_VV: return true;
case OP_NEXT_TABLE_ITER_fb:
case OP_NEXT_TABLE_ITER_NO_VARS_fb:
case OP_NEXT_TABLE_ITER_VAL_VAR_Vfb:
case OP_NEXT_TABLE_ITER_VAL_VAR_NO_VARS_Vfb:
case OP_NEXT_VECTOR_ITER_Vsb:
case OP_NEXT_VECTOR_BLANK_ITER_sb:
case OP_NEXT_VECTOR_ITER_VAL_VAR_VVsb:
case OP_NEXT_VECTOR_BLANK_ITER_VAL_VAR_Vsb:
case OP_NEXT_STRING_ITER_Vsb:
case OP_NEXT_STRING_BLANK_ITER_sb: return true;
default: return false;
}
@ -281,7 +282,7 @@ bool ZInst::AssignsToSlot1() const {
bool ZInst::AssignsToSlot(int slot) const {
switch ( op ) {
case OP_NEXT_VECTOR_ITER_VAL_VAR_VVVV: return slot == 1 || slot == 2;
case OP_NEXT_VECTOR_ITER_VAL_VAR_VVsb: return slot == 1 || slot == 2;
default: return slot == 1 && AssignsToSlot1();
}
@ -298,6 +299,84 @@ string ZInst::ConstDump() const {
return d.Description();
}
TraversalCode ZInst::Traverse(TraversalCallback* cb) const {
TraversalCode tc;
if ( t ) {
tc = t->Traverse(cb);
HANDLE_TC_STMT_PRE(tc);
if ( t2 ) {
tc = t2->Traverse(cb);
HANDLE_TC_STMT_PRE(tc);
}
}
if ( aux ) {
tc = aux->Traverse(cb);
HANDLE_TC_STMT_POST(tc);
}
return TC_CONTINUE;
}
TraversalCode ZInstAux::Traverse(TraversalCallback* cb) const {
TraversalCode tc;
if ( id_val ) {
tc = id_val->Traverse(cb);
HANDLE_TC_STMT_PRE(tc);
}
// Don't traverse the "func" field, as if it's a recursive function
// we can wind up right back here.
if ( lambda ) {
tc = lambda->Traverse(cb);
HANDLE_TC_STMT_PRE(tc);
}
if ( event_handler ) {
auto g = lookup_ID(event_handler->Name(), GLOBAL_MODULE_NAME, false, false, false);
ASSERT(g);
tc = g->Traverse(cb);
HANDLE_TC_STMT_PRE(tc);
}
if ( attrs ) {
tc = attrs->Traverse(cb);
HANDLE_TC_STMT_PRE(tc);
}
if ( value_var_type ) {
tc = value_var_type->Traverse(cb);
HANDLE_TC_STMT_PRE(tc);
}
for ( auto& lvt : types ) {
tc = lvt->Traverse(cb);
HANDLE_TC_STMT_PRE(tc);
}
if ( elems ) {
for ( int i = 0; i < n; ++i ) {
auto& e_i = elems[i];
auto& c = e_i.Constant();
if ( c ) {
tc = c->GetType()->Traverse(cb);
HANDLE_TC_STMT_PRE(tc);
}
auto& t = e_i.GetType();
if ( t ) {
tc = t->Traverse(cb);
HANDLE_TC_STMT_PRE(tc);
}
}
}
return TC_CONTINUE;
}
void ZInstI::Dump(FILE* f, const FrameMap* frame_ids, const FrameReMap* remappings) const {
int n = NumFrameSlots();
// fprintf(f, "v%d ", n);
@ -358,8 +437,9 @@ string ZInstI::VName(int n, const FrameMap* frame_ids, const FrameReMap* remappi
bool ZInstI::DoesNotContinue() const {
switch ( op ) {
case OP_GOTO_V:
case OP_GOTO_b:
case OP_HOOK_BREAK_X:
case OP_WHEN_RETURN_X:
case OP_RETURN_C:
case OP_RETURN_V:
case OP_RETURN_X: return true;
@ -397,7 +477,7 @@ bool ZInstI::IsDirectAssignment() const {
bool ZInstI::HasCaptures() const {
switch ( op ) {
case OP_LAMBDA_VV:
case OP_LAMBDA_Vi:
case OP_WHEN_V:
case OP_WHEN_TIMEOUT_VV:
case OP_WHEN_TIMEOUT_VC: return true;
@ -557,7 +637,7 @@ void ZInstI::UpdateSlots(std::vector<int>& slot_mapping) {
}
bool ZInstI::IsGlobalLoad() const {
if ( op == OP_LOAD_GLOBAL_TYPE_VV )
if ( op == OP_LOAD_GLOBAL_TYPE_Vg )
// These don't have flavors.
return true;
@ -566,7 +646,7 @@ bool ZInstI::IsGlobalLoad() const {
if ( global_ops.empty() ) { // Initialize the set.
for ( int t = 0; t < NUM_TYPES; ++t ) {
TypeTag tag = TypeTag(t);
ZOp global_op_flavor = AssignmentFlavor(OP_LOAD_GLOBAL_VV, tag, false);
ZOp global_op_flavor = AssignmentFlavor(OP_LOAD_GLOBAL_Vg, tag, false);
if ( global_op_flavor != OP_NOP )
global_ops.insert(global_op_flavor);
@ -576,11 +656,11 @@ bool ZInstI::IsGlobalLoad() const {
return global_ops.count(op) > 0;
}
bool ZInstI::IsCaptureLoad() const { return op == OP_LOAD_CAPTURE_VV || op == OP_LOAD_MANAGED_CAPTURE_VV; }
bool ZInstI::IsCaptureLoad() const { return op == OP_LOAD_CAPTURE_Vi || op == OP_LOAD_MANAGED_CAPTURE_Vi; }
void ZInstI::InitConst(const ConstExpr* ce) {
auto v = ce->ValuePtr();
t = ce->GetType();
SetType(ce->GetType());
c = ZVal(v, t);
if ( ZAM_error )

82
src/script_opt/ZAM/ZInst.h
View file

@ -6,6 +6,7 @@
#include "zeek/Desc.h"
#include "zeek/Func.h"
#include "zeek/TraverseTypes.h"
#include "zeek/script_opt/ZAM/BuiltInSupport.h"
#include "zeek/script_opt/ZAM/Support.h"
#include "zeek/script_opt/ZAM/ZOp.h"
@ -109,6 +110,8 @@ public:
// Returns a string describing the constant.
std::string ConstDump() const;
TraversalCode Traverse(TraversalCallback* cb) const;
ZOp op = OP_NOP;
ZAMOpType op_type = OP_X;
@ -124,9 +127,19 @@ public:
// Meta-data associated with the execution.
protected:
// These are protected to ensure that setting 't' is done via SetType(),
// so we can keep is_managed consistent with it. We don't need that
// for 't2' but keep them together for consistency.
// Type, usually for interpreting the constant.
TypePtr t = nullptr;
TypePtr t2 = nullptr; // just a few ops need two types
TypePtr t;
TypePtr t2; // just a few ops need two types
public:
const TypePtr& GetType() const { return t; }
const TypePtr& GetType2() const { return t2; }
// Auxiliary information. We could in principle use this to
// consolidate a bunch of the above, though at the cost of
@ -140,7 +153,7 @@ public:
// Whether v1 represents a frame slot type for which we
// explicitly manage the memory.
bool is_managed = false;
std::optional<bool> is_managed;
};
// A intermediary ZAM instruction, one that includes information/methods
@ -214,7 +227,7 @@ public:
// True if this instruction always branches elsewhere. Different
// from DoesNotContinue() in that returns & hook breaks do not
// continue, but they are not branches.
bool IsUnconditionalBranch() const { return op == OP_GOTO_V; }
bool IsUnconditionalBranch() const { return op == OP_GOTO_b; }
// True if this instruction is of the form "v1 = v2".
bool IsDirectAssignment() const;
@ -254,19 +267,19 @@ public:
bool IsLoad() const { return op_type == OP_VV_FRAME || IsNonLocalLoad(); }
// True if the instruction corresponds to storing a global.
bool IsGlobalStore() const { return op == OP_STORE_GLOBAL_V; }
bool IsGlobalStore() const { return op == OP_STORE_GLOBAL_g; }
void CheckIfManaged(const TypePtr& t) {
if ( ZVal::IsManagedType(t) )
is_managed = true;
}
void CheckIfManaged(const TypePtr& t) { is_managed = ZVal::IsManagedType(t); }
void SetType(TypePtr _t) {
t = std::move(_t);
ASSERT(t);
if ( t )
CheckIfManaged(t);
}
void SetType2(TypePtr _t) { t2 = std::move(_t); }
// Whether the instruction should be included in final code
// generation.
bool live = true;
@ -337,6 +350,21 @@ public:
return zv;
}
// The same, but for read-only access for which memory-management is
// not required.
const ZVal& ToDirectZVal(const ZVal* frame) const {
if ( c )
return zc;
if ( i >= 0 )
return frame[i];
// Currently the way we use AuxElem's we shouldn't get here, but
// just in case we do, return something sound rather than mis-indexing
// the frame.
static ZVal null_zval;
return null_zval;
}
int Slot() const { return i; }
int IntVal() const { return i; }
const ValPtr& Constant() const { return c; }
@ -357,6 +385,20 @@ private:
bool is_managed = false;
};
enum ControlFlowType {
CFT_IF,
CFT_BLOCK_END,
CFT_ELSE,
CFT_LOOP,
CFT_LOOP_COND,
CFT_NEXT,
CFT_BREAK,
CFT_DEFAULT,
CFT_INLINED_RETURN,
CFT_NONE,
};
// Auxiliary information, used when the fixed ZInst layout lacks
// sufficient expressiveness to represent all of the elements that
// an instruction needs.
@ -440,6 +482,8 @@ public:
// Same but for constants.
void Add(int i, ValPtr c) { elems[i].SetConstant(c); }
TraversalCode Traverse(TraversalCallback* cb) const;
// Member variables. We could add accessors for manipulating
// these (and make the variables private), but for convenience we
// make them directly available.
@ -448,11 +492,8 @@ public:
AuxElem* elems = nullptr;
bool elems_has_slots = true;
// Ingredients associated with lambdas ...
ScriptFuncPtr primary_func;
// ... and its name.
std::string lambda_name;
// Info for constructing lambdas.
LambdaExprPtr lambda;
// For "when" statements.
std::shared_ptr<WhenInfo> wi;
@ -461,11 +502,11 @@ public:
std::unique_ptr<CatArg>* cat_args = nullptr;
// Used for accessing function names.
IDPtr id_val = nullptr;
IDPtr id_val;
// Interpreter call expression associated with this instruction,
// for error reporting and stack backtraces.
CallExprPtr call_expr = nullptr;
CallExprPtr call_expr;
// Used for direct calls.
Func* func = nullptr;
@ -473,11 +514,14 @@ public:
// Whether we know that we're calling a BiF.
bool is_BiF_call = false;
// Associated control flow information.
std::map<ControlFlowType, int> cft;
// Used for referring to events.
EventHandler* event_handler = nullptr;
// Used for things like constructors.
AttributesPtr attrs = nullptr;
AttributesPtr attrs;
// Whether the instruction can lead to globals/captures changing.
// Currently only needed by the optimizer, but convenient to
@ -550,8 +594,8 @@ extern std::unordered_map<ZOp, std::unordered_map<TypeTag, ZOp>> assignment_flav
// value is superfluous.
extern std::unordered_map<ZOp, ZOp> assignmentless_op;
// Maps flavorful assignments to what op-type their non-assignment
// Maps flavorful assignments to what operand class their non-assignment
// counterpart uses.
extern std::unordered_map<ZOp, ZAMOpType> assignmentless_op_type;
extern std::unordered_map<ZOp, ZAMOpType> assignmentless_op_class;
} // namespace zeek::detail

2
src/script_opt/ZAM/ZOp.cc
View file

@ -57,7 +57,7 @@ bool op_side_effects[] = {
std::unordered_map<ZOp, std::unordered_map<TypeTag, ZOp>> assignment_flavor;
std::unordered_map<ZOp, ZOp> assignmentless_op;
std::unordered_map<ZOp, ZAMOpType> assignmentless_op_type;
std::unordered_map<ZOp, ZAMOpType> assignmentless_op_class;
ZOp AssignmentFlavor(ZOp orig, TypeTag tag, bool strict) {
static bool did_init = false;

13
src/script_opt/ZAM/ZOp.h
View file

@ -4,6 +4,9 @@
#pragma once
#include <string>
#include <unordered_map>
namespace zeek::detail {
// Opcodes associated with ZAM instructions.
@ -59,6 +62,16 @@ enum ZAMOp1Flavor {
OP1_INTERNAL, // we're doing some internal manipulation of the slot
};
// Used to describe ZAM instructions for validation.
struct ZAMInstDesc {
std::string op_class; // associated class
std::string op_types; // operand types
std::string op_eval; // evaluation
};
// Provides access to the validation description of each operation.
extern std::unordered_map<ZOp, ZAMInstDesc> zam_inst_desc;
// Maps an operand to its flavor.
extern ZAMOp1Flavor op1_flavor[];

4
src/zeek-setup.cc
View file

@ -918,8 +918,8 @@ SetupResult setup(int argc, char** argv, Options* zopts) {
analyze_scripts(options.no_unused_warnings);
if ( analysis_options.report_recursive ) {
// This option is report-and-exit.
if ( analysis_options.report_recursive || analysis_options.validate_ZAM ) {
// These options are report-and-exit.
early_shutdown();
exit(0);
}

2
testing/btest/Baseline.zam/bifs.from_json-2/.stderr
View file

@ -1,2 +1,2 @@
### BTest baseline data generated by btest-diff. Do not edit. Use "btest -U/-u" to update. Requires BTest >= 0.63.
error in <...>/from_json.zeek, line 4: from_json() requires a type argument (from_json([], 10, from_json_default_key_mapper))
error in <...>/from_json.zeek, line 4: from_json() requires a type argument (from_json([], <internal>::#0, from_json_default_key_mapper))

2
testing/btest/Baseline.zam/core.option-runtime-errors-2/.stderr
View file

@ -1,2 +1,2 @@
### BTest baseline data generated by btest-diff. Do not edit. Use "btest -U/-u" to update. Requires BTest >= 0.63.
error in <...>/option-runtime-errors.zeek, line 3: Incompatible type for set of ID 'A': got 'string', need 'count' (Option::set(A, hi, ))
error in <...>/option-runtime-errors.zeek, line 3: Incompatible type for set of ID 'A': got 'string', need 'count' (Option::set(A, <internal>::#0, ))

2
testing/btest/Baseline.zam/core.option-runtime-errors-3/.stderr
View file

@ -1,2 +1,2 @@
### BTest baseline data generated by btest-diff. Do not edit. Use "btest -U/-u" to update. Requires BTest >= 0.63.
error in <...>/option-runtime-errors.zeek, line 3: ID 'A' is not an option (Option::set(A, 6, ))
error in <...>/option-runtime-errors.zeek, line 3: ID 'A' is not an option (Option::set(A, <internal>::#0, ))

2
testing/btest/Baseline.zam/core.option-runtime-errors/.stderr
View file

@ -1,2 +1,2 @@
### BTest baseline data generated by btest-diff. Do not edit. Use "btest -U/-u" to update. Requires BTest >= 0.63.
error in <...>/option-runtime-errors.zeek, line 9: Could not find ID named 'B' (Option::set(B, 6, ))
error in <...>/option-runtime-errors.zeek, line 9: Could not find ID named 'B' (Option::set(B, <internal>::#0, ))

2
testing/btest/Baseline.zam/opt.validate-ZAM/output Normal file
View file

@ -0,0 +1,2 @@
### BTest baseline data generated by btest-diff. Do not edit. Use "btest -U/-u" to update. Requires BTest >= 0.63.
1237 valid, 1860 tested, 426 skipped

1
testing/btest/Baseline/bifs.sub_bytes/.stderr Normal file
View file

@ -0,0 +1 @@
### BTest baseline data generated by btest-diff. Do not edit. Use "btest -U/-u" to update. Requires BTest >= 0.63.

9
testing/btest/Baseline/bifs.sub_bytes/out Normal file
View file

@ -0,0 +1,9 @@
### BTest baseline data generated by btest-diff. Do not edit. Use "btest -U/-u" to update. Requires BTest >= 0.63.
bcde
bcde
bcde
bcde
bcde
bcde
bcde
bcde

1
testing/btest/Baseline/language.vector-type-checking/out
View file

@ -17,3 +17,4 @@ error in count and <...>/vector-type-checking.zeek, line 39: arithmetic mixed wi
error in <...>/vector-type-checking.zeek, line 39 and count: type mismatch (thousand-two and count)
error in <...>/vector-type-checking.zeek, line 39: inconsistent types in vector constructor (vector(thousand-two))
error in <...>/vector-type-checking.zeek, line 45: type clash in assignment (lea = vector(thousand-three))
error in <...>/vector-type-checking.zeek, line 57: cannot compare string vectors with pattern vectors (vector(foo) == vector(<...>/))

22
testing/btest/bifs/sub_bytes.zeek Normal file
View file

@ -0,0 +1,22 @@
# @TEST-DOC: Test the sub_bytes() function.
#
# @TEST-EXEC: zeek -b %INPUT >out
# @TEST-EXEC: btest-diff out
# @TEST-EXEC: TEST_DIFF_CANONIFIER=$SCRIPTS/diff-remove-abspath btest-diff .stderr
# These tests are to ensure that script optimization gets all the permutations
# correct, for varying combinations of constant and variable arguments.
global s = "abcdefghij";
global a = 2;
global b = 4;
print sub_bytes(s, a, b);
print sub_bytes(s, 2, b);
print sub_bytes(s, a, 4);
print sub_bytes(s, 2, 4);
print sub_bytes("abcdefghij", a, b);
print sub_bytes("abcdefghij", 2, b);
print sub_bytes("abcdefghij", a, 4);
print sub_bytes("abcdefghij", 2, 4);

13
testing/btest/language/vector-type-checking.zeek
View file

@ -44,3 +44,16 @@ event zeek_init()
{
local lea: MyVec = vector("thousand-three"); # type clash
}
# check operation that's okay as a scalar but not as a vector
event zeek_init()
{
if ( "foo" == /fo*/ )
print "should not complain";
}
event zeek_init()
{
if ( vector("foo") == vector(/fo*/) )
print "should complain";
}

5
testing/btest/opt/validate-ZAM.zeek Normal file
View file

@ -0,0 +1,5 @@
# @TEST-DOC: ZAM maintenance script for validating synthesized operations.
# @TEST-REQUIRES: test "${ZEEK_ZAM}" == "1"
#
# @TEST-EXEC: zeek -b -O validate-ZAM %INPUT >output
# @TEST-EXEC: btest-diff output