Commit graph

374 commits

Author SHA1 Message Date
Arne Welzel
cbaf43e8ea VectorVal: Embed vector_val
Similar motivation as for RecordVal, save an extra malloc/free
and pointer indirection.

This breaks the `auto& RawVec()` API which previously returned
a reference to the std::vector*. It now returns a reference
to the vector instead. It's commented as intended for internal
and compiled code, so even though it's public API,

The previous `std::vector<std::optional<ZVal>>*&` return type was also very
likely not intended (all consumers just dereference it anyhow). I'm certain
this API was never meant to modify the actual pointer value.

I've switched to explicit typing, too.
2023-09-22 21:52:52 +02:00
Arne Welzel
f362935a66 RecordVal: Embed record_val
This should remove one malloc/free per created and destroyed record instance
and avoid one extra pointer indirection to access fields.
2023-09-22 19:43:07 +02:00
Arne Welzel
aaa81cae5d CompositeHash: Skip record initialization when recovering vals
Initializing fields of recovered records caused running &default expression
of fields just so that they are re-assigned in the next step with the
recovered fields. The second test case still shows that the loop var
is initialized as well even though that's not needed.

Add tests for iterating over records with &default attributes for both,
tables and vectors.

Fixes #3267
2023-09-08 13:02:34 +02:00
Arne Welzel
73a7fdad95 TableVal: Unify &default and &default_insert lookups
Introduce DefaultAttr() helper to avoid a bit of duplicated code.
2023-08-04 12:31:27 +02:00
Arne Welzel
431767d04b Add &default_insert attribute for tables
This is based on the discussion in zeek/zeek#2668. Using &default with tables
can be confusing as the default value is not inserted. The following example
prints an empty table at the end even new Service records was instantiated.

    type Service: record {
        occurrences: count &default=0;
        last_seen: time &default=network_time();
    };

    global services: table[string] of Service &default=Service();

    event zeek_init()
        {
        services["http"]$occurrences += 1;
        services["http"]$last_seen = network_time();

        print services;
        }

Changing above &default to &default_insert will insert the newly created
default value upon a missed lookup and act less surprising.

Other examples that caused confusion previously revolved around table of sets
 or table of vectors and `add` or `+=` not working as expected.

    tbl_of_vector["http"] += 1
    add tbl_of_set["http"][1];
2023-08-04 12:30:36 +02:00
Vern Paxson
6b0d595dae avoid constructing TypeList's on-the-fly for ListVal's with fixed types 2023-07-17 16:31:30 -07:00
Tim Wojtulewicz
90d0bc64fa Replace empty destructor bodies with =default definitions 2023-07-07 09:17:05 -07:00
Vern Paxson
1505fd4aa1 added some class accessors/set-ers 2023-06-30 09:36:14 +02:00
Arne Welzel
480d52ca1f from_json: Support function to normalize key names
When a JSON document contains key names containing colons or other
special characters that are not valid in Zeek identifiers, from_json()
cannot be used to parse such input.

This change allows a customizable normalization function.

Closes #3142.
2023-06-29 15:57:49 +02:00
Arne Welzel
1facc34e09 Fixup Val.h/Val.cc: Actually move ValFromJSON into zeek::detail
Lost during merge..
2023-05-09 11:23:32 +02:00
Arne Welzel
264284150b Merge remote-tracking branch 'amazing-pp/topic/fupeng/from_json_bif'
* amazing-pp/topic/fupeng/from_json_bif:
  Implement from_json bif

Minor updates during merge: Moved ValFromJSON into zeek::detail for the
time being, removed gotos, normalized some error messages to lower case,
minimal test extension and added a raw reader input framework test reading
"json lines" as a demo, adding notes about the implicit type
conversions.
2023-05-09 10:36:58 +02:00
Fupeng Zhao
584e68434d Implement from_json bif 2023-05-06 00:42:46 +00:00
Vern Paxson
c19eba62d6 restored RecordType::Create, now marked as deprecated
tidying of namespaces and private class members
simplification of flagging record field initializations that should be skipped
address peculiar MSVC compilation complaint for IntrusivePtr's
2023-04-18 09:41:45 -07:00
Vern Paxson
ee358affda clarifications and tidying for record field initializations 2023-04-15 20:12:49 -07:00
Vern Paxson
0787c130d0 optimize record construction by deferring initializations of aggregates 2023-04-10 11:44:11 -07:00
Vern Paxson
4600ca41f6 logging speedup by switching to raw record access 2023-04-10 11:43:19 -07:00
Tim Wojtulewicz
4f902c0f39 Add configure option for preallocating PortVal objects 2023-03-15 10:12:32 -07:00
Vern Paxson
b7f7d32bf7 Fix for EnumVal's returning their underlying value
Change EnumVal()->AsEnum() to zeek_int_t.
2023-03-08 10:10:24 +01:00
Tim Wojtulewicz
a8fc63e182 Merge remote-tracking branch 'microsoft/master'
* microsoft/master: (71 commits)
  Clang formatting
  Mask ports before inserting them into the map
  Fix compiler warning from applied patch
  Remove statistics plugin in favor of stats bif
  Add EventHandler version of stats plugin
  Mark a few EventHandler methods const
  Changed implementation from std::map to std::unordered_map of Val.cc
  Removed const, Windows build is now working
  Added fixes suggested in PR
  Update src/packet_analysis/protocol/ip/IP.cc
  Apply suggestions from code review
  Clang format again but now with v13.0.1
  Rewrote usages of define(_MSC_VER) to ifdef _MSC_VER
  Clang format it all
  Fixed initial CR comments
  Add NEWS entry about Windows port
  Add a couple of extra unistd.h includes to fix a build failure
  Use std::chrono instead of gettimeofday
  Update libkqueue submodule [nomail]
  Don't call tokenize_string if the input string is empty
  ...
2022-11-11 15:23:21 -07:00
Tomer Lev
9a3855cc38 Changed implementation from std::map to std::unordered_map of Val.cc 2022-11-10 19:09:57 +02:00
Josh Soref
cd201aa24e Spelling src
These are non-functional changes.

* accounting
* activation
* actual
* added
* addresult
* aggregable
* aligned
* alternatively
* ambiguous
* analysis
* analyzer
* anticlimactic
* apparently
* application
* appropriate
* arithmetic
* assignment
* assigns
* associated
* authentication
* authoritative
* barrier
* boundary
* broccoli
* buffering
* caching
* called
* canonicalized
* capturing
* certificates
* ciphersuite
* columns
* communication
* comparison
* comparisons
* compilation
* component
* concatenating
* concatenation
* connection
* convenience
* correctly
* corresponding
* could
* counting
* data
* declared
* decryption
* defining
* dependent
* deprecated
* detached
* dictionary
* directional
* directly
* directory
* discarding
* disconnecting
* distinguishes
* documentation
* elsewhere
* emitted
* empty
* endianness
* endpoint
* enumerator
* essentially
* evaluated
* everything
* exactly
* execute
* explicit
* expressions
* facilitates
* fiddling
* filesystem
* flag
* flagged
* for
* fragments
* guarantee
* guaranteed
* happen
* happening
* hemisphere
* identifier
* identifies
* identify
* implementation
* implemented
* implementing
* including
* inconsistency
* indeterminate
* indices
* individual
* information
* initial
* initialization
* initialize
* initialized
* initializes
* instantiate
* instantiated
* instantiates
* interface
* internal
* interpreted
* interpreter
* into
* it
* iterators
* length
* likely
* log
* longer
* mainly
* mark
* maximum
* message
* minimum
* module
* must
* name
* namespace
* necessary
* nonexistent
* not
* notifications
* notifier
* number
* objects
* occurred
* operations
* original
* otherwise
* output
* overridden
* override
* overriding
* overwriting
* ownership
* parameters
* particular
* payload
* persistent
* potential
* precision
* preexisting
* preservation
* preserved
* primarily
* probably
* procedure
* proceed
* process
* processed
* processes
* processing
* propagate
* propagated
* prototype
* provides
* publishing
* purposes
* queue
* reached
* reason
* reassem
* reassemble
* reassembler
* recommend
* record
* reduction
* reference
* regularly
* representation
* request
* reserved
* retrieve
* returning
* separate
* should
* shouldn't
* significant
* signing
* simplified
* simultaneously
* single
* somebody
* sources
* specific
* specification
* specified
* specifies
* specify
* statement
* subdirectories
* succeeded
* successful
* successfully
* supplied
* synchronization
* tag
* temporarily
* terminating
* that
* the
* transmitted
* true
* truncated
* try
* understand
* unescaped
* unforwarding
* unknown
* unknowndata
* unspecified
* update
* usually
* which
* wildcard

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>
2022-11-09 12:08:15 -05:00
Elad Solomon
3f349b8a37 Optimize initial memory consumption 2022-11-09 18:15:34 +02:00
Tomer Lev
0dbfd9b04e Building supervisor for window 2022-11-09 18:15:33 +02:00
Tim Wojtulewicz
bd7df9e2bf Add &ordered attribute for tables/sets 2022-10-16 15:31:42 -07:00
Tim Wojtulewicz
313e303fda Merge remote-tracking branch 'AmazingPP/topic/amazingpp/table-values-and-keys'
* AmazingPP/topic/amazingpp/table-values-and-keys:
  Add more test cases
  Add table_keys function
  Add table_values function
2022-08-11 08:49:34 -07:00
Vern Paxson
6185d1c389 ZAM fix for vector "in" operator 2022-07-30 18:17:13 -07:00
Tim Wojtulewicz
f624c18383 Deprecate bro_int_t and bro_uint_t 2022-07-12 12:01:23 -07:00
AmazingPP
ba552ceeaf Add table_keys function 2022-07-09 01:11:31 +08:00
Tim Wojtulewicz
9f97a91d39 Remove unnecssary includes of Dict.h 2022-07-05 13:33:34 -07:00
Tim Wojtulewicz
47e7fe2cd1 Convert Dictionary types to be templated classes
This has the fortunate side-effect of also making it so we can store
the value objects as typed pointers, instead of void*.
2022-07-05 13:33:34 -07:00
Tim Wojtulewicz
fb16ce3711 Remove other general deprecations 2022-06-30 19:17:13 +00:00
Tim Wojtulewicz
70e63d4749 Remove deprecated MemoryAllocation() methods and related code 2022-06-30 18:56:52 +00:00
Vern Paxson
58cdc0be09 to avoid recursion, track all aggregates, not just records
isolate the internal methods
2022-05-05 16:51:59 -07:00
Vern Paxson
7fd94f82a8 simpler public calling interface for computing footprint 2022-05-02 15:08:37 -07:00
Vern Paxson
15123b6768 use stack-based set to prevent infinite recursion rather than a static one 2022-05-02 14:16:20 -07:00
Vern Paxson
edf276520a make including count of container elements non-optional 2022-04-29 09:03:35 -07:00
Vern Paxson
16c37034de added value_footprint() and global_container_footprints() BiFs 2022-04-28 16:42:14 -07:00
Tim Wojtulewicz
e6e9144da6 Add unit testing for DNS_Mgr and related classes 2022-04-22 13:10:37 -07:00
Vern Paxson
18fe9d84f6 bug fix for empty table constructors with &default attributes (plus a typo) 2022-03-31 19:28:23 -07:00
Tim Wojtulewicz
fe935a572f Merge remote-tracking branch 'origin/topic/vern/event-trace'
* origin/topic/vern/event-trace:
  low-level style tweaks
  --event-trace / -E option to generate event trace
  hooks to support event tracing
  classes providing event-tracing/dumping functionality
  provide access to Val internals for event tracing purposes
  set_network_time() BiF in support of event replaying
  low-level naming tweaks / comments / const-ified accessor
2022-03-25 15:31:42 -07:00
Vern Paxson
c3e2871a5e simplification of Val classes now that they don't have to support initialization 2022-03-23 15:36:35 -07:00
Vern Paxson
cde677bd97 provide access to Val internals for event tracing purposes 2022-03-23 10:03:13 -07:00
Tim Wojtulewicz
4f470c24ed String/StringVal: Replace char*/string constructors with string_view 2022-02-04 13:52:04 -07:00
Vern Paxson
d758585e42 updated Bro->Zeek in comments in the source tree 2022-01-24 14:26:20 -08:00
Tim Wojtulewicz
3d9d6e953b Merge remote-tracking branch 'origin/topic/vern/when-lambda'
* origin/topic/vern/when-lambda:
  explicitly provide the frame for evaluating a "when" timeout expression
  attempt to make "when" btest deterministic
  tests for new "when" semantics/errors
  update existing test suite usage of "when" statements to include captures
  update uses of "when" in base scripts to include captures
  captures for "when" statements update Triggers to IntrusivePtr's and simpler AST traversal introduce IDSet type, migrate associated "ID*" types to "const ID*"
  logic (other than in profiling) for assignments that yield separate values
  option for internal use to mark a function type as allowing non-expression returns
  removed some now-obsolete profiling functionality
  minor commenting clarifications
2022-01-14 14:41:42 -07:00
Johanna Amann
3a3f9af40d Make some StringVal functions const
I noticed these while working on the TLS changes - I do not see a good
reason why they should not be const.
2022-01-14 11:04:11 +00:00
Vern Paxson
f895008c34 captures for "when" statements
update Triggers to IntrusivePtr's and simpler AST traversal
introduce IDSet type, migrate associated "ID*" types to "const ID*"
2022-01-07 14:50:35 -08:00
Tim Wojtulewicz
edf90a51e4 GH-1624: Migrate check_and_promote and a few Expr methods to IntrusivePtr 2022-01-07 12:33:54 -07:00
Tim Wojtulewicz
9af6b2f48d clang-format: Set penalty for breaking after assignment operator 2021-09-27 10:49:48 -07:00
Tim Wojtulewicz
b2f171ec69 Reformat the world 2021-09-16 15:35:39 -07:00