Commit graph

389 commits

Author SHA1 Message Date
Christian Kreibich
a29f862f95 Document the field_escape_pattern in the to_json() BiF
This argument, and its corresponding use in Val.cc's BuildJSON(),
were never explained.
2024-07-02 14:46:16 -07:00
Vern Paxson
1f9fa4304d refine Val "footprint" to equate long strings with multiple objects 2024-04-29 12:39:36 -07:00
Vern Paxson
5311904bb1 removing vestigial same_val() function 2024-04-25 09:15:12 -07:00
Vern Paxson
91cab9931d ZAM optimizations for record creation
includes reworking of managing "auxiliary" information for ZAM instructions
2024-01-25 20:49:12 +01:00
Vern Paxson
bae87fb606 script optimization fixes for "concretizing" vector-of-any's 2024-01-15 15:03:56 +01:00
Vern Paxson
a11ee9038b streamlining of constructing script-level tables 2023-12-16 17:33:46 +01:00
Tim Wojtulewicz
ef5b169acd Add some uses of std::move in constructors and simple functions for pass-by-value arguments 2023-11-28 13:40:28 -07:00
Arne Welzel
cf9afd7b77 TableVal: Replace raw subnets/pattern_matcher with unique_ptr 2023-11-21 11:16:17 +01:00
Arne Welzel
c113b9b297 Expr/Val: Add support for in set[pattern] 2023-11-21 10:34:17 +01:00
Arne Welzel
e39f280e3d zeek.bif: Implement table_pattern_matcher_stats() bif for introspection
Provide a script accessible way to introspect the DFA stats that can be
leveraged to gather runtime statistics of the underlying DFA. This
re-uses the existing MatcherStats used by ``get_matcher_stats()``.
2023-11-21 10:34:17 +01:00
Arne Welzel
501b582bc7 TablePatternMatcher: Use const StringValPtr& instead of const StringVal* 2023-11-21 10:34:16 +01:00
Arne Welzel
c426304c27 Val: Move TablePatternMatcher into detail namespace
There's anyway only prototype in the headers, so detail seems better
than the public zeek namespace.
2023-11-21 10:34:16 +01:00
Vern Paxson
699549eb45 support for indexing "table[pattern] of T" with strings to get multi-matches 2023-11-21 10:34:15 +01:00
Dominik Charousset
cebb85b1e8 Fix unsafe and inefficient uses of copy_string
Add a new overload to `copy_string` that takes the input characters plus
size. The new overload avoids inefficient scanning of the input for the
null terminator in cases where we know the size beforehand. Furthermore,
this overload *must* be used when dealing with input character sequences
that may have no null terminator, e.g., when the input is from a
`std::string_view` object.
2023-11-03 15:25:38 +01:00
Benjamin Bannier
f5a76c1aed Reformat Zeek in Spicy style
This largely copies over Spicy's `.clang-format` configuration file. The
one place where we deviate is header include order since Zeek depends on
headers being included in a certain order.
2023-10-30 09:40:55 +01:00
Arne Welzel
cbaf43e8ea VectorVal: Embed vector_val
Similar motivation as for RecordVal, save an extra malloc/free
and pointer indirection.

This breaks the `auto& RawVec()` API which previously returned
a reference to the std::vector*. It now returns a reference
to the vector instead. It's commented as intended for internal
and compiled code, so even though it's public API,

The previous `std::vector<std::optional<ZVal>>*&` return type was also very
likely not intended (all consumers just dereference it anyhow). I'm certain
this API was never meant to modify the actual pointer value.

I've switched to explicit typing, too.
2023-09-22 21:52:52 +02:00
Arne Welzel
f362935a66 RecordVal: Embed record_val
This should remove one malloc/free per created and destroyed record instance
and avoid one extra pointer indirection to access fields.
2023-09-22 19:43:07 +02:00
Arne Welzel
aaa81cae5d CompositeHash: Skip record initialization when recovering vals
Initializing fields of recovered records caused running &default expression
of fields just so that they are re-assigned in the next step with the
recovered fields. The second test case still shows that the loop var
is initialized as well even though that's not needed.

Add tests for iterating over records with &default attributes for both,
tables and vectors.

Fixes #3267
2023-09-08 13:02:34 +02:00
Arne Welzel
73a7fdad95 TableVal: Unify &default and &default_insert lookups
Introduce DefaultAttr() helper to avoid a bit of duplicated code.
2023-08-04 12:31:27 +02:00
Arne Welzel
431767d04b Add &default_insert attribute for tables
This is based on the discussion in zeek/zeek#2668. Using &default with tables
can be confusing as the default value is not inserted. The following example
prints an empty table at the end even new Service records was instantiated.

    type Service: record {
        occurrences: count &default=0;
        last_seen: time &default=network_time();
    };

    global services: table[string] of Service &default=Service();

    event zeek_init()
        {
        services["http"]$occurrences += 1;
        services["http"]$last_seen = network_time();

        print services;
        }

Changing above &default to &default_insert will insert the newly created
default value upon a missed lookup and act less surprising.

Other examples that caused confusion previously revolved around table of sets
 or table of vectors and `add` or `+=` not working as expected.

    tbl_of_vector["http"] += 1
    add tbl_of_set["http"][1];
2023-08-04 12:30:36 +02:00
Vern Paxson
6b0d595dae avoid constructing TypeList's on-the-fly for ListVal's with fixed types 2023-07-17 16:31:30 -07:00
Tim Wojtulewicz
90d0bc64fa Replace empty destructor bodies with =default definitions 2023-07-07 09:17:05 -07:00
Vern Paxson
1505fd4aa1 added some class accessors/set-ers 2023-06-30 09:36:14 +02:00
Arne Welzel
480d52ca1f from_json: Support function to normalize key names
When a JSON document contains key names containing colons or other
special characters that are not valid in Zeek identifiers, from_json()
cannot be used to parse such input.

This change allows a customizable normalization function.

Closes #3142.
2023-06-29 15:57:49 +02:00
Arne Welzel
1facc34e09 Fixup Val.h/Val.cc: Actually move ValFromJSON into zeek::detail
Lost during merge..
2023-05-09 11:23:32 +02:00
Arne Welzel
264284150b Merge remote-tracking branch 'amazing-pp/topic/fupeng/from_json_bif'
* amazing-pp/topic/fupeng/from_json_bif:
  Implement from_json bif

Minor updates during merge: Moved ValFromJSON into zeek::detail for the
time being, removed gotos, normalized some error messages to lower case,
minimal test extension and added a raw reader input framework test reading
"json lines" as a demo, adding notes about the implicit type
conversions.
2023-05-09 10:36:58 +02:00
Fupeng Zhao
584e68434d Implement from_json bif 2023-05-06 00:42:46 +00:00
Vern Paxson
c19eba62d6 restored RecordType::Create, now marked as deprecated
tidying of namespaces and private class members
simplification of flagging record field initializations that should be skipped
address peculiar MSVC compilation complaint for IntrusivePtr's
2023-04-18 09:41:45 -07:00
Vern Paxson
ee358affda clarifications and tidying for record field initializations 2023-04-15 20:12:49 -07:00
Vern Paxson
0787c130d0 optimize record construction by deferring initializations of aggregates 2023-04-10 11:44:11 -07:00
Vern Paxson
4600ca41f6 logging speedup by switching to raw record access 2023-04-10 11:43:19 -07:00
Tim Wojtulewicz
4f902c0f39 Add configure option for preallocating PortVal objects 2023-03-15 10:12:32 -07:00
Vern Paxson
b7f7d32bf7 Fix for EnumVal's returning their underlying value
Change EnumVal()->AsEnum() to zeek_int_t.
2023-03-08 10:10:24 +01:00
Tim Wojtulewicz
a8fc63e182 Merge remote-tracking branch 'microsoft/master'
* microsoft/master: (71 commits)
  Clang formatting
  Mask ports before inserting them into the map
  Fix compiler warning from applied patch
  Remove statistics plugin in favor of stats bif
  Add EventHandler version of stats plugin
  Mark a few EventHandler methods const
  Changed implementation from std::map to std::unordered_map of Val.cc
  Removed const, Windows build is now working
  Added fixes suggested in PR
  Update src/packet_analysis/protocol/ip/IP.cc
  Apply suggestions from code review
  Clang format again but now with v13.0.1
  Rewrote usages of define(_MSC_VER) to ifdef _MSC_VER
  Clang format it all
  Fixed initial CR comments
  Add NEWS entry about Windows port
  Add a couple of extra unistd.h includes to fix a build failure
  Use std::chrono instead of gettimeofday
  Update libkqueue submodule [nomail]
  Don't call tokenize_string if the input string is empty
  ...
2022-11-11 15:23:21 -07:00
Tomer Lev
9a3855cc38 Changed implementation from std::map to std::unordered_map of Val.cc 2022-11-10 19:09:57 +02:00
Josh Soref
cd201aa24e Spelling src
These are non-functional changes.

* accounting
* activation
* actual
* added
* addresult
* aggregable
* aligned
* alternatively
* ambiguous
* analysis
* analyzer
* anticlimactic
* apparently
* application
* appropriate
* arithmetic
* assignment
* assigns
* associated
* authentication
* authoritative
* barrier
* boundary
* broccoli
* buffering
* caching
* called
* canonicalized
* capturing
* certificates
* ciphersuite
* columns
* communication
* comparison
* comparisons
* compilation
* component
* concatenating
* concatenation
* connection
* convenience
* correctly
* corresponding
* could
* counting
* data
* declared
* decryption
* defining
* dependent
* deprecated
* detached
* dictionary
* directional
* directly
* directory
* discarding
* disconnecting
* distinguishes
* documentation
* elsewhere
* emitted
* empty
* endianness
* endpoint
* enumerator
* essentially
* evaluated
* everything
* exactly
* execute
* explicit
* expressions
* facilitates
* fiddling
* filesystem
* flag
* flagged
* for
* fragments
* guarantee
* guaranteed
* happen
* happening
* hemisphere
* identifier
* identifies
* identify
* implementation
* implemented
* implementing
* including
* inconsistency
* indeterminate
* indices
* individual
* information
* initial
* initialization
* initialize
* initialized
* initializes
* instantiate
* instantiated
* instantiates
* interface
* internal
* interpreted
* interpreter
* into
* it
* iterators
* length
* likely
* log
* longer
* mainly
* mark
* maximum
* message
* minimum
* module
* must
* name
* namespace
* necessary
* nonexistent
* not
* notifications
* notifier
* number
* objects
* occurred
* operations
* original
* otherwise
* output
* overridden
* override
* overriding
* overwriting
* ownership
* parameters
* particular
* payload
* persistent
* potential
* precision
* preexisting
* preservation
* preserved
* primarily
* probably
* procedure
* proceed
* process
* processed
* processes
* processing
* propagate
* propagated
* prototype
* provides
* publishing
* purposes
* queue
* reached
* reason
* reassem
* reassemble
* reassembler
* recommend
* record
* reduction
* reference
* regularly
* representation
* request
* reserved
* retrieve
* returning
* separate
* should
* shouldn't
* significant
* signing
* simplified
* simultaneously
* single
* somebody
* sources
* specific
* specification
* specified
* specifies
* specify
* statement
* subdirectories
* succeeded
* successful
* successfully
* supplied
* synchronization
* tag
* temporarily
* terminating
* that
* the
* transmitted
* true
* truncated
* try
* understand
* unescaped
* unforwarding
* unknown
* unknowndata
* unspecified
* update
* usually
* which
* wildcard

Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>
2022-11-09 12:08:15 -05:00
Elad Solomon
3f349b8a37 Optimize initial memory consumption 2022-11-09 18:15:34 +02:00
Tomer Lev
0dbfd9b04e Building supervisor for window 2022-11-09 18:15:33 +02:00
Tim Wojtulewicz
bd7df9e2bf Add &ordered attribute for tables/sets 2022-10-16 15:31:42 -07:00
Tim Wojtulewicz
313e303fda Merge remote-tracking branch 'AmazingPP/topic/amazingpp/table-values-and-keys'
* AmazingPP/topic/amazingpp/table-values-and-keys:
  Add more test cases
  Add table_keys function
  Add table_values function
2022-08-11 08:49:34 -07:00
Vern Paxson
6185d1c389 ZAM fix for vector "in" operator 2022-07-30 18:17:13 -07:00
Tim Wojtulewicz
f624c18383 Deprecate bro_int_t and bro_uint_t 2022-07-12 12:01:23 -07:00
AmazingPP
ba552ceeaf Add table_keys function 2022-07-09 01:11:31 +08:00
Tim Wojtulewicz
9f97a91d39 Remove unnecssary includes of Dict.h 2022-07-05 13:33:34 -07:00
Tim Wojtulewicz
47e7fe2cd1 Convert Dictionary types to be templated classes
This has the fortunate side-effect of also making it so we can store
the value objects as typed pointers, instead of void*.
2022-07-05 13:33:34 -07:00
Tim Wojtulewicz
fb16ce3711 Remove other general deprecations 2022-06-30 19:17:13 +00:00
Tim Wojtulewicz
70e63d4749 Remove deprecated MemoryAllocation() methods and related code 2022-06-30 18:56:52 +00:00
Vern Paxson
58cdc0be09 to avoid recursion, track all aggregates, not just records
isolate the internal methods
2022-05-05 16:51:59 -07:00
Vern Paxson
7fd94f82a8 simpler public calling interface for computing footprint 2022-05-02 15:08:37 -07:00
Vern Paxson
15123b6768 use stack-based set to prevent infinite recursion rather than a static one 2022-05-02 14:16:20 -07:00