Commit graph

209 commits

Author SHA1 Message Date
Tim Wojtulewicz
86fdf0eaa9 Mark global val_mgr as deprecated and fix uses of it to use namespaced version 2020-07-02 16:15:00 -07:00
Tim Wojtulewicz
464efbe66a Move Dictionary/PDict, List/PList, and Queue/PQueue to zeek namespace 2020-06-30 21:12:26 -07:00
Tim Wojtulewicz
64332ca22c Move all Val classes to the zeek namespaces 2020-06-30 20:48:09 -07:00
Tim Wojtulewicz
ec9eff0bd5 Use type aliases for IntrusivePtr definitions 2020-06-30 20:37:30 -07:00
Tim Wojtulewicz
9364e6a5b7 Move IntrusivePtr and utility methods to the zeek namespace 2020-06-30 20:19:12 -07:00
Tim Wojtulewicz
137e416a03 Rename BroType to Type 2020-06-10 14:27:36 -07:00
Tim Wojtulewicz
ed13972924 Move Type types to zeek namespace 2020-06-09 17:20:45 -07:00
Tim Wojtulewicz
b4aa5d9456 Merge remote-tracking branch 'origin/topic/jsiwek/gh-973-fix-topk-get-top-return-type'
* origin/topic/jsiwek/gh-973-fix-topk-get-top-return-type:
  GH-973: Fix the return type of topk_get_top()
2020-06-03 13:29:31 -07:00
Jon Siwek
5b4313b593 Deprecate Val(double, TypeTag) ctor, add TimeVal/DoubleVal subclasses
This also updates all usages of the deprecated Val ctor to use
either IntervalVal, TimeVal, or DoubleVal ctors.  The reason for
doing away with the old constructor is that using it with TYPE_INTERVAL
isn't strictly correct since there exists a more specific subclass,
IntervalVal, with overriden ValDescribe() method that ought to be used
to print such values in a more descriptive way.
2020-06-02 23:33:40 -07:00
Jon Siwek
49fc7924e0 GH-973: Fix the return type of topk_get_top()
e.g. when using `string` elements it should return `vector of string`,
not `vector of list of string` like it did before.
2020-06-02 19:42:12 -07:00
Jon Siwek
6a1c312451 Add same_type() overloads for IntrusivePtr args 2020-05-21 21:33:02 -07:00
Jon Siwek
de1e3d7d6d Deprecate VectorVal::Assign methods taking raw Val*
And adapt usages to pass in to alternate method taking IntrusivePtr
2020-05-21 15:31:04 -07:00
Jon Siwek
3f92df51b7 Improve TableVal HashKey management
* Deprecated ComputeHash() methods and replaced with MakeHashKey()
  which returns std::unique_ptr<HashKey>

* Deprecated RecoverIndex() and replaced with RecreateIndex()
  which takes HashKey& and returns IntrusivePtr.

* Updated the new TableVal Assign()/Remove() methods to take either
  std::unique_ptr<HashKey> or HashKey& as appropriate for clarity of
  ownership expectations.
2020-05-20 22:16:47 -07:00
Jon Siwek
e01d2c1b37 Deprecate ComputeHash(Val*) methods, replace with ComputeHash(Val&) 2020-05-20 15:47:19 -07:00
Jon Siwek
5742810293 Switch OpaqueVal::SerializeType() to IntrusivePtr 2020-05-18 16:10:10 -07:00
Jon Siwek
377c8a4762 Switch BlommFilterVal/CardinalityVal to use IntrusivePtr<BroType> 2020-05-18 16:07:26 -07:00
Jon Siwek
aa9d9c904f Switch some TopkVal methods to use IntrusivePtr 2020-05-18 15:55:30 -07:00
Jon Siwek
d35e5520f8 Switch TopkVal to store IntrusivePtr<BroType> 2020-05-18 15:37:00 -07:00
Jon Siwek
43f513ca44 Switch OpaqueVal::UnserializeType() to return IntrusivePtr 2020-05-18 15:25:46 -07:00
Jon Siwek
4debad8caf Switch zeek:🆔:lookup to zeek:🆔:find
For parity with Scope since it now uses Find instead of Lookup
2020-05-14 18:00:18 -07:00
Jon Siwek
a5762c12cc Move various elements into ID.h and zeek::id namespace
* A handful of generic/useful/common global type pointers that used
  to be in NetVar.h

* Lookup functions that used to be Var.h
2020-05-14 17:24:20 -07:00
Jon Siwek
d34b24e776 Deprecate global Val pointers in NetVar.h
All of these have fairly niche uses, so better maintained as
lookup/static closer to the usage site.
2020-05-14 17:23:20 -07:00
Jon Siwek
1eb723fc9d Deprecate Val::Type(), replace with GetType() 2020-05-14 17:23:20 -07:00
Jon Siwek
d4dba40727 Deprecate VectorVal(VectorType*) ctora
Adds a new one taking an IntrusivePtr.
2020-05-14 17:23:20 -07:00
Johanna Amann
27d87919a1 Hashing: Remove unnecessary include 2020-05-12 00:30:33 +00:00
Johanna Amann
04ed125941 Merge remote-tracking branch 'origin/master' into topic/johanna/hash-unification 2020-05-06 23:18:33 +00:00
Tim Wojtulewicz
07a1821bcc Fix a few more IntrusivePtr deprecation warnings 2020-04-27 12:59:19 -07:00
Johanna Amann
0136648342 Merge remote-tracking branch 'origin/topic/jsiwek/empty-bloomfilter-lookup'
* origin/topic/jsiwek/empty-bloomfilter-lookup:
  Remove error message from empty bloomfilter lookups
2020-04-27 19:23:16 +00:00
Jon Siwek
9e56881c70 Update various BIFs to return IntrusivePtr 2020-04-27 11:50:35 -07:00
Johanna Amann
360c06a3f8 Start refactoring hashing.
This commit moves some of the hash datastructures and code from
util.cc into Hash.cc - where it seems more appropriate.

It also starts to make more Keyed hash functions available - still
using siphash as the default 64 bit keyed hash, but also making
128 and 256 bit highway hashes available.

There already are a few other functions that are defined but not
yet implemented - these will be "static" keyed hashes - which use
an installation specific key. These will be used to, e.g., get
rid of md5 hashing for the generation of file UIDs.
2020-04-24 18:27:09 -07:00
Johanna Amann
5e7915ae7a Remove the siphash->hmac-md5 switch after 36 bytes.
Currently, siphash is used for strings up to 36 bytes. hmac-md5 is used
for longer strings.

This switch-over is a remnant of the previous hash-function that was
used, which apparently was slower with longer input strings.

This change serves no purpose anymore. I performed a few performance tests
on strings of varying sizes:

For a 40 byte string with 10 million iterations:

siphash: 0.31 seconds
hmac-md5: 3.8 seconds

For a 1080 byte string with 10 million iterations:

siphash: 4.2 seconds
hmac-md5: 17 seconds

For a 18360 byte string with 10 million iterations:

siphash: 69 seconds
hmac-md5: 240 seconds

Hence, this commit removes the use of hmac-md5.

This change causes reordering of lines in a few logs.

This commit also changes the datastructure for the seed in probabilistic/Hasher
to get rid of a type-punning warning.
2020-04-24 13:14:29 -07:00
Jon Siwek
f452f26d11 Remove error message from empty bloomfilter lookups
If a bloomfilter doesn't have a type, that just means no
bloomfilter_add() has been called yet, so seems undesirable to emit an
error for a lookup against something that's known to be empty.
2020-04-24 10:15:57 -07:00
Johanna Amann
3937fff57f Replace siphash with Google implementation
This adds the entirety of the highwayhash implementation of Google.
This includes siphash as well as severl highwayhash variants - which
are faster.

This first commit only switches out the siphash implementation. All
hashes that are generated are exactly the same as before. However, this
does make all other hashes available to be used by us.

I did some performance tests vs the previous siphash implementation by
running the 2009-M57-day11-18 trace 100x through both cases. The average
runtime was virtually the same (within 0.014 seconds of each other).

Note that the way that I included the highwayhash implementation in our
cmake setup is... well, let's say hacky. This definitely needs to be
changed a bit before including this in a real build.
2020-04-23 16:05:03 -07:00
Jon Siwek
094d6de979 Update all BIFs to return IntrusivePtr instead of Val* 2020-04-16 17:00:01 -07:00
Jon Siwek
93f4c5871b Update deprecated ValManager::GetCount usages 2020-04-16 16:46:36 -07:00
Jon Siwek
9af84bb2b0 Update deprecated ValManager GetTrue/GetFalse usages 2020-04-16 16:40:59 -07:00
Johanna Amann
876c803d75 Merge remote-tracking branch 'origin/topic/timw/776-using-statements'
* origin/topic/timw/776-using-statements:
  Remove 'using namespace std' from SerialTypes.h
  Remove other using statements from headers
  GH-776: Remove using statements added by PR 770

Includes small fixes in files that changed since the merge request was
made.

Also includes a few small indentation fixes.
2020-04-09 13:31:07 -07:00
Tim Wojtulewicz
41c3256faa plugin/probabilistic/zeekygen: Replace nulls with nullptr 2020-04-07 16:08:34 -07:00
Tim Wojtulewicz
d53c1454c0 Remove 'using namespace std' from SerialTypes.h
This unfortunately cuases a ton of flow-down changes because a lot of other
code was depending on that definition existing. This has a fairly large chance
to break builds of external plugins, considering how many internal ones it broke.
2020-04-07 15:59:59 -07:00
Tim Wojtulewicz
fd5e15b116 The Great Embooleanating
A large number of functions had return values and/or arguments changed
to use ``bool`` types instead of ``int``.
2020-03-31 06:41:54 +00:00
Max Kellermann
50d53b2973 Type: use class IntrusivePtr in VectorType 2020-03-06 09:06:38 +01:00
Max Kellermann
73cea5dcad Type: use class IntrusivePtr in TypeList 2020-03-06 09:06:38 +01:00
Max Kellermann
ba35ebec4c Type: return IntrusivePtr 2020-03-06 09:06:38 +01:00
Max Kellermann
55398ed5d8 OpaqueVal: use class IntrusivePtr 2020-02-21 20:50:11 +01:00
Max Kellermann
53f49e0057 CompHash: use class IntrusivePtr for the type field 2020-02-20 15:57:39 +01:00
Max Kellermann
08128b244f IntrusivePtr: rename detach() to release()
Follow the C++ standard library conventions (here: `std::unique_ptr`).
2020-02-20 08:58:21 +01:00
Max Kellermann
ace11e74d2 probabilistic/Topk: include cleanup 2020-02-13 10:12:03 +01:00
Max Kellermann
4aac78cf29 Val: forward-declare class PDict, reduce includes 2020-02-12 10:10:26 +01:00
Tim Wojtulewicz
da7749fc43 Add a couple of missing #includes that clang-tidy complains about (clang-diagnostic-error) 2020-02-11 14:11:22 -08:00
Tim Wojtulewicz
eda1b4a23e Use const references over copying variables (performance-unnecessary-copy-initialization, performance-for-range-copy) 2020-02-11 11:02:08 -08:00