Commit graph

389 commits

Author SHA1 Message Date
Max Kellermann
837c8f759b Val: remove unimplemented prototype recover_val()
In the initial commit 10 years ago
(61757ac78b), the implementation was
gone already.
2020-02-27 14:08:00 +01:00
Max Kellermann
3331abb4f2 Val: cast_value_to_type() returns IntrusivePtr 2020-02-27 14:08:00 +01:00
Max Kellermann
96951c1300 Val: use IntrusivePtr in check_and_promote() 2020-02-27 14:08:00 +01:00
Max Kellermann
e2bf12d54a Val: pass IntrusivePtr<> to TableVal::ExpandAndInit()
Clarifies ownership and fixes memory leaks.

Closes https://github.com/zeek/zeek/issues/811
2020-02-24 15:55:52 +01:00
Jon Siwek
f0c713046c Merge branch 'refactor_obj' of https://github.com/MaxKellermann/zeek
Minor formatting/style changes in merge.

* 'refactor_obj' of https://github.com/MaxKellermann/zeek:
  Val: use C++ initializers
  Val: add BroValUnion constructors
  Val: reduce duplicate code by using delegating constructors
  Val: remove unused default constructors and `friend` declarations
  Val: remove the unnecessary BroValUnion typedef
  Type: remove unnecessary enum typedefs
  Type: use C++ initializers
  Type: move code from BroType::BroType() to constexpr functions
  Type: remove useless BroType destructor
  Obj: disallow copying BroObj
  Obj: use C++ initializers
  Obj: make `no_location` constexpr
2020-02-21 20:06:09 -08:00
Max Kellermann
59b5e6b7dc Val: use C++ initializers 2020-02-16 19:23:40 +01:00
Max Kellermann
4a5473c572 Val: add BroValUnion constructors
Allows using C++ initializers.
2020-02-16 19:23:37 +01:00
Max Kellermann
74f93091e6 Val: remove unused default constructors and friend declarations 2020-02-16 19:23:28 +01:00
Max Kellermann
99f1804c62 Val: remove the unnecessary BroValUnion typedef 2020-02-16 19:22:35 +01:00
Max Kellermann
c78fa4bcf9 Val: eliminate the "RE.h" include 2020-02-13 09:13:59 +01:00
Max Kellermann
e98cf0a4a0 Val: eliminate the "BroString.h" include 2020-02-13 09:13:59 +01:00
Max Kellermann
728184ea93 Val: eliminate the "CompHash.h" include 2020-02-13 09:13:59 +01:00
Max Kellermann
4aac78cf29 Val: forward-declare class PDict, reduce includes 2020-02-12 10:10:26 +01:00
Max Kellermann
dfac4572e7 Val: eliminate the "Scope.h" include 2020-02-12 10:10:26 +01:00
Johanna Amann
d6d2bbe878 &on_change: incooperate feedback of Jon. 2020-02-06 14:40:38 -08:00
Johanna Amann
b6a244f784 Merge remote-tracking branch 'origin/master' into topic/johanna/table-on-change 2020-02-05 12:41:19 -08:00
Johanna Amann
18ab7838a9 Table expiry now raises &on_change handlers
Expiry is remprted as ELEMENT_EXPIRED (which was newly introduced), to
allow distinction from table deletion.
2020-02-05 12:37:52 -08:00
Johanna Amann
7f9f66fce9 Prevent recursion of &on_change handlers.
This change prevents &on_change handlers for a table from running if an
&on_change handler for the same table is already running.
2020-02-05 11:45:16 -08:00
Max Kellermann
6a815b4b06 UID, ..: un-inline methods to reduce header dependencies
Only 1% build time speedup, but still, it declutters the headers a bit.

Before this patch:

 2565.17user 141.83system 2:25.46elapsed 1860%CPU (0avgtext+0avgdata 1489076maxresident)k
 72576inputs+9130920outputs (1667major+49400430minor)pagefaults 0swaps

After this patch:

 2537.19user 142.94system 2:26.90elapsed 1824%CPU (0avgtext+0avgdata 1434268maxresident)k
 16240inputs+8887152outputs (1931major+48728888minor)pagefaults 0swaps
2020-02-04 20:51:02 +01:00
Max Kellermann
0db61f3094 include cleanup
The Zeek code base has very inconsistent #includes.  Many sources
included a few headers, and those headers included other headers, and
in the end, nearly everything is included everywhere, so missing
#includes were never noticed.  Another side effect was a lot of header
bloat which slows down the build.

First step to fix it: in each source file, its own header should be
included first to verify that each header's includes are correct, and
none is missing.

After adding the missing #includes, I replaced lots of #includes
inside headers with class forward declarations.  In most headers,
object pointers are never referenced, so declaring the function
prototypes with forward-declared classes is just fine.

This patch speeds up the build by 19%, because each compilation unit
gets smaller.  Here are the "time" numbers for a fresh build (with a
warm page cache but without ccache):

Before this patch:

 3144.94user 161.63system 3:02.87elapsed 1808%CPU (0avgtext+0avgdata 2168608maxresident)k
 760inputs+12008400outputs (1511major+57747204minor)pagefaults 0swaps

After this patch:

 2565.17user 141.83system 2:25.46elapsed 1860%CPU (0avgtext+0avgdata 1489076maxresident)k
 72576inputs+9130920outputs (1667major+49400430minor)pagefaults 0swaps
2020-02-04 20:51:02 +01:00
Johanna Amann
745829dbe1 Merge remote-tracking branch 'origin/master' into topic/johanna/table-changes 2020-01-31 10:07:38 -08:00
Johanna Amann
7166cb7373 Merge remote-tracking branch 'origin/master' into topic/johanna/table-on-change 2020-01-31 10:06:52 -08:00
Robin Sommer
d0b206fa36 Remove APIs that were explicitly deprecated to be removed in 3.1.
Special handling for bro_{init,done,script_loaded} events: if still
used, they cause Zeek to abort at startup.
2020-01-29 12:08:09 +00:00
Johanna Amann
68f0fe9e8c Automatic bro table->brokerstore insert operations
We now have an &broker_store attribute which automatically sends
inserts/deletes into a set/table to broker.

This might work - I actually did not test if the data ends up in the
broker store in the end. A limitation is that the table/set currently
only can have a one-element type since Broker doesn't support the list
type.
2020-01-23 13:13:10 -08:00
Johanna Amann
382c4ddc2f &on_change: add insertion/change notifications.
Also - work with sets in addition to tables.
2020-01-13 19:39:40 -08:00
Johanna Amann
b343cf9bb7 &on_change working for removals from tables.
Insertions and changes are still noops.
2020-01-13 15:28:56 -08:00
Johanna Amann
bf31587c2c Beginning implementation of &on_change for tables. 2020-01-09 12:47:16 -05:00
Tim Wojtulewicz
98d1aadb9e GHI-599: avoid memory leak with default pattern matching and json formatting 2019-12-13 10:07:51 -07:00
Jon Siwek
63fe835acf Allow record_fields() string arguments that name a record type 2019-11-01 12:46:17 -07:00
Dominik Charousset
c1f3fe7829 Switch from header guards to pragma once 2019-09-17 14:10:30 +02:00
Tim Wojtulewicz
54752ef9a1 Deprecate the internal int/uint types in favor of the cstdint types they were based on 2019-08-12 13:50:07 -07:00
Robin Sommer
1bf0cd29fd Edit pass over changes before merge. 2019-07-20 00:00:51 +00:00
Zeke Medley
1692be4c2d Fix bug in selective cloning. 2019-07-16 15:15:05 -07:00
Jon Siwek
8c45937798 Merge branch 'topic/jsiwek/template-containers-merge'
* topic/jsiwek/template-containers-merge:
  Fix a potential usage of List::remove_nth(-1)
  Change List::remote(const T&) to return a bool
  Fix debug build due to old int_list usage within assert
  Convert uses of loop_over_list to ranged-for loops
  Remove loop_over_queue (as an example for later removing loop_over_list)
  Change int_list in CCL.h to be a vector, fix uses of int_list to match
  Remove List<> usage from strings.bif
  Replace uses of the old Queue/PQueue generation code with new template versions
  Convert BaseQueue/Queue/PQueue into templates, including iterator support
  Replace uses of the old Dict generation code with new template versions
  Convert PDict into template
  Replace uses of the old List generation code with new template versions
  Convert BaseList/List/PList into templates, including iterator support

* Generally squashed fixups from topic/timw/template-containers

* Add missing include file in List.h: <cassert>
2019-07-15 19:51:27 -07:00
Tim Wojtulewicz
50943a580c Replace uses of the old Dict generation code with new template versions 2019-07-15 18:54:07 -07:00
Zeke Medley
8d9355eb51 Merge branch 'master' of https://github.com/zeek/zeek into topic/zeke/closures 2019-07-12 10:33:43 -07:00
Johanna Amann
1f329ad541 Merge remote-tracking branch 'origin/topic/timw/150-to-json'
* origin/topic/timw/150-to-json:
  Update submodules for JSON work
  Update unit tests for JSON logger to match new output
  Modify JSON log writer to use the external JSON library
  Update unit test output to match json.zeek being deprecated and slight format changes to JSON output
  Add proper JSON serialization via C++, deprecate json.zeek
  Add new method for escaping UTF8 strings for JSON output
  Move do_sub method from zeek.bif to StringVal class method
  Move record_fields method from zeek.bif to Val class method
  Add ToStdString method for StringVal
2019-07-11 11:17:32 -07:00
Tim Wojtulewicz
e390244442 Remove unused variable from RecordVal (Coverity-1402820) 2019-07-08 15:18:13 -07:00
Tim Wojtulewicz
33a7927704 Add proper JSON serialization via C++, deprecate json.zeek 2019-07-02 12:52:26 -07:00
Tim Wojtulewicz
2cb87c3309 Move do_sub method from zeek.bif to StringVal class method 2019-07-02 12:52:26 -07:00
Tim Wojtulewicz
528bad72de Move record_fields method from zeek.bif to Val class method 2019-07-02 12:52:25 -07:00
Tim Wojtulewicz
dffefe0683 Add ToStdString method for StringVal 2019-07-02 12:52:25 -07:00
Zeke Medley
6e84a5eb8e Merge branch 'master' of https://github.com/zeek/zeek into topic/zeke/closures 2019-07-01 13:43:48 -07:00
Jon Siwek
7d2d63551d Improve deprecation warning messages 2019-06-27 18:36:27 -07:00
Zeke Medley
28253b24f9 Table defaults capture closures. 2019-06-27 14:38:38 -07:00
Zeke Medley
1ed672287b Merge branch 'master' of https://github.com/zeek/zeek into topic/zeke/closures 2019-06-21 09:41:41 -07:00
Johanna Amann
9564651655 Merge remote-tracking branch 'origin/master' 2019-06-20 20:25:05 -07:00
Jon Siwek
8f19bbe589 Improve C++ header includes to improve build time
Recent changes ended up including all the Broker headers more places
than necessary, causing compile time to increase 2x.
2019-06-20 19:50:23 -07:00
Zeke Medley
a3001f1b2b Add lambda expressions with closures to Zeek.
This allows anonymous functions in Zeek to capture their closures.
they do so by creating a copy of their enclosing frame and joining
that with their own frame.

There is no way to specify what specific items to capture from the
closure like C++, nor is there a nonlocal keyword like Python.
Attemptying to declare a local variable that has already been caught
by the closure will error nicely. At the worst this is an inconvenience
for people who are using lambdas which use the same variable names
as their closures.

As a result of functions copying their enclosing frames there is no
way for a function with a closure to reach back up and modify the
state of the frame that it was created in. This lets functions that
generate functions work as expected. The function can reach back and
modify its copy of the frame that it is captured in though.

Implementation wise this is done by creating two new subclasses in
Zeek. The first is a LambdaExpression which can be thought of as a
function generator. It gathers all of the ingredients for a function
at parse time, and then when evaluated creats a new version of that
function with the frame it is being evaluated in as a closure. The
second subclass is a ClosureFrame. This acts for most intents and
purposes like a regular Frame, but it routes lookups of values to its
closure as needed.
2019-06-20 18:43:56 -07:00
Jon Siwek
aefd9322fd Fix TableVal::DoClone to use CloneState cache 2019-06-20 18:34:36 -07:00