Commit graph

9176 commits

Author SHA1 Message Date
Johanna Amann
750fa2e226 Merge remote-tracking branch 'origin/topic/jsiwek/gh-427-fix-zeekygen-default-vals'
* origin/topic/jsiwek/gh-427-fix-zeekygen-default-vals:
  GH-427: improve default ID values shown by Zeekygen
2019-06-25 10:39:15 -07:00
Jon Siwek
9e9440e88d GH-427: improve default ID values shown by Zeekygen
The default value of an ID is now truly the one used to initialize it,
unaltered by any subsequent redefs.

Redefs are now shown separately, along with the expression that
modifies the ID's value.
2019-06-24 19:02:05 -07:00
Johanna Amann
c139ad07f4 Unbreak build on Linux.
Turns out os-x does not to include memory...
2019-06-24 15:51:04 -07:00
Johanna Amann
31973f76ce Rename paraglob_get to paraglob_match
After going back to the original bug, I think this naming makes things a
bit more understandable.

GH-268
2019-06-24 15:44:21 -07:00
Johanna Amann
54613ad2b8 Merge remote-tracking branch 'origin/topic/jsiwek/gh-435-rpc-crash'
* origin/topic/jsiwek/gh-435-rpc-crash:
  GH-435: fix null pointer deref in RPC analyzer
2019-06-24 15:26:27 -07:00
Johanna Amann
5f9a9bbcbe Merge branch 'paraglob' of https://github.com/ZekeMedley/zeek
* 'paraglob' of https://github.com/ZekeMedley/zeek:
  Add leak test to paraglob.
  Catch paraglob serialization errors in DoClone.
  Update paraglob serialization.
  Stop execution on paraglob error.
  Update paraglob submodule
  Change C++11 detection in paraglob.
  Make paraglob serializable and copyable.
  Initial paraglob integration.

I made a bunch of small changes:
 * paraglob now deals better with \0 characters
 * I rolled back the changes to Binary Serialization format,
 * there were some small formatting issue
 * the error output was slightly unsafe
 * build_unique is now in util.h.

and perhaps a few more small things.
2019-06-24 15:21:46 -07:00
Zeke Medley
9efca707b6 Add leak test to paraglob. 2019-06-24 12:40:16 -07:00
Jon Siwek
8cb8c83daf GH-435: fix null pointer deref in RPC analyzer
Was possible for some failed parsing calls in
RPC_Interpreter::DeliverRPC to set buf to a null-pointer, only for it
to later access it when checking whether the message length appeared
to be legit.

Adding the null-initialization of buf in RPC_Interpreter::Timeout()
was just precautionary -- didn't look like any real bugs with anyone
accessing its uninitialized state at the moment.
2019-06-24 10:43:35 -07:00
Zeke Medley
6fd7d8c4d4 Sync with master for merge. 2019-06-24 09:40:43 -07:00
Zeke Medley
86741e7d06 Update tests to reflect new error msg. 2019-06-24 09:34:02 -07:00
Zeke Medley
f27209e84c Merge branch 'master' of https://github.com/zeek/zeek into topic/zeke/closures 2019-06-21 12:31:34 -07:00
Zeke Medley
b280c62ea5 ClosureFrame takes ownership of its captured vals. 2019-06-21 12:31:10 -07:00
Jon Siwek
16785a2d70 Merge remote-tracking branch 'origin/topic/timw/171-deprecation-warnings'
* origin/topic/timw/171-deprecation-warnings:
  GH-171: support warning messages alongside deprecated attributes

Made a minor tweak to give a better error message when using
&deprecated= with something other than a string literal expression.
2019-06-21 12:29:46 -07:00
Zeke Medley
d468df2a21 Merge branch 'master' of https://github.com/zeek/zeek into topic/zeke/closures 2019-06-21 12:03:20 -07:00
Zeke Medley
8257a644d3 Lambdas selectively clone from the closure. 2019-06-21 12:02:41 -07:00
Jon Siwek
7cf623b2ff Merge remote-tracking branch 'origin/topic/johanna/gh-417-consider-removing-p0f'
* origin/topic/johanna/gh-417-consider-removing-p0f:
  Remove old, unmaintained p0f support.
2019-06-21 11:18:20 -07:00
Zeke Medley
63117de050 Merge branch 'master' of https://github.com/zeek/zeek into topic/zeke/closures 2019-06-21 10:11:02 -07:00
Jon Siwek
6918e54936 Updating submodule(s).
[nomail]
2019-06-21 09:56:57 -07:00
Zeke Medley
1ed672287b Merge branch 'master' of https://github.com/zeek/zeek into topic/zeke/closures 2019-06-21 09:41:41 -07:00
Tim Wojtulewicz
a5e03d25d4 GH-171: support warning messages alongside deprecated attributes 2019-06-20 22:57:15 -07:00
Jon Siwek
f95390aabe Merge branch 'master' of https://github.com/zeek/zeek 2019-06-20 20:55:24 -07:00
Jon Siwek
d72f5458f1 Merge branch 'rdp_client_cluster_data' of https://github.com/neslog/zeek
* 'rdp_client_cluster_data' of https://github.com/neslog/zeek:
  Fixing types.
  Correcting types.
  Removing misc data from Client Cluster data trying to assign values.
  Adding options field to RDP::ClientChannelDef Adding Client Cluster Data

Adjustments:

- Reformatting
- Added comments
- Changed the REDIRECTED_SESSIONID_FIELD_VALID field to a bool
2019-06-20 20:41:16 -07:00
Johanna Amann
9564651655 Merge remote-tracking branch 'origin/master' 2019-06-20 20:25:05 -07:00
Jon Siwek
6ccf1f928b Merge remote-tracking branch 'origin/topic/johanna/warn-if-no-cmake-files'
* origin/topic/johanna/warn-if-no-cmake-files:
  Make configure complain if submodules are not checked out.
2019-06-20 19:57:52 -07:00
Jon Siwek
8f19bbe589 Improve C++ header includes to improve build time
Recent changes ended up including all the Broker headers more places
than necessary, causing compile time to increase 2x.
2019-06-20 19:50:23 -07:00
Zeke Medley
a3001f1b2b Add lambda expressions with closures to Zeek.
This allows anonymous functions in Zeek to capture their closures.
they do so by creating a copy of their enclosing frame and joining
that with their own frame.

There is no way to specify what specific items to capture from the
closure like C++, nor is there a nonlocal keyword like Python.
Attemptying to declare a local variable that has already been caught
by the closure will error nicely. At the worst this is an inconvenience
for people who are using lambdas which use the same variable names
as their closures.

As a result of functions copying their enclosing frames there is no
way for a function with a closure to reach back up and modify the
state of the frame that it was created in. This lets functions that
generate functions work as expected. The function can reach back and
modify its copy of the frame that it is captured in though.

Implementation wise this is done by creating two new subclasses in
Zeek. The first is a LambdaExpression which can be thought of as a
function generator. It gathers all of the ingredients for a function
at parse time, and then when evaluated creats a new version of that
function with the frame it is being evaluated in as a closure. The
second subclass is a ClosureFrame. This acts for most intents and
purposes like a regular Frame, but it routes lookups of values to its
closure as needed.
2019-06-20 18:43:56 -07:00
Jon Siwek
aefd9322fd Fix TableVal::DoClone to use CloneState cache 2019-06-20 18:34:36 -07:00
Johanna Amann
b1be8abe4e Merge remote-tracking branch 'remotes/origin/topic/robin/gh59-when'
Fixed a few small bugs - Modifiable had an uninitialized member and the
Registry looped over a map while deleting elements from it.

Fixes GH-319

* remotes/origin/topic/robin/gh59-when:
  Renaming src/StateAccess.{h,cc} to src/Notifier.{h,cc}.
  Clean up new code.
  Remove MutableVal class.
  Redo API for notifiers.
  Remove most of MutableVal (but not the class itelf yet)
  Remove enum Opcode.
  Remove StateAccess class.
  Redo NotfifierRegistry to no longer rely on StateAccess.
  Add new test for when-statement watching global variables.
  Couple of compile fixes.
2019-06-20 18:19:18 -07:00
Zeke Medley
5d55586293 Catch paraglob serialization errors in DoClone. 2019-06-20 15:29:53 -07:00
Zeke Medley
f1779a2518 Update paraglob serialization. 2019-06-20 15:13:31 -07:00
Jon Siwek
61d19d25e1 Remove old Broccoli SSL options
- ssl_ca_certificate
- ssl_private_key
- ssl_passphrase
2019-06-20 14:19:55 -07:00
Zeke Medley
a5f6757d7d Merge remote-tracking branch 'upstream/master' into paraglob 2019-06-20 14:14:48 -07:00
Zeke Medley
a8d5779593 Merge branch 'master' of https://github.com/ZekeMedley/zeek into paraglob 2019-06-20 14:04:53 -07:00
Jon Siwek
b9d3d4d63b Remove unused SerialInfo.h and SerialTypes.h headers 2019-06-20 14:01:16 -07:00
Jon Siwek
399496efa8 Merge remote-tracking branch 'origin/topic/johanna/remove-serializer'
* origin/topic/johanna/remove-serializer:
  Fix memory leak introduced by removing opaque of ocsp_resp.
  Change return value of OpaqueVal::DoSerialize.
  Add missing ShallowClone implementation for SetType
  Remove opaque of ocsp_resp.
  Remove remnants of event serializer.
  Fix cardinalitycounter deserialization.
  Smaller compile fixes for the new opaque serialization.
  Reimplement serialization infrastructure for OpaqueVals.
  Couple of compile fixes.
  Remove const from ShallowClone.
  Remove test-case for removed functionality
  Implement a Shallow Clone operation for types.
  Remove value serialization.

Various changes I made:

- Fix memory leak in type-checker for opaque vals wrapped in broker::data

- Noticed the two "copy-all" leak tests weren't actually checking for
  memory leaks because the heap checker isn't active until after zeek_init()
  is evaluated.

- Change OpaqueVal::DoClone to use the clone caching mechanism

- Improve copy elision for broker::expected return types in the various
  OpaqueVal serialize methods

  - Not all compilers end up properly treating the return of
    local/automatic variable as an rvalue that can be moved, and ends up
    copying it instead.

  - Particularly, until GCC 8, this pattern ends up copying instead of
    moving, and we still support platforms whose default compiler
    pre-dates that version.

  - Generally seems it's something that wasn't addressed until C++14.
    See http://www.open-std.org/jtc1/sc22/wg21/docs/cwg_defects.html#1579

- Change OpaqueVal::SerializeType to return broker::expected

- Change probabilistic DoSerialize methods to return broker::expected
2019-06-20 13:38:54 -07:00
Johanna Amann
437520f45f Make configure complain if submodules are not checked out.
Since people forgetting to checkout submodules is such a common failure
case - update configure to give an error message is the cmake directory
seems to be missing.

This just checks for the presence of cmake/COPYING when a .git directory
is found; if cmake/COPYING is not present an error message is displayed.
2019-06-20 11:19:17 -07:00
jatkinosn
3a19af86c5 Fixing types.
Added handling for fields sub fields.

Added test script and output.
2019-06-20 10:47:05 -04:00
jatkinosn
7b42c3a201 Correcting types. 2019-06-20 09:32:37 -04:00
Jon Siwek
3648b1465e Updating submodule(s).
[nomail]
2019-06-19 19:44:00 -07:00
Johanna Amann
4792c94212 Merge remote-tracking branch 'origin/master' into topic/johanna/remove-serializer 2019-06-19 18:34:49 -07:00
Johanna Amann
979f64f16e Merge remote-tracking branch 'origin/topic/timw/393-vector-slicing'
* origin/topic/timw/393-vector-slicing:
  Fix memory leak in vector slice assignment
  Misc. tweaks to vector slicing implementation
  Add ability to grow/shrink a vector using slicing, also adds Insert/Remove methods for VectorVal
  Allow assignment for vectors using slices
  Check for integral slice indexes, add extra test for [:]
  Return an empty vector if the indices for slicing don't make sense
  GH-393: Add slice notation for vectors
2019-06-19 17:33:51 -07:00
jatkinosn
bd0bf3f84f Removing misc data from Client Cluster data trying to assign values. 2019-06-19 16:10:29 -04:00
jatkinosn
0282941838 Adding options field to RDP::ClientChannelDef
Adding Client Cluster Data
2019-06-19 15:12:56 -04:00
Johanna Amann
632e83de57 Remove old, unmaintained p0f support.
Addresses GH-417
2019-06-19 09:29:32 -07:00
Johanna Amann
4eed36ea01 Fix memory leak introduced by removing opaque of ocsp_resp. 2019-06-19 09:28:15 -07:00
Jon Siwek
385f500497 Fix memory leak in vector slice assignment
Two parts to this:

  * Only allow vector slice assignment in statement contexts, not in
    arbitrary assignment expressions.  E.g. it's not clear what the
    resulting value of `(v[1:2] = vector(1))` is for further expression
    chaining.  For reference, Python doesn't allow it either.

  * Add a subclass of AssignExpr to specialize the behavior for index
    slice assignments (because its behavior regarding expression
    chaining is different per the previous point) and Unref the RHS
    of things like `v[1:2] = vector(1)` after IndexExpr::Assign is
    finished inserting it (since no one else takes ownership of it).

Instead of using an Expr subclass, IndexSliceAssignExpr, we could
use a proper Stmt, since that's the only context we currently use it
for, but if we did ever to decide on allowing its use in arbitrary
expression contexts, then I expect we'll need it this way anyway
(just with a different IndexSliceAssignExpr::Eval implementation).
2019-06-18 18:09:01 -07:00
Jon Siwek
91835752b7 Misc. tweaks to vector slicing implementation
* Minor style/format changes

* Fix a signed/unsigned comparison compiler warning

* Use a non-fatal error for non-integral slice indices so we can
  report any further scripting errors instead of stopping the parse
  right there
2019-06-18 17:36:32 -07:00
Johanna Amann
e0f10fd6d3 Change return value of OpaqueVal::DoSerialize.
Now it returns a broker::expected<broker::data>, instead of directly
returning a broker::data before. This means that broker::data does no
longer have to be abused to convey error information.

In a way it would be kind of neat to have more fine-granular broker error
types for this use-case - at the moment everything returns
broker::ec::invalid_data, which seems to be the only reasonable choice.
2019-06-18 15:45:40 -07:00
Johanna Amann
53cde131e9 Add missing ShallowClone implementation for SetType
It turns out that SetType was missing a ShallowClone implementation.
Which meant that when a SetType was cloned, a TableType was received
(which has one less member). Which resulted in invalid memory accesses
when using the clone.

Thank you valgrind :)
2019-06-18 14:37:09 -07:00
Johanna Amann
5365d60202 Merge remote-tracking branch 'origin/master' into topic/johanna/remove-serializer 2019-06-18 11:42:25 -07:00