Commit graph

10798 commits

Author SHA1 Message Date
Tim Wojtulewicz
11ecc7d7eb Add new CI task for running benchmarks on a remote host
- Adds centos-8 docker instance to the CI configuration
- Adds new benchmark.sh script for passing a build artifact to a remote host to run benchmarks
2020-06-01 14:46:50 -07:00
Jon Siwek
5bcdc46272 Update submodule(s)
[nomail]
2020-06-01 12:17:29 -07:00
Tim Wojtulewicz
fcb28c54d2 Update submodule(s)
[nomail]
2020-06-01 11:00:13 -07:00
Tim Wojtulewicz
503ef26a17 Merge remote-tracking branch 'origin/topic/jsiwek/gh-893-intrusive-ptr-migration'
* origin/topic/jsiwek/gh-893-intrusive-ptr-migration: (151 commits)
  Integrate review feedback
  Switch Broker Val converter visitor to return IntrusivePtr
  Change BroFunc ctor to take const-ref IntrusivePtr<ID>
  Add version of Frame::SetElement() taking IntrusivePtr<ID>
  Change Scope/Func inits from id_list* to vector<IntrusivePtr<ID>>
  Change Scope::GenerateTemporary() to return IntrusivePtr
  Deprecate Scope::ReturnType(), replace with GetReturnType()
  Deprecate Scope::ScopeID(), replace with GetID()
  Switch parsing to use vector<IntrusivePtr<Attr>> from attr_list
  Deprecate TableVal::FindAttr(), replace with GetAttr()
  Deprecate TypeDecl::FindAttr(), replace with GetAttr()
  Deprecate ID::FindAttr(), replace with GetAttr()
  Deprecate Attributes::FindAttr(), replace with Find()
  Deprecate Attributes::AddAttrs(Attributes*)
  Add Attributes ctor that takes IntrusivePtrs
  Change Attributes to store std:vector<IntrusivePtr<Attr>>
  Change Attr::SetAttrExpr() to non-template
  Deprecate Attr::AttrExpr(), replace with GetExpr()
  Deprecate ID::Attrs(), replace with GetAttrs()
  Remove weak_ref param from ID::SetVal()
  ...
2020-06-01 10:58:02 -07:00
Jon Siwek
9c133b9b10 Integrate review feedback
* Add deprecation for MIME_Entity::ContentType(), use GetContentType()

* Add deprecation for MIME_Entity::ContentSubType(), use GetContentSubType()

* Add deprecation for MIME_Message::BuildHeaderVal(), use ToHeaderVal()

* Add deprecation for MIME_Message::BuildHeaderTable(), use ToHeaderTable()

* Add deprecation for mime::new_string_val(), use mime::to_stringval()

* Add deprecation for ARP_Analyzer::ConstructAddrVal(), use ToAddrVal()

* Add deprecation for ARP_Analyzer::EthAddrToStr(), use ToEthAddrStr()

* Change the Func::Call() replacement to be named Func::Invoke()
2020-05-29 19:14:35 -07:00
Johanna Amann
ebb106c9b8 Store updates: downlevel and store table elements as keys.
In this commit we change our approach and just store table keys as
broker keys and table values as broker indexes. This means we only can
have a single table in a broker store.

This seems to work perfectly - and leads to vastly less complex code.

Code should work, but is actually pretty untested. Might break with
certain types. Does not yet handle item expiration.

Also - this has the tiny issue that all remote operations currently lead
to loops (the operation will immediately be sent to the broker-store
again), which is not quite optimal.
2020-05-29 17:56:11 -07:00
Jon Siwek
0fc59c357f Update submodule(s)
[nomail]
2020-05-29 17:34:05 -07:00
Jon Siwek
2f918ed9b2 Merge branch 'topic/dopheide/known-services' of https://github.com/dopheide-esnet/zeek
- Updated the logic significantly: still filters out ICMP from being
  considered an active service (like before) and adds a new
  "Known::service_udp_requires_response" option (defaults to true) for
  whether to require UDP server response before being considered an
  active service.

* 'topic/dopheide/known-services' of https://github.com/dopheide-esnet/zeek:
  Log services with unknown protocols
2020-05-29 17:19:47 -07:00
Johanna Amann
558e89b3ba Broker Store updates: get a bit more infrastructure in place.
This compiles, but besides giving debug messages (and partially
performing inserts/updates) it is not really helpful and definitely WIP.

This also shows that I might have to re-think the approach that we will
take here. So far, we actually insert tables as tables into
Brokerstores. This opens up the potential to just have several tables
synchronized via a single brokerstore.

However, it turns out, that the current store_event API sends the
complete table with each update. Which is problematic for obvious
reasons - and not really sustainable.
2020-05-29 15:28:42 -07:00
Jon Siwek
d2eb701b7e Fix potential leak of Analyzers added to tree during Analyzer::Done
Credit to OSS-Fuzz for discovery
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=22630
2020-05-28 16:23:58 -07:00
Jon Siwek
1c08be1c0f Fix crash on using some deprecated environment variables
If the global Reporter hasn't been created before trying to use a
deprecated environment variable, emit the warning to stderr directly
instead of via Reporter.

Fixes GH-989
2020-05-28 15:24:25 -07:00
Jon Siwek
138c9402c3 Update NEWS 2020-05-28 14:51:08 -07:00
Jon Siwek
54e40d4544 Merge branch 'topic/ak/rdp-nego-flags' of https://github.com/anthonykasza/zeek
- Added check in rdp_connect_request before accessing member of the
  switch since it wasn't necessarily the case that got parsed (guard
  against nullptr deref).

* 'topic/ak/rdp-nego-flags' of https://github.com/anthonykasza/zeek:
  change: now using alternate event prototypes
  add: two new RDP scriptland events for negotiation flags
2020-05-28 14:33:12 -07:00
Jon Siwek
8c03e47b12 Update test baselines for new Broker connection status/error strings 2020-05-28 14:11:03 -07:00
Johanna Amann
8db83a5ed2 Make &broker_store take argument of type string.
It turns out that directly passing a Broker::Store is not really a
bright idea. Because - if we do that - we have to later try to intercept
when the master/clone is generated to figure out what the actual name of
the backing store is.

Turns out that it is much easier to just use the name directly - and
check if a store with that name exists when we want to insert something.

I might want to reconsider this in the future in any case. At the moment
this approach just stores one table into an entire store. In theory, one
store should be able to handle several tables, but... that's more
complex. So let's start with this for now.
2020-05-28 13:47:13 -07:00
Jon Siwek
82ce64ca70 Switch Broker Val converter visitor to return IntrusivePtr 2020-05-27 17:56:25 -07:00
Jon Siwek
1f45e690a0 Change BroFunc ctor to take const-ref IntrusivePtr<ID> 2020-05-27 17:40:02 -07:00
Jon Siwek
0d19e8fb4c Add version of Frame::SetElement() taking IntrusivePtr<ID>
Expect the version using raw ID* could go away eventually, but this is
convenience for the meantime.
2020-05-27 17:31:02 -07:00
Jon Siwek
2cee468eac Change Scope/Func inits from id_list* to vector<IntrusivePtr<ID>> 2020-05-27 17:27:40 -07:00
Jon Siwek
46e23b49fb Change Scope::GenerateTemporary() to return IntrusivePtr 2020-05-27 16:51:25 -07:00
Jon Siwek
a13899c95e Deprecate Scope::ReturnType(), replace with GetReturnType() 2020-05-27 16:38:31 -07:00
Jon Siwek
b0c95e30d0 Deprecate Scope::ScopeID(), replace with GetID() 2020-05-27 16:36:14 -07:00
Anthony Kasza
2bb3309dd4 change: now using alternate event prototypes 2020-05-27 09:21:53 -06:00
Jon Siwek
8b6de5852c Switch parsing to use vector<IntrusivePtr<Attr>> from attr_list
This allows improved passing/storing of Attr references to Exprs,
TypeDecl, Scope, etc.
2020-05-26 18:19:29 -07:00
Jon Siwek
ce6f69cd19 Deprecate TableVal::FindAttr(), replace with GetAttr() 2020-05-26 15:46:46 -07:00
Jon Siwek
e365105872 Deprecate TypeDecl::FindAttr(), replace with GetAttr() 2020-05-26 15:41:25 -07:00
Jon Siwek
5fc78a548c Deprecate ID::FindAttr(), replace with GetAttr() 2020-05-26 15:31:31 -07:00
Jon Siwek
bee321711f Deprecate Attributes::FindAttr(), replace with Find() 2020-05-26 15:25:08 -07:00
Jon Siwek
6daa33364b Deprecate Attributes::AddAttrs(Attributes*)
Replaced with version taking an IntrusivePtr parameter
2020-05-26 15:12:20 -07:00
Jon Siwek
ccd1cbbc54 Add Attributes ctor that takes IntrusivePtrs 2020-05-26 15:05:38 -07:00
Johanna Amann
433e1154da Merge branch 'add_bzar_dce_rpc_consts' of https://github.com/ct-square/zeek
* 'add_bzar_dce_rpc_consts' of https://github.com/ct-square/zeek:
  Remove dupplicate DCE-RPC endpoint
  Add DCE-RPC constants from BZAR project

Closes GH-953
2020-05-26 22:04:33 +00:00
Jon Siwek
102e58b80b Change Attributes to store std:vector<IntrusivePtr<Attr>>
This also changes the return type of Attributes::Attrs() from attr_list*
2020-05-26 13:05:24 -07:00
Jon Siwek
007533295a Change Attr::SetAttrExpr() to non-template 2020-05-26 12:21:23 -07:00
Jon Siwek
97636e97a5 Deprecate Attr::AttrExpr(), replace with GetExpr() 2020-05-26 12:13:56 -07:00
Jon Siwek
2ebc20a164 Deprecate ID::Attrs(), replace with GetAttrs() 2020-05-26 11:34:44 -07:00
Anthony Kasza
3333a00eb2 add: two new RDP scriptland events for negotiation flags 2020-05-26 12:11:22 -06:00
Jon Siwek
c171af47f5 GH-979: Update libkqueue to fix use on CentOS 6 2020-05-26 11:07:40 -07:00
Jon Siwek
28b4206519 Remove weak_ref param from ID::SetVal()
It was not used anywhere.
2020-05-23 09:29:27 -07:00
Jon Siwek
198d604dde Store weak ref boolean along with Frame element Vals 2020-05-23 09:19:50 -07:00
Jon Siwek
9f4eca081f Deprecate Frame::GetElement(ID*), replace with GetElementByID() 2020-05-23 09:08:45 -07:00
Jon Siwek
1ccbe743d0 Deprecate Frame::NthElement(), replace with GetElement() 2020-05-23 08:52:34 -07:00
Jon Siwek
e9e2e388f8 Switch Frame::SetElement() to use IntrusivePtr 2020-05-23 08:52:15 -07:00
Jon Siwek
1c617c4f7a Store IntrusivePtrs in Frame 2020-05-22 23:43:03 -07:00
Jon Siwek
272db640aa Deprecate Plugin::HookCallFunction(), replace with HookFunctionCall()
This also changes the argument type of Func::operator() to zeek::Args*
to allow plugins to be able to alter function arguments in place as
was previously documented.
2020-05-22 21:01:38 -07:00
Jon Siwek
46c5dea733 Switch plugin::Manager::HookCallFunction() to return IntrusivePtr
The plugin::Plugin side of things is not (yet) changed.
2020-05-22 17:48:35 -07:00
Jon Siwek
b1042e2824 Port remaining file analysis API to use IntrusivePtr 2020-05-22 16:32:40 -07:00
Jon Siwek
57a6069cd1 Deprecate file analyzer construction methods taking raw RecordVal*
Replaced with versions that instead take IntrusivePtr
2020-05-22 16:13:15 -07:00
Jon Siwek
ecb7c7c27e Deprecate file_analysis::Analyzer::Args(), replace with GetArgs() 2020-05-22 12:34:40 -07:00
Jon Siwek
faa4738807 Deprecate file_analysis::File::GetVal(), replace with ToVal() 2020-05-22 12:27:37 -07:00
Jon Siwek
27c3c207e4 Change file_analysis::File::GetID() to return const-ref 2020-05-22 00:22:17 -07:00