Commit graph

6667 commits

Author SHA1 Message Date
Johanna Amann
824ccde6fc Merge remote-tracking branch 'origin/master' into topic/johanna/243 2019-05-20 10:14:11 -07:00
Daniel Thayer
fe3d508796 Additional Bro to Zeek renaming
Most of these changes are either cmake-related or plugin-related.
Added a new test "plugins/legacy.zeek" to test that legacy Bro plugins
still work.

Also added a symlink bro-path-dev.in because some legacy Bro packages
won't install without it.
2019-05-19 16:51:36 -05:00
Jon Siwek
dd173f4961 Always emit scripting errors to stderr during zeek_init
Otherwise, setting Reporter::errors_to_stderr=F causes important
error messages to be lost (and this setting is the default for
ZeekCtl).  E.g. now that we terminate if there's errors during
zeek_init, GH-369 shows that the only error message given was
"fatal error: errors occurred while initializing", which is not
helpful in determining the actual issue.
2019-05-17 14:18:45 -07:00
Johanna Amann
ffa173abc0 Implement a Shallow Clone operation for types.
This is needed to track name changes for the documentation.

With this things, which do not need val-cloning, generally seem to work
again. There are a whole bunch of test failures at the moment.
2019-05-17 11:13:04 -07:00
Daniel Thayer
3f9e7138bd More bro-to-zeek renaming in the unit tests 2019-05-16 02:27:54 -05:00
Jon Siwek
bf42dd02bd Merge remote-tracking branch 'origin/topic/dnthayer/gh-359'
* origin/topic/dnthayer/gh-359:
  Changes needed due to bro-to-zeek renaming in broker
2019-05-15 15:03:31 -07:00
Johanna Amann
2bb529f5b7 Merge remote-tracking branch 'origin/topic/jsiwek/dns-mgr-fixes'
* origin/topic/jsiwek/dns-mgr-fixes:
  Fix timing out DNS lookups that were already resolved
  Remove an unhelpful/optimistic DNS_Mgr optimization
  Fix DNS_Mgr priority_queue usage
  Remove dead code from DNS_Mgr
  Improve DNS_Mgr I/O loop: prevent starvation due to busy Broker
  Fix a ref counnting bug in DNS_Mgr
2019-05-15 09:02:41 -07:00
Daniel Thayer
3bbd11b1cd Changes needed due to bro-to-zeek renaming in broker 2019-05-15 00:22:05 -05:00
Jon Siwek
b3c4b986ef Fix maybe-uninitialized compiler warning 2019-05-14 19:01:41 -07:00
Jon Siwek
6ad7099f7e Merge remote-tracking branch 'origin/topic/robin/gh-239'
* origin/topic/robin/gh-239:
  Undo a change to btest.cfg from a recent commit
  Updating submodule.
  Fix zeek-wrapper
  Update for renaming BroControl to ZeekControl.
  Updating submodule.
  GH-239: Rename bro to zeek, bro-config to zeek-config, and bro-path-dev to zeek-path-dev.
2019-05-14 13:27:40 -07:00
Jon Siwek
58d55d0f95 GH-365: improve un-indexable type error message 2019-05-13 20:04:03 -07:00
Jon Siwek
3ae4ffc66e Improve Broker I/O loop integration: less mutex locking
Checking a subscriber for available messages required locking a mutex,
but we should never actually need to do that in the main-loop to check
for Broker readiness since we can rely on file descriptor polling.
2019-05-10 09:16:29 -07:00
Johanna Amann
474efe9e69 Remove value serialization.
Note - this compiles, but you cannot run Bro anymore - it crashes
immediately with a 0-pointer access. The reason behind it is that the
required clone functionality does not work anymore.
2019-05-09 11:54:38 -07:00
Jon Siwek
cb6b9a1f1a Allow tuning Broker log batching via scripts
Via redefining "Broker::log_batch_size" or "Broker::log_batch_interval"
2019-05-08 12:44:55 -07:00
Johanna Amann
312713810f Merge remote-tracking branch 'origin/topic/jsiwek/force-idle-broker'
* origin/topic/jsiwek/force-idle-broker:
  Force the Broker IOSource to idle periodically
2019-05-08 09:04:25 -07:00
Johanna Amann
6f82899466 Merge remote-tracking branch 'origin/topic/jsiwek/gh-353'
* origin/topic/jsiwek/gh-353:
  GH-353: Add `/<re>/i` case-insensitive signature syntax
2019-05-08 08:58:41 -07:00
Jon Siwek
dbb49b17f4 Reduce data copying in Broker message processing 2019-05-07 20:15:31 -07:00
Jon Siwek
9029acd7e5 Merge remote-tracking branch 'origin/topic/johanna/243'
* origin/topic/johanna/243:
  Fix missing include file on Linux
  Deprecations: Update NEWS, and tie up a few loose ends.
  Remove support for using &&/|| with patterns.
  Remove RemoteSerializer and related code/types.
  Remove PersistenceSerializer.
  Remove synchrnized and persistent attributes.
2019-05-06 18:45:10 -07:00
Jon Siwek
5484c40b1f GH-353: Add /<re>/i case-insensitive signature syntax 2019-05-06 14:22:12 -07:00
Johanna Amann
9b49c7cbc6 Fix missing include file on Linux 2019-05-06 18:56:47 +00:00
Johanna Amann
72ec093d56 Deprecations: Update NEWS, and tie up a few loose ends.
Broccoli was still present in the source in a few places, debug outputs
that do no longer exist were too.

Part of GH-243
2019-05-06 11:12:47 -07:00
Mauro Palumbo
3f7eb7436f add SMB2_0_INFO_FILESYSTEM option and move SMB2_FILE_FSCONTROL_INFO and SMB2_FILE_FSOBJECTID_INFO under this option 2019-05-05 22:40:33 +02:00
Mauro Palumbo
8afa4616ff Extend File EA info to several items, minor fixes 2019-05-05 21:11:14 +02:00
Mauro Palumbo
9253357d4b minor fix 2019-05-05 18:29:22 +02:00
Mauro Palumbo
c90eec6b54 update master and merge into this branch 2019-05-05 16:46:41 +02:00
Jon Siwek
69eee058c1 Improve processing of broker data store responses
Now retrieves and processes all N available responses at once instead
of one-by-one-until-empty.  The later may be problematic from two
points: (1) hitting the shared queue/mailbox matching logic once per
response instead of once per Process() and (2) looping until empty is
not clearly bounded -- imagining a condition where there's a thread
trying to push a large influx of responses into the mailbox while at
the same time we're trying to take from it until it's empty.
2019-05-04 11:13:48 -07:00
Johanna Amann
ed644e39a0 Remove support for using &&/|| with patterns.
This was never documented and previously deprecated.
2019-05-03 15:26:21 -07:00
Johanna Amann
4c19e60488 Merge remote-tracking branch 'origin/master' into topic/johanna/remove-deprecated-functions-events 2019-05-03 15:15:22 -07:00
Johanna Amann
dcd6454530 Remove RemoteSerializer and related code/types.
Also removes broccoli from the source tree.
2019-05-03 15:00:13 -07:00
Jon Siwek
84ca12fdb4 Rename Zeexygen to Zeekygen 2019-05-02 21:39:21 -07:00
Jon Siwek
1a77c1b287 Merge remote-tracking branch 'origin/topic/johanna/remove-deprecated-functions-events'
* origin/topic/johanna/remove-deprecated-functions-events:
  Remove deprecated functions/events
2019-05-02 19:18:45 -07:00
Johanna Amann
ca1b1dd6bb Remove PersistenceSerializer. 2019-05-02 13:45:36 -07:00
Johanna Amann
61c84a0a40 Remove synchrnized and persistent attributes.
Code that was used by them is still there.
2019-05-02 13:10:37 -07:00
Johanna Amann
6d47077222 Merge remote-tracking branch 'origin/topic/jsiwek/gh-340'
* origin/topic/jsiwek/gh-340:
  GH-340: Improve IPv4/IPv6 regexes, extraction, and validity functions
2019-05-02 12:28:31 -07:00
Johanna Amann
5d44735209 Remove deprecated functions/events
This commit removed functions/events that have been deprecated in Bro
2.6. It also removes the detection code that checks if the old
communication framework is used (since all the functions that are
checked were removed).

Addresses parts of GH-243
2019-05-02 12:06:39 -07:00
Jon Siwek
46799f7540 Fix timing out DNS lookups that were already resolved
This could happen in the case of making repeated lookup requests
for the same thing within a short period of time: cleaning up an
old request that already got resolved would mistakenly see a new,
yet-to-be-resolved request with identical host/addr and mistakenly
assume it's in need of being timed out.
2019-05-01 23:08:52 -07:00
Jon Siwek
fd11c63efe Remove an unhelpful/optimistic DNS_Mgr optimization
DNS_Mgr is always "idle", so Process() is always called when the
fd signals there's really something ready (except when flushing
at termination-time), so checking whether all pending request maps
are empty within Process() doesn't help much.  If they are empty,
but there's somehow something to pull off the socket, the main loop
is just going to keep trying to call Process() until it gets read
(which would be bad if it's preventing another IOSource from getting
real work done).
2019-05-01 22:55:43 -07:00
Jon Siwek
5bb2a6b1c0 Fix DNS_Mgr priority_queue usage
It was sorting by memory address stored in AsyncRequest pointers
rather than their actual timestamp.
2019-05-01 22:51:54 -07:00
Jon Siwek
5bccb44ad4 Remove dead code from DNS_Mgr 2019-05-01 22:50:47 -07:00
Jon Siwek
6db576195c Improve DNS_Mgr I/O loop: prevent starvation due to busy Broker 2019-05-01 22:46:10 -07:00
Jon Siwek
a8281ff9f9 Fix a ref counnting bug in DNS_Mgr 2019-05-01 22:42:41 -07:00
Robin Sommer
789cb376fd GH-239: Rename bro to zeek, bro-config to zeek-config, and bro-path-dev to zeek-path-dev.
This also installs symlinks from "zeek" and "bro-config" to a wrapper
script that prints a deprecation warning.

The btests pass, but this is still WIP. broctl renaming is still
missing.

#239
2019-05-01 21:43:45 +00:00
Johanna Amann
29d9b5b554 Merge remote-tracking branch 'origin/topic/jsiwek/plist-and-event-cleanup'
* origin/topic/jsiwek/plist-and-event-cleanup:
  Add comments to QueueEvent() and ConnectionEvent()
  Add methods to queue events without handler existence check
  Cleanup/improve PList usage and Event API
2019-05-01 08:41:13 -07:00
Jon Siwek
32473b85b0 Force the Broker IOSource to idle periodically
Previously, if there was always input in each Process() call, then
the Broker IOSource would never go idle and could completely starve
out a packet IOSource since it would always report readiness with
a timestamp value of the last known network_time (which prevents
selecting a packet IOSource for processing, due to incoming packets
likely having timestamps that are later).
2019-04-30 20:53:38 -07:00
Jon Siwek
c67da0a3cb Add comments to QueueEvent() and ConnectionEvent()
And also their "Fast" variants.
2019-04-29 19:21:18 -07:00
Jon Siwek
f7c1cde7c7 Remove 'dns_resolver' option, replace w/ ZEEK_DNS_RESOLVER env. var.
The later simply doesn't work well in conjunction with hostname
literals.  i.e. "google.com" (without quotes) needs to be resolved
to a set of addresses at parse-time, so if a user wishes to use a
custom resolver, we need that to be configured independently from
the order in which scripts get parsed.  Configuring 'dns_resolver'
via scripting "redef" is clearly dependent on parse order.

Note 'dns_resolver' hasn't been in any release version yet, so
I'm removing it outright, no deprecation.  The ZEEK_DNS_RESOLVER
environment variable now serves the original purpose.
2019-04-29 18:09:29 -07:00
Robin Sommer
b9dad02615 Reimplement copy().
The old implementation used the serialization framework, which is
going away. This is a new standalone implementation that should also
be quite a bit faster.

WIP: Not fully implemented and tested yet.
2019-04-29 16:52:01 -07:00
Johanna Amann
4dc6ac5382 Include all data of the server-hello random
Before we cut the first 4 bytes, which makes it impossible to recognize
several newer packets (like the hello retry).
2019-04-29 15:25:47 -04:00
Johanna Amann
27438644ae Merge remote-tracking branch 'origin/master' into topic/johanna/tls13-details 2019-04-29 13:03:12 -04:00
Jon Siwek
49908ac865 Fix parsing of hybrid IPv6-IPv4 addr literals with no zero compression 2019-04-26 19:29:40 -07:00