Commit graph

339 commits

Author SHA1 Message Date
Jon Siwek
a994be9eeb Merge remote-tracking branch 'origin/topic/seth/zeek_init'
* origin/topic/seth/zeek_init:
  Some more testing fixes.
  Update docs and tests for bro_(init|done) -> zeek_(init|done)
  Implement the zeek_init handler.
2019-04-19 11:24:29 -07:00
Seth Hall
8cefb9be42 Implement the zeek_init handler.
Implements the change and a test.
2019-04-14 08:37:35 -04:00
Daniel Thayer
9cbeb44986 Merge remote-tracking branch 'upstream/master' 2019-04-14 01:35:50 -05:00
Jon Siwek
b6862c5c59 Add methods to queue events without handler existence check
Added ConnectionEventFast() and QueueEventFast() methods to avoid
redundant event handler existence checks.

It's common practice for caller to already check for event handler
existence before doing all the work of constructing the arguments, so
it's desirable to not have to check for existence again.

E.g. going through ConnectionEvent() means 3 existence checks:
one you do yourself before calling it, one in ConnectionEvent(), and then
another in QueueEvent().

The existence check itself can be more than a few operations sometimes
as it needs to check a few flags that determine if it's enabled, has
a local body, or has any remote receivers in the old comm. system or
has been flagged as something to publish in the new comm. system.
2019-04-11 20:30:25 -07:00
Jon Siwek
8bc65f09ec Cleanup/improve PList usage and Event API
Majority of PLists are now created as automatic/stack objects,
rather than on heap and initialized either with the known-capacity
reserved upfront or directly from an initializer_list (so there's no
wasted slack in the memory that gets allocated for lists containing
a fixed/known number of elements).

Added versions of the ConnectionEvent/QueueEvent methods that take
a val_list by value.

Added a move ctor/assign-operator to Plists to allow passing them
around without having to copy the underlying array of pointers.
2019-04-11 20:30:25 -07:00
Daniel Thayer
18bd74454b Rename all scripts to have ".zeek" file extension 2019-04-11 21:12:40 -05:00
Jon Siwek
78dcbcc71a Merge remote-tracking branch 'origin/topic/jsiwek/flowbuffer-policy'
* origin/topic/jsiwek/flowbuffer-policy:
  Use a default binpac flowbuffer policy

Added options to tune binpac flowbuffer policy
2019-04-10 09:55:27 -07:00
Jon Siwek
5e9b119a08 Use a default binpac flowbuffer policy 2019-04-05 12:35:50 -07:00
Jon Siwek
0790c1c559 Merge remote-tracking branch 'origin/topic/jsiwek/gh-211'
* origin/topic/jsiwek/gh-211:
  GH-208: change invalid subnet expressions to a runtime error
  GH-211: improve consistency of how scripting errors are handled

Removed the 'allow_init_errors' option.
2019-02-07 10:33:52 -06:00
Jon Siwek
67484a90fa GH-211: improve consistency of how scripting errors are handled
Scripting errors/mistakes now consistently generate a runtime error
which have the behavior of unwinding the call stack all the way out of
the current event handler.

Before, such errors were not treated consistently and either aborted
the process entirely or emitted a message while continuing to execute
subsequent statements without well-defined behavior (possibly causing
a cascade of errors).

The previous behavior also would only unwind out of the current
function (if within a function body), not out the current event
handler, which is especially problematic for functions that return
a value: the caller is essentially left a mess with no way to deal
with it.

This also changes the behavior of the startup/initialization process
to abort if there's errors during bro_init() rather than continue one
to the main run loop.  The `allow_init_errors` option may change this
new, default behavior.
2019-01-30 11:20:09 -06:00
Johanna Amann
d4f7dae768 Merge remote-tracking branch 'origin/topic/jsiwek/val_mgr'
* origin/topic/jsiwek/val_mgr:
  Pre-allocate and re-use Vals for bool, int, count, enum and empty string
  Preallocate booleans and small counts

I added a tiny change to CompHash to make sure that nothing messes this
up in the future.
2019-01-18 15:17:34 -08:00
Jon Siwek
38cb3dbf2e GH-188: fix crash when shutting down with pending reporter errors
This fixes some use-after-free issues in the shutdown order of
various systems: I/O, plugin, logging, and reporter systems may
interact during shutdown if there's errors emitted (or maybe just
still pending) during the shutdown.

Fixes GH-188
2019-01-14 15:40:17 -06:00
Jon Siwek
2982765128 Pre-allocate and re-use Vals for bool, int, count, enum and empty string 2019-01-09 18:29:23 -06:00
Justin Azoff
dcbef9cbe3 Preallocate booleans and small counts
Like PortManager, preallocate Vals for booleans and counts < 4096
2019-01-07 13:40:17 -06:00
Jon Siwek
cbaab3d4fa GH-199: change bro --help exit status from 1 to 0
Fixes #199
2018-10-31 22:28:38 -05:00
Jon Siwek
60d0343845 Clarify 'old_comm_usage_is_ok' error message 2018-09-04 09:26:12 -05:00
Jon Siwek
611c00a605 Merge remote-tracking branch 'origin/topic/johanna/bit-1976'
* origin/topic/johanna/bit-1976:
  Allow event/function definitions to be wrapped in directives.

Fixed to work with attributes (e.g. &priority).
2018-08-29 18:28:54 -05:00
Jon Siwek
67524f26d5 Immediately apply broker subscriptions made during bro_init()
Otherwise that's begging for unit test failures due to races
2018-08-10 17:18:21 -05:00
Jon Siwek
c11039cb73 Make old comm. system usages an error unless old_comm_usage_is_ok is set 2018-06-15 17:15:46 -05:00
Jon Siwek
1b4e0116f4 Allow BRO_DEFAULT_LISTEN_ADDRESS to control broker listen address
This environment variable is now set to listen only on IPv4 loopback
when running unit tests (instead of using the default INADDR_ANY).

This also moves some of the @loads out from init-bare.bro into a new
init-frameworks-and-bifs.bro in order to better support calling BIFs
(like `getenv`) from variable initializations in those particular
frameworks.
2018-06-01 15:38:11 -05:00
Robin Sommer
fe7e1ee7f0 Merge topic/actor-system throug a squashed commit. 2018-05-18 22:39:23 +00:00
Robin Sommer
8df77aee09 Merge remote-tracking branch 'origin/topic/jsiwek/bit-1909'
I've moved the resizing loop into a static method of RecordVal.

* origin/topic/jsiwek/bit-1909:
  BIT-1909: fix invalid redef'd record field accesses
2018-04-27 10:35:37 -07:00
Jon Siwek
8152508330 BIT-1909: fix invalid redef'd record field accesses 2018-04-11 16:23:26 -05:00
Johanna Amann
a8c0580b45 Merge branch 'topic/johanna/openssl-1.1'
* topic/johanna/openssl-1.1:
  Fix recently introduced double free in OpenSSL code.
  Adjust coding style & fix test failures.
  Adapt most of the X509 support to OpenSSL 1.1
2018-01-30 14:33:32 -08:00
Jon Siwek
1e4964de77 Preallocate all possible PortVals.
The performance benefit is small (maybe ~1% at most), however, it's a
trivial change without downsides.
2017-12-11 15:29:28 -06:00
Hilko Bengen
ea4cf7dbe9 Adapt most of the X509 support to OpenSSL 1.1 2017-12-06 00:07:49 +01:00
Johanna Amann
7aa219758c Merge remote-tracking branch 'origin/master' into topic/johanna/ocsp-new 2017-03-16 12:28:08 -07:00
Pete
05746ab7fc print version string to stdout on --version
When running a *nix command from the prompt, and output is expected, that output should be sent to stdout, not stderr.  The --version option is such a case.  The outputted version string is not an indication of error or a diagnostic output; it is the expected output, thus should follow standard conventions and be output to stdout..
2017-03-12 12:53:12 -04:00
Johanna Amann
e1bcc4509f OCSP: rewrite events and data structures.
Instead of having a big event, that tries to parse all the data into a
huge datastructure, we do the more common thing and use a series of
smaller events to parse requests and responses.

The new events are:

ocsp_request -> raised for an ocsp request, giving version and requestor

ocsp_request_certificate -> raised n times per request, once per cert

ocsp_response_status -> raised for each ocsp response, giving status

ocsp_response_bytes -> raised for each ocsp response with information

ocsp_response_certificate -> raised for each cert in an ocsp response
2017-02-10 17:03:18 -08:00
Johanna Amann
9c6cebf324 OCSP: a bunch of whitespace fixes 2017-02-10 17:03:04 -08:00
Johanna Amann
dfc871f831 Merge remote-tracking branch 'origin/master' into topic/johanna/ocsp 2017-02-08 10:35:12 -08:00
Robin Sommer
982612b66f Merge remote-tracking branch 'origin/topic/johanna/remove-z'
BIT-1663 #merged

* origin/topic/johanna/remove-z:
  Remove -z/--analysis option.
2016-08-05 15:26:48 -07:00
Johanna Amann
39d811ff06 Remove code for XML serialization.
Code was apparently defunct and was not user-accessible for a long time.
2016-08-05 14:02:28 -07:00
Johanna Amann
b7ab6a5332 Remove -z/--analysis option.
It was only used for one special case that is not commonly used by
anyone.
2016-08-04 17:44:38 -07:00
Robin Sommer
d7c10ca7c3 Removing OpenSSL dependency for plugins.
Compiling a plugin required having access to OpenSSL headers because
they were pulled in by Bro headers that the plugin had to include.
Removinng then OpenSSL dependency from those Bro headers.

I'm also reverting a4e5591e. This is a different fix for the same
problem, and reverting that commit gives us a test case. :-)
2016-08-01 12:32:27 -07:00
Johanna Amann
499ed5b566 Remove the -K/-J options for setting keys.
The options were never really used and do not seem especially useful;
initialization with a seed file still works.

This also fixes a bug with the initialization of the siphash key.
2016-07-13 16:57:53 -07:00
Johanna Amann
e1218cc7fa Change Hashing from H3 to Siphash.
This commit mostly changes the hash function that is used for Internal
hashing of data < 36 bytes from H3 to Siphash. This change is motivated
by the fact that it turns out that H3 apparently does not deliver a very
good source of data uniqueness; running HLL with H3 as a hashing
function results in quite poor results (up to of 75% off in my tests).
In difference, running HLL with Siphash (or HMAC-MD5) changes this
factor to ~2%.

This also fixes a long-standing bug in Hash.h which truncated our hash
values to 32 bit on most machines.

Furthermore, it once again fixes a problem with the Rank function in
HLL.
2016-07-13 06:44:51 -07:00
Johanna Amann
b2371752e4 Merge remote-tracking branch 'origin/topic/dnthayer/ticket1616'
I changed the patch slightly - now debug.log is only created, if a debug
stream is enabled.

BIT-1616 #merged

* origin/topic/dnthayer/ticket1616:
  Don't create debug.log immediately upon startup
2016-06-07 11:09:22 -07:00
Daniel Thayer
0cae2ca003 Don't create debug.log immediately upon startup
Instead of creating the debug.log immediately when bro starts,
now it is created only after the debug streams are enabled.
This avoids having an empty log being created when it shouldn't be,
in usages such as "bro -h", "bro -v", or "bro -B help" (and also
when using broctl, which needs to run "bro -v").
2016-06-04 17:16:25 -05:00
Seth Hall
6d836b7956 More stats improvements
Broke out the stats collection into a bunch of new Bifs
in stats.bif.  Scripts that use stats collection functions
have also been updated.  More work to do.
2016-01-07 16:20:24 -05:00
Johanna Amann
da9b5425e4 Merge remote-tracking branch 'origin/master' into topic/johanna/ocsp 2015-12-14 16:05:41 -08:00
Robin Sommer
952c64a793 Delaying BinPAC initializaton until afte plugins have been activated. 2015-12-04 16:40:43 -08:00
Robin Sommer
36b5a4db08 Merge branch 'master' of https://github.com/knielander/bro
I reworked this a bit:

    - Moved the globals into a new Pcap::* namespace, and renamed them
      slightly.

    - Moved the definitions of the globals into pcap/const.bif.

    - Also moved the existing 'snaplen' into Pcap::* and removed
      SnapLen() from the PktSrc API (it's really a pcap thing).

    - Likewise moved the existing functions precompile_pcap_filter,
      install_pcap_filter, and pcap_error, into Pcap::*.

    - Did some more refactoring for the pcap code.

* 'master' of https://github.com/knielander/bro:
  Refactored patch (removed options, less ambiguous name)
  Allow Bro to run in fanout mode.
  Allow libpcap buffer size to be set manually.
  Allow Bro to run in fanout mode.
  Allowed libpcap buffer size to be set via configuration.
2015-08-30 22:09:32 -07:00
Kris Nielander
ba4c816b0e Refactored patch (removed options, less ambiguous name) 2015-08-24 23:45:21 +02:00
Kris Nielander
d8c9b7255e Allow Bro to run in fanout mode. 2015-08-09 22:45:23 +02:00
Kris Nielander
f5429ee794 Allow libpcap buffer size to be set manually. 2015-08-09 22:45:09 +02:00
Liang Zhu
e9f028be4c Merge remote-tracking branch 'origin/master' into topic/liangzhu/analyzer-ocsp 2015-07-28 13:47:21 -07:00
Robin Sommer
3957091e1b Renaming config.h to bro-config.h.
A couple times now I had this conflicting with files of the same name
in other projects.
2015-07-28 11:57:04 -07:00
Liang Zhu
da122a6a14 Merge remote-tracking branch 'origin/master' into topic/liangzhu/analyzer-ocsp 2015-07-02 16:48:51 -07:00
Daniel Thayer
368c1463ab Restore the --load-seeds cmd-line option
Also enabled the short options -G/-H for --load-seeds/--save-seeds.
2015-06-23 13:16:31 -05:00