Commit graph

3113 commits

Author SHA1 Message Date
Zeke Medley
cef94832f1 Frame merge and cleanup for merge. 2019-07-25 11:19:17 -07:00
Tim Wojtulewicz
ad19f1e1bb GHI-486: Switch over to using LLVM utf8-checking code to better validate characters 2019-07-24 10:58:00 -07:00
Dev Bali
6fcb23066d Added Jon's test cases as unit tests 2019-07-23 11:59:33 -07:00
Dev Bali
66557d3178 Added optional script and redef bool to enable utf-8 in ASCII logs 2019-07-23 11:59:33 -07:00
Johanna Amann
3159577821 Merge remote-tracking branch 'origin/topic/zeke/expire-func'
* origin/topic/zeke/expire-func:
  Ignore abs-path in test.
  Report argument # type check failed on.
  Update test baseline.
  Improve func arg type checking.
  &expire_func(table, arg1, arg2, ...) + type checking.
2019-07-22 12:44:55 -07:00
Jon Siwek
07e0dba3c6 Add DPD::max_violations option
This allows one to tune the number of protocol violations to tolerate
from any given analyzer type before just disabling a given instance
of it.

Also removes the "disabled_aids" field from the DPD::Info record
since it serves no purpose: in this case, calling disable_analyzer
multiple times for the same analyzer is a no-op.
2019-07-19 13:00:02 -07:00
Jon Siwek
794748dda8 Add missing expr_name: vector_coerce 2019-07-18 19:39:16 -07:00
Zeke Medley
63a3f4e5bc Ignore abs-path in test. 2019-07-18 16:30:42 -07:00
Zeke Medley
d3574d231b Update test baseline. 2019-07-18 14:06:34 -07:00
Zeke Medley
0ca6b3e013 &expire_func(table, arg1, arg2, ...) + type checking. 2019-07-18 12:11:27 -07:00
Zeke Medley
1692be4c2d Fix bug in selective cloning. 2019-07-16 15:15:05 -07:00
Jon Siwek
db5b73cb8a Fix ref-counting bug in EnumType copy ctor
Which was used by the new ShallowClone method and causes multiple enum
declarations/definitions to now crash.  Such code is not typically
seen in scripts, but being able to define an enum in both .bif and
.bro files has historically been supported.
2019-07-15 12:42:56 -07:00
Zeke Medley
ef75092875 Use BROKER_PORT. 2019-07-12 16:00:12 -07:00
Zeke Medley
db8379edc2 Update baseline to reflect new anonymous function naming. 2019-07-12 14:07:57 -07:00
Zeke Medley
8d9355eb51 Merge branch 'master' of https://github.com/zeek/zeek into topic/zeke/closures 2019-07-12 10:33:43 -07:00
Zeke Medley
f0798c4b49 Allow serialization of closures over Broker.
anonymous-functions, their closures, can now be sent over broker.
In order to send an anonymous function the receiver must have parsed
a definition of the functon, but it need not to have been evaluated.
See testing/btest/language/closure-sending.zeek for an example of how
this can be done.

This also sends their closures as well as the closures of regular
functions.
2019-07-12 10:31:40 -07:00
Tim Wojtulewicz
ec4913ac18 GH-157: Mark some attributes as not allowed for global variables 2019-07-11 13:34:39 -07:00
Johanna Amann
1f329ad541 Merge remote-tracking branch 'origin/topic/timw/150-to-json'
* origin/topic/timw/150-to-json:
  Update submodules for JSON work
  Update unit tests for JSON logger to match new output
  Modify JSON log writer to use the external JSON library
  Update unit test output to match json.zeek being deprecated and slight format changes to JSON output
  Add proper JSON serialization via C++, deprecate json.zeek
  Add new method for escaping UTF8 strings for JSON output
  Move do_sub method from zeek.bif to StringVal class method
  Move record_fields method from zeek.bif to Val class method
  Add ToStdString method for StringVal
2019-07-11 11:17:32 -07:00
Jon Siwek
002109973d Improve stability of a unit test 2019-07-03 13:04:57 -07:00
Tim Wojtulewicz
d732fb4d4b Update unit tests for JSON logger to match new output 2019-07-02 13:14:52 -07:00
Tim Wojtulewicz
d27c846ec5 Update unit test output to match json.zeek being deprecated and slight format changes to JSON output 2019-07-02 13:14:52 -07:00
Zeke Medley
6e84a5eb8e Merge branch 'master' of https://github.com/zeek/zeek into topic/zeke/closures 2019-07-01 13:43:48 -07:00
Zeke Medley
409f27955b Call parent constructor from LambdaExpr. 2019-07-01 13:36:28 -07:00
Johanna Amann
3cdda7647f Merge remote-tracking branch 'origin/topic/jsiwek/gh-443-fix-timestamp-0-logs'
* origin/topic/jsiwek/gh-443-fix-timestamp-0-logs:
  GH-443: fix uses of timestamp 0 in cluster diagnostic logs
2019-07-01 01:29:41 -07:00
Johanna Amann
1ebd3adf20 Merge remote-tracking branch 'origin/topic/jsiwek/gh-243-wrap-up-deprecation-removal'
* origin/topic/jsiwek/gh-243-wrap-up-deprecation-removal:
  Improve deprecation warning messages
  Remove deprecated DNS events
  Remove BackDoor analyzer
  Remove InterConn analyzer
  Remove deprecated/unused irc_servers option
  Remove deprecated print_hook event
  Remove dead code: dump_used_event_handlers
  Remove unused software_version_found events
  Remove deprecated open_log_file and log_file_name functions
  Remove deprecated/unused "packet" type
  Un-deprecate anonymizer BIFs
  Un-deprecate file rotation functions
2019-07-01 01:14:29 -07:00
Zeke Medley
f47390f66a Merge branch 'master' of https://github.com/zeek/zeek into topic/zeke/closures 2019-06-28 16:22:18 -07:00
Seth Hall
9795782ecb
Merge pull request #324 from zeek/topic/jsiwek/gh-320
Improve RFB (VNC) protocol parsing
2019-06-28 17:27:16 -04:00
Jon Siwek
bc77b65b0a Merge remote-tracking branch 'origin/topic/johanna/gh-214-notice-on-workers'
* origin/topic/johanna/gh-214-notice-on-workers:
  Change notices to be processed on worker.

Fixes GH-214
2019-06-28 11:51:04 -07:00
Seth Hall
e3b080c741
Fixed a small issue due to the name changes 2019-06-28 14:49:57 -04:00
Jon Siwek
430f9a92c6 GH-443: fix uses of timestamp 0 in cluster diagnostic logs
For broker.log and cluster.log: there was a race condition.  A worker's
first IOSource that it processes is potentially Broker if there were
no packets available yet and thread scheduling happens to work out
such that network connections (inside CAF threads) become established
before we enter the main I/O loop.  Such peering establishments would
generate logs with timestamp 0 as there was not yet any code path
taken that would update network_time.

For reporter.log: any non-worker (packet-processing) node would just
unnecessarily use a timestamp of 0 for their reporter messages.
2019-06-27 23:00:42 -07:00
Jon Siwek
7dc3fca754 Remove BackDoor analyzer 2019-06-27 18:25:43 -07:00
Jon Siwek
a940cf3fb5 Remove InterConn analyzer 2019-06-27 18:05:32 -07:00
Jon Siwek
e9fefa6501 Remove deprecated print_hook event 2019-06-27 17:43:20 -07:00
Jon Siwek
bfd037989b Remove deprecated open_log_file and log_file_name functions 2019-06-27 17:43:20 -07:00
Jon Siwek
ea43c154cf Un-deprecate file rotation functions
- rotate_file
- rotate_file_by_name
- calc_next_rotate

These still have use-cases even though no longer used for our logging
functionality.  E.g. rotate_file_by_name may be used to rotate
pcap dump files.

Also the log_rotate_base_time option was marked deprecated, but still
used in the new logging framework.
2019-06-27 16:13:22 -07:00
Zeke Medley
fef8aeb123 Merge branch 'master' of https://github.com/zeek/zeek into topic/zeke/closures 2019-06-27 14:39:08 -07:00
Zeke Medley
28253b24f9 Table defaults capture closures. 2019-06-27 14:38:38 -07:00
Jon Siwek
dafc44e8b9 Merge remote-tracking branch 'origin/topic/johanna/gh-375-remove-brofile-cache'
* origin/topic/johanna/gh-375-remove-brofile-cache:
  Remove the BroFile cache

Fixes GH-375
2019-06-27 12:09:31 -07:00
Johanna Amann
5052dc03fc Remove the BroFile cache
GH-375
2019-06-26 16:32:18 -07:00
Zeke Medley
d7a73c270d Merge branch 'master' of https://github.com/zeek/zeek into topic/zeke/closures 2019-06-26 15:15:19 -07:00
Zeke Medley
8ed18ca194 Make ClosureFrame safe & cleanup
TODO: make anonymous-funcs associated with tables capture closures,
implement copy constructor for Frame, & other cleanup.
2019-06-26 15:05:57 -07:00
Seth Hall
eb690a18cc
Merge branch 'master' into topic/jsiwek/gh-320 2019-06-26 14:47:01 -04:00
Johanna Amann
44b657687a Fix creating a StringVal from std::string.
Currently, creating a StringVal from a std::string did not work with
data that contains \0 characters. This easy fix changes this - and
should also lead to a small speed increase for code using this
constructor.

This obviously means that more data might copied now in some cases that
were previously cut off at the first 0-byte. Our test-suite did not
reveal any such cases.
2019-06-26 11:41:39 -07:00
Johanna Amann
3ec9fb0f7f Change notices to be processed on worker.
In the past they were processed on the manager - which requires big
records to be sent around.

This has a potential of incompatibilities if someone relied on global
state for notice processing.

GH-214
2019-06-25 13:51:27 -07:00
Jon Siwek
f810de11fa Make a paraglob unit test parallelizable 2019-06-25 12:50:30 -07:00
Zeke Medley
670816ad48 Merge branch 'master' of https://github.com/zeek/zeek into topic/zeke/closures 2019-06-25 10:48:39 -07:00
Jon Siwek
9e9440e88d GH-427: improve default ID values shown by Zeekygen
The default value of an ID is now truly the one used to initialize it,
unaltered by any subsequent redefs.

Redefs are now shown separately, along with the expression that
modifies the ID's value.
2019-06-24 19:02:05 -07:00
Johanna Amann
31973f76ce Rename paraglob_get to paraglob_match
After going back to the original bug, I think this naming makes things a
bit more understandable.

GH-268
2019-06-24 15:44:21 -07:00
Johanna Amann
5f9a9bbcbe Merge branch 'paraglob' of https://github.com/ZekeMedley/zeek
* 'paraglob' of https://github.com/ZekeMedley/zeek:
  Add leak test to paraglob.
  Catch paraglob serialization errors in DoClone.
  Update paraglob serialization.
  Stop execution on paraglob error.
  Update paraglob submodule
  Change C++11 detection in paraglob.
  Make paraglob serializable and copyable.
  Initial paraglob integration.

I made a bunch of small changes:
 * paraglob now deals better with \0 characters
 * I rolled back the changes to Binary Serialization format,
 * there were some small formatting issue
 * the error output was slightly unsafe
 * build_unique is now in util.h.

and perhaps a few more small things.
2019-06-24 15:21:46 -07:00
Zeke Medley
9efca707b6 Add leak test to paraglob. 2019-06-24 12:40:16 -07:00