Commit graph

2220 commits

Author SHA1 Message Date
Frerich Raabe
bb879fc2aa Fixed decompose_uri choking on URIs with empty port component
A call such as

  decompose_uri("git://git.kernel.org:/pub/scm/linux/");

would raise an error along the lines of

  error in /usr/local/zeek-3.0.0/share/zeek/base/utils/urls.zeek, line 122: bad conversion to count (to_count(parts[1]) and )

This was because an empty string got passsed to the to_count()
function.

Let's improve the behaviour and rather consider the portnum component
of the URI to be uninitialized.
2020-01-24 22:24:00 +01:00
Jon Siwek
7a748526c0 Use consistent zeek_init priority for Log::create_stream calls
Typically in base scripts, Log::create_stream() is called in zeek_init()
handler with &priority=5 such that it will have already been created
in the default zeek_init() &priority=0.
2020-01-22 13:58:20 -08:00
Jon Siwek
bbdf5f8938 Move supervisor control events into SupervisorControl namespace 2020-01-22 13:28:20 -08:00
Jon Siwek
9c0d252c2b Merge branch 'master' into topic/jsiwek/supervisor 2020-01-21 12:17:56 -08:00
Jon Siwek
8247c42368 Add Supervisor documentation
Minor additions/changes to improve API I noticed along the way
2020-01-17 18:36:32 -08:00
Jon Siwek
1972190b89 Add supervisor btests 2020-01-16 19:21:53 -08:00
Jon Siwek
4d712d6203 Cleanup minor Supervisor TODOs
e.g. Mainly making default parameter for restart/destroy/status API
calls to operate on all nodes.
2020-01-14 13:41:46 -08:00
Seth Hall
d9ed76c90a
Fix warning when reading files from non-network sources
If files are being read from non-network sources, there was a warning in the SSL base scripts about missing the f$conns field.
2020-01-14 10:53:02 -05:00
Jon Siwek
0ff99c3df8 Separate supervisor node config and status data structures 2020-01-13 20:09:05 -08:00
Jon Siwek
520c6e3ebf Merge branch 'master' into topic/jsiwek/supervisor 2020-01-13 10:27:34 -08:00
Jon Siwek
5191e14eff Add stdout/stderr redirection option to supervised node config 2020-01-10 19:48:31 -08:00
Jon Siwek
263a5f404a Add cpu affinity option to supervised node config 2020-01-10 18:25:42 -08:00
Jon Siwek
00cd04b0ae Extend Supervisor Node config with list of custom scripts 2020-01-08 14:52:46 -08:00
Jon Siwek
297317b232 Organize command-line options for Supervisor filtering/inheritance
Also have stem process execv() with original command-line arguments so
that they're re-parsed and inherited correctly by supervised-nodes in
the event the stem process needs to be re-created.
2020-01-08 14:05:19 -08:00
Justin Azoff
80469a1fde fix NTLM field value access
The fields being checked for existence were not the same as the fields
being accessed.
2020-01-08 11:40:28 -05:00
Jon Siwek
6046da9993 Merge branch 'master' into topic/jsiwek/supervisor 2020-01-07 16:57:58 -08:00
Jon Siwek
b114766205 Add Supervisor::is_supervised()
And use to to avoid redef'ing the log rotation postprocessor to
"archive-log" by default since it's unlikely PATH is configured to find
that script.
2020-01-07 10:27:07 -08:00
Jon Siwek
aaa702fb4d Add option to change supervised node's working directory 2020-01-06 18:39:14 -08:00
Tim Wojtulewicz
5c85b083b6 Merge remote-tracking branch 'origin/topic/jsiwek/gh-700-packet-sources-bif'
* origin/topic/jsiwek/gh-700-packet-sources-bif:
  GH-700: add packet_sources() BIF
2019-12-05 11:58:06 -07:00
Jon Siwek
30d0b21ecc Merge remote-tracking branch 'origin/topic/dev/print-to-log'
Adjustments during merge:

- kept the UNKNOWN Log::ID as placeholder value
- changed the coverage.find-bro-logs test to check for arbitrary $path
  field values instead of just string literals
- don't force EnumVal to unsigned integer since the relevant union member
  is the signed integer and added the relevant enum values/types to
  .bif files for easier access
- compare FILE* versus file name to check for stdout equality (don't
  think it matters much, just a bit more efficient)
- minor whitespace/style tweaks

* origin/topic/dev/print-to-log:
  Added a non boolean configuration and other changes as suggested by Jon
  Allow Print Statements to be redirected to a Log# This is a combination of 3 commits.
2019-12-02 13:47:09 -08:00
Jon Siwek
6e7b101df2 GH-700: add packet_sources() BIF
Provides access to properties of packet sources, like interface or pcap
file name.
2019-11-28 09:44:38 -08:00
Dev Bali
4be6871df2 Added a non boolean configuration and other changes as suggested by Jon 2019-11-26 21:53:21 -08:00
Robin Sommer
a00f139f1c Merge remote-tracking branch 'origin/topic/jsiwek/gh-646-conn-removal'
Clarified doc strings a bit.

* origin/topic/jsiwek/gh-646-conn-removal:
  GH-646: add new "successful_connection_remove" event
2019-11-18 12:08:12 +00:00
Dev Bali
c9016f1397 Allow Print Statements to be redirected to a Log# This is a combination of 3 commits. 2019-11-13 19:59:42 -08:00
Jon Siwek
31f60853c9 GH-646: add new "successful_connection_remove" event
And switch Zeek's base scripts over to using it in place of
"connection_state_remove".  The difference between the two is
that "connection_state_remove" is raised for all events while
"successful_connection_remove" excludes TCP connections that were never
established (just SYN packets).  There can be performance benefits
to this change for some use-cases.

There's also a new event called ``connection_successful`` and a new
``connection`` record field named "successful" to help indicate this new
property of connections.
2019-11-11 19:52:59 -08:00
Jon Siwek
cc37e505e4 Merge remote-tracking branch 'origin/master' into topic/jsiwek/supervisor 2019-11-05 10:11:47 -08:00
Jon Siwek
872adda5b1 Merge branch 'topic/jsbarber/ftp-cluster-fix-patch' of https://github.com/jsbarber/zeek
Minor cleanup in merge: remove print statements and unnecessary @if
directive.

* 'topic/jsbarber/ftp-cluster-fix-patch' of https://github.com/jsbarber/zeek:
  Publish ftp_data_expected updates to other workers for synchronization
2019-11-04 17:31:59 -08:00
Jeff Barber
d698bddc7d Publish ftp_data_expected updates to other workers for synchronization 2019-10-30 15:50:22 -06:00
Robin Sommer
9d7c9f37d4 Merge branch 'Fix_http_build_url' of ssh://github.com/Olerdrive/zeek
I changed the format string of the port to %d and added a test.
2019-10-28 09:32:32 +00:00
Alexey Mokeev
6ba1d1dcd3 Make http::build_url work correctly 2019-10-27 11:17:53 +03:00
Jon Siwek
81ab0b0d05 Use explicit path name for NTP log stream
For consistency (we do this for all other logs) and just to avoid
the extra path function calls.
2019-10-25 10:38:58 -07:00
Jon Siwek
29f386e388 Implement minimal supervised cluster configuration
More aspects of the cluster configuration to get fleshed out later,
but a basic cluster like one would use for a live deployment
can now be instantiated and run under supervision.  The new
clusterized-pcap-processing supervisor mode is also not done yet.
2019-10-23 17:37:53 -07:00
Jon Siwek
25a8ba99fa Change supervisor event response topic names
They now include the request ID at the end of their topic, to possibly
help with subscription filtering.
2019-10-21 09:16:26 -07:00
Jon Siwek
7a6355f64f Change supervisor event API to use strings for request IDs 2019-10-21 09:12:50 -07:00
Jon Siwek
773b39e52e Finish implementing supervisor infrastructure
The process hierarchy and all supervisor control commands are now
working (e.g. status, create, destroy, restart), but nodes are
not currently spawned with the desired configuration parameters so
they don't yet operate as real cluster nodes (e.g. worker, logger,
manager, proxy).
2019-10-18 17:57:20 -07:00
Jon Siwek
7c08488dfc Add skeleton logic for handling supevisor control messages 2019-10-16 15:13:15 -07:00
Jon Siwek
4c066ca914 Remove unnecessary &optional from a Config::set_value() parameter 2019-10-15 17:46:05 -07:00
Jon Siwek
e46cf88435 Add Supervisor BIF/event API skeleton 2019-10-15 12:59:45 -07:00
Jon Siwek
052feacbda GH-618: add "tcp_options" event containing TCP option values 2019-10-03 18:59:02 -07:00
Jon Siwek
aeef4bf030 Merge branch 'topic/jgras/dpd-late-match' of https://github.com/J-Gras/zeek
* 'topic/jgras/dpd-late-match' of https://github.com/J-Gras/zeek:
  Improve dpd_late_match event generation.
  Improve logging of speculative service.
  Update test-all-policy script.
  Add speculative service script.
  Allow to handle late DPD matches.
2019-09-17 11:17:41 -07:00
Jan Grashoefer
5901b6d33c Allow to handle late DPD matches.
If dpd_match_only_beginning is disabled, matches of protocol signatures
can be handeld using protocol_late_match. To prevent further matching in
this case, dpd_late_match_stop may be activated.
2019-08-22 19:00:11 +02:00
Jon Siwek
2d7c926291 GH-545: add "addl" parameter to flow_weird and net_weird events 2019-08-20 22:45:22 -04:00
Jon Siwek
9ccf3549fd Merge remote-tracking branch 'origin/topic/timw/360-log-gzip'
* origin/topic/timw/360-log-gzip:
  Add new LogAscii::gzip_file_extension option.

Adjusted documentation during merge.
2019-08-13 13:49:45 -07:00
Jon Siwek
31d30bb47e GH-535: fix typo of "C_HESIOD" in DNS::classes
Fixes GH-535
2019-08-12 13:32:28 -07:00
Jon Siwek
acd1be6e7b Merge branch 'export_intel_events' of https://github.com/mauropalumbo75/zeek
* 'export_intel_events' of https://github.com/mauropalumbo75/zeek:
  minor restyle and add comments
  add an empty read_error event to the intel framework (in the export block, so that users can implement further checks with it)
  move event Intel::read_entry to export block

Adjusted whitespace in merge.
2019-08-09 09:36:19 -07:00
Mauro Palumbo
f93e41fd3f minor restyle and add comments 2019-08-07 10:32:36 +02:00
Mauro Palumbo
8e418d3c7b add an empty read_error event to the intel framework (in the export block, so that users can implement further checks with it) 2019-08-06 12:20:47 +02:00
Mauro Palumbo
1011abd5e0 move event Intel::read_entry to export block 2019-08-06 11:17:34 +02:00
Jon Siwek
35c42b4b09 GH-474: change MQTT::max_payload_size to be a runtime option 2019-08-05 18:11:54 -07:00
Johanna Amann
0f96a9dedf Disable MQTT by default
To enable MQTT, one has to load policy/scripts/mqtt. Like with smb in
2.5, the consts are loaded by default.
2019-08-05 17:04:39 -07:00