Commit graph

10464 commits

Author SHA1 Message Date
Robin Sommer
edaeb2f308 Merge remote-tracking branch 'origin/topic/jsiwek/gh-837-improve-broker-mismatch-errors'
* origin/topic/jsiwek/gh-837-improve-broker-mismatch-errors:
  GH-837: Add test cases for mismatched Broker SSL configs
  GH-837: emit Reporter errors for Broker errors
  Fix incorrect/missing Broker error status code numbers
2020-07-17 10:09:05 +00:00
Jon Siwek
85fbdaf429 GH-837: Add test cases for mismatched Broker SSL configs 2020-07-16 18:32:06 -07:00
Jon Siwek
c84a51ac09 GH-837: emit Reporter errors for Broker errors
Instead of only writing them in broker.log, which may be easy to
overlook.
2020-07-16 18:07:00 -07:00
Jon Siwek
7e03233d55 Fix incorrect/missing Broker error status code numbers 2020-07-16 18:03:41 -07:00
Jon Siwek
fd8db24daf Merge remote-tracking branch 'origin/topic/jsiwek/gh-1062-fix-tag-subtypes'
* origin/topic/jsiwek/gh-1062-fix-tag-subtypes:
  GH-1062: fix integer conversion errors related to Tag subtypes
2020-07-14 19:11:09 -07:00
Jon Siwek
7f2aae1721 GH-1062: fix integer conversion errors related to Tag subtypes 2020-07-14 15:55:28 -07:00
Jon Siwek
da105b3bf9 Update submodule(s)
[nomail]
2020-07-14 14:58:50 -07:00
Johanna Amann
da2c968ba0 Merge remote-tracking branch 'origin/topic/jsiwek/gh-1024-broker-store-handle-type-checks'
* origin/topic/jsiwek/gh-1024-broker-store-handle-type-checks:
  Improve Broker store API's handling of invalid arguments
  Add builtin_exception() functions
  GH-1024: fix crash on passing wrong types to Broker store API

This was a bit of a bigger merge since Zeek changed inbetween the time
of the PR and me actually merging it.

I put the new functions into the zeek::detail namespace -- since it
seems unlikely that those will be used by something external.

I also renamed them to fit better with the naming scheme of the new
error functions.

Fixes GH-1024
2020-07-14 00:05:09 +00:00
Tim Wojtulewicz
37dbca965e Update submodule(s)
[nomail]
2020-07-13 12:39:32 -07:00
Tim Wojtulewicz
75cb35e2d9 Merge remote-tracking branch 'origin/topic/jsiwek/gh-1019-deprecate-icmp-conn'
* origin/topic/jsiwek/gh-1019-deprecate-icmp-conn:
  GH-1019: deprecate icmp_conn params for ICMP events
2020-07-13 12:12:44 -07:00
Tim Wojtulewicz
64af3ec67a Merge remote-tracking branch 'origin/topic/jsiwek/deprecation-improvements'
* origin/topic/jsiwek/deprecation-improvements:
  Fix wrong frame offsets for locals of alternate event/hook prototypes
  Add deprecation expression to deprecated prototype/parameter messages
  Improve "use of deprecated prototype" warning message
  Emit deprecation warning for use of &deprecated function parameters
2020-07-13 12:11:12 -07:00
Jon Siwek
034be246ba Update submodule(s)
[nomail]
2020-07-13 09:20:30 -07:00
Jon Siwek
11df252f5f Update submodule(s)
[nomail]
2020-07-10 16:52:18 -07:00
Jon Siwek
ba0ba7c684 Add more error checks to shadow log parsing
i.e. Coverity warns about possible use of ftell() negative return value
2020-07-10 11:21:49 -07:00
Jon Siwek
6908d1b919 GH-1019: deprecate icmp_conn params for ICMP events
Previously, a single `icmp_conn` record was built per ICMP "connection"
and re-used for all events generated from it.  This may have been a
historical attempt at performance optimization, but:

  * By default, Zeek does not load any scripts that handle ICMP events.

  * The one script Zeek ships with that does handle ICMP events,
    "detect-traceroute", is already noted as being disabled due to
    potential performance problems of doing that kind of analysis.

  * Re-use of the original `icmp_conn` record tends to misreport
    TTL and length values since they come from original packet instead
    of the current one.

  * Even if we chose to still re-use `icmp_conn` records and just fill
    in a new TTL and length value each packet, a user script could have
    stored a reference to the record and not be expecting those values
    to be changed out from underneath them.

Now, a new `icmp_info` record is created/populated in all ICMP events
and should be used instead of `icmp_conn`.  It also removes the
orig_h/resp_h fields as those are redundant with what's already
available in the connection record.
2020-07-10 11:06:28 -07:00
Jon Siwek
20294d372c Fix wrong frame offsets for locals of alternate event/hook prototypes
Local frame offsets were being assigned based on number of the alternate
prototype's parameters, which may end up having less total parameters
than the canonical prototype, causing the local value to incorrectly
overwrite an event/hook argument value.
2020-07-10 10:59:12 -07:00
Jon Siwek
8597b998bb Add deprecation expression to deprecated prototype/parameter messages 2020-07-10 10:57:57 -07:00
Jon Siwek
ac1ec7668d Improve "use of deprecated prototype" warning message
The location information now points out the place of the deprecated
prototype instead of the location where the ID was initially declared
(which may not itself be a deprecated prototype).
2020-07-10 10:57:32 -07:00
Jon Siwek
26b3d406b4 Emit deprecation warning for use of &deprecated function parameters
Particularly, this is meant for using &deprecated on canonical
event/hook prototype parameters to encourage users to create handlers
to another, non-deprecated prototype.  i.e. for canonical prototypes,
we may not always want to put &deprecated directly on the prototype
itself since that signals deprecation of the ID entirely.
2020-07-10 10:57:05 -07:00
Jon Siwek
39f549ed68 Merge remote-tracking branch 'origin/topic/jsiwek/3.2-supervisor'
* origin/topic/jsiwek/3.2-supervisor:
  Integrate Supervisor code review suggestions
  Add Supervisor::{stdout,stderr}_hook
  Add Log::rotation_format_func and Log::default_rotation_dir options
  Allow Zeek function equality comparisons
  Implement leftover log rotation/archival for supervised nodes
  Move some supervisor structures to detail namespace
  Improve default supervisor stdout/stderr handling
  Factor Supervisor initialization data to separate struct from Stem
  Move ParentProcessCheckTimer to detail namespace
2020-07-09 14:13:26 -07:00
Jon Siwek
7669f560d1 Integrate Supervisor code review suggestions 2020-07-09 13:56:11 -07:00
Jon Siwek
48362cc0c0 Merge remote-tracking branch 'origin/topic/jsiwek/remove-lexer-dead-code'
* origin/topic/jsiwek/remove-lexer-dead-code:
  scan.l: Remove "constant" did_module_restore logic
2020-07-08 17:47:07 -07:00
Arne Welzel
93e7f40b70 scan.l: Remove "constant" did_module_restore logic
Doesn't seem like did_module_restore has any effect on the code flow.
2020-07-08 16:20:57 -07:00
Jon Siwek
d15dd37b84 Merge remote-tracking branch 'origin/topic/jsiwek/fix-ci-freebsd-swig'
* origin/topic/jsiwek/fix-ci-freebsd-swig:
  Fix FreeBSD CI script to install right SWIG package
2020-07-08 12:25:31 -07:00
Jon Siwek
3c1f5914c6 Fix FreeBSD CI script to install right SWIG package 2020-07-08 11:07:56 -07:00
Robin Sommer
1fe822a745 Merge remote-tracking branch 'origin/topic/jsiwek/fix-while-stmt-ctor'
* origin/topic/jsiwek/fix-while-stmt-ctor:
  Fix WhileStmt to call Stmt(Tag) ctor
2020-07-08 15:41:50 +00:00
Robin Sommer
776250fd8c Merge remote-tracking branch 'origin/topic/jsiwek/gh-928-realpath-load-dedup'
* origin/topic/jsiwek/gh-928-realpath-load-dedup:
  GH-928: use realpath() instead of inode to de-duplicate scripts
2020-07-08 15:40:24 +00:00
Jon Siwek
7bcba16837 Update submodule(s)
[nomail]
2020-07-08 08:33:23 -07:00
Jon Siwek
10709c627b Add Supervisor::{stdout,stderr}_hook
These allow capturing/handling the stdout/stderr of child processes
via Zeek scripts.
2020-07-07 20:21:32 -07:00
Jon Siwek
a06ef66edc Add Log::rotation_format_func and Log::default_rotation_dir options
These may be redefined to customize log rotation path prefixes,
including use of a directory.  File extensions are still up to
individual log writers to add themselves during the actual rotation.

These new also allow for some simplication to the default
ASCII postprocessor function: it eliminates the need for it doing an
extra/awkward rename() operation that only changes the timestamp format.

This also teaches the supervisor framework to use these new options
to rotate ascii logs into a log-queue/ directory with a specific
file name format (intended for an external archiver process to
monitor separately).
2020-07-07 18:42:37 -07:00
Jon Siwek
6e67a40d24 Allow Zeek function equality comparisons
Equality between two functions acts like a pointer comparison.
2020-07-07 18:39:23 -07:00
Jon Siwek
11949ce37a Implement leftover log rotation/archival for supervised nodes
This helps prevent a node from being killed/crashing in the middle
of writing a log, restarting, and eventually clobbering that log
file that never underwent the rotation/archival process.

The old `archive-log` and `post-terminate` scripts as used by
ZeekControl previously implemented this behavior, but the new logic is
entirely in the ASCII writer.  It uses ".shadow" log files stored
alongside the real log to help detect such scenarios and rotate them
correctly upon the next startup of the Zeek process.
2020-07-07 18:39:23 -07:00
Jon Siwek
a46e24091a Move some supervisor structures to detail namespace
* StemHandle -> detail::SupervisorStemHandle
* SupervisedNode -> detail::SupervisedNode
* Supervisor::Node -> detail::SupervisorNode
2020-07-07 18:39:23 -07:00
Jon Siwek
9087621714 Improve default supervisor stdout/stderr handling
The stdout/stderr of child processes is now redirected over a pipe back
to the supervisor process so that it can prefix the output with
the name of the emitting node.
2020-07-07 18:39:23 -07:00
Jon Siwek
0acd5fea0c Factor Supervisor initialization data to separate struct from Stem 2020-07-07 18:38:38 -07:00
Jon Siwek
d00b3a8daa Move ParentProcessCheckTimer to detail namespace 2020-07-07 18:38:38 -07:00
Jon Siwek
320b14ff09 GH-928: use realpath() instead of inode to de-duplicate scripts
Duplicate script `@load` directives are now detected by comparing
against canonical paths formed by realpath().  This fixes the previous,
unexpected behavior of treating scripts that hardlink to same
inode as duplicates: such links will now be loaded as distinct scripts
since their canonical path differs.
2020-07-07 17:29:52 -07:00
Jon Siwek
7f347a3b1e Update submodule(s)
[nomail]
2020-07-07 12:47:30 -07:00
Jon Siwek
7b15b82009 Merge remote-tracking branch 'origin/topic/timw/1040-str-split'
* origin/topic/timw/1040-str-split:
  GH-1040: Add zero-indexed version of str_split
2020-07-06 21:06:51 -07:00
Jon Siwek
09483619ef Merge remote-tracking branch 'origin/topic/timw/1041-compress-path'
* origin/topic/timw/1041-compress-path:
  GH-1041: Move compress_path to a bif that uses normalize_path
2020-07-06 20:41:05 -07:00
Tim Wojtulewicz
e6871ed3e9 GH-1040: Add zero-indexed version of str_split 2020-07-06 17:05:40 -07:00
Jon Siwek
76a38caf34 Fix WhileStmt to call Stmt(Tag) ctor
Also removed Stmt() default ctor to help ensure derived classes
initialize the Stmt tag (and other members).
2020-07-06 12:22:23 -07:00
Tim Wojtulewicz
560ee0c05e GH-1041: Move compress_path to a bif that uses normalize_path 2020-07-06 11:43:44 -07:00
Jon Siwek
e891c310fb Update submodule(s)
[nomail]
2020-07-06 10:31:24 -07:00
Robin Sommer
3d3d5e7eb4 Merge remote-tracking branch 'origin/topic/jsiwek/add-backtrace-bif'
(Added a Doxygen comment.)

* origin/topic/jsiwek/add-backtrace-bif:
  Add backtrace() and print_backtrace()
2020-07-06 14:56:28 +00:00
Jon Siwek
8d4df512d0 Update submodule(s)
[nomail]
2020-07-04 00:04:19 -07:00
Jon Siwek
9770acd029 Update submodule(s)
[nomail]
2020-07-03 14:42:11 -07:00
Jon Siwek
61b52b54ba Fix --enable-mobile-ipv6 build 2020-07-03 14:30:13 -07:00
Jon Siwek
3a59948238 Fix namespace of GetCurrentLocation() to zeek::detail 2020-07-03 14:18:02 -07:00
Jon Siwek
a1c19840ce Add backtrace() and print_backtrace() 2020-07-03 14:09:31 -07:00