Commit graph

1079 commits

Author SHA1 Message Date
Tim Wojtulewicz
30973af94a Merge remote-tracking branch 'origin/topic/johanna/remove-stepping-stone'
* origin/topic/johanna/remove-stepping-stone:
  Remove the Stepping Stone analyzer
2021-06-21 16:14:25 -07:00
Vlad Grigorescu
9d1e9a20e4 Expand the tests, and have email_admin email src contacts in one e-mail, and dst contacts in another. 2021-06-17 14:49:24 -05:00
Vlad Grigorescu
9d0bd96e1f Simplify the logic a big by making Notice$email_dest have a default rather than be optional 2021-06-17 10:31:04 -05:00
Vlad Grigorescu
e579497247 Merge branch 'master' into topic/vlad/page_hostnames 2021-06-11 10:18:15 -05:00
Johanna Amann
e0d284ec9f Merge branch 'logging/script-logdir' of https://github.com/kramse/zeek
* 'logging/script-logdir' of https://github.com/kramse/zeek:
  Copy of ascii-empty test, just changed path in the beginning
  Logdir: Change requested by 0xxon, no problem
  Introduce script-land variable that can be used to set logdir.

Closes GH-772
2021-06-10 12:19:15 +01:00
Henrik Kramselund Jereminsen
74561e922f Logdir: Change requested by 0xxon, no problem
Suggestion from 0xxon to look at Input Framework inspired this change
2021-06-10 10:33:26 +01:00
Henrik Kramselund Jereminsen
6bde33aca7 Introduce script-land variable that can be used to set logdir.
Addresses GH-772
2021-06-10 10:32:11 +01:00
Johanna Amann
011ac442a1 Remove the Stepping Stone analyzer
This commit removes the stepping stone analyzer. It has been deactivated
by default since at least Zeek 2.0, is dysfunctional in cluster settings
and has a bunch of other issued.

Relates to GH-1573
2021-06-03 14:28:12 +01:00
Dominik Charousset
7767c3d36c Sync new broker options, fix name inconsistencies 2021-05-25 17:22:45 +02:00
Dominik Charousset
f9cd05f00b Integrate new Broker metric exporter parameters 2021-05-24 17:20:48 +02:00
Benjamin Bannier
5b2bf374fd GH-1517: Add Geneve decap support
This patch adds the ability to decap Geneve packets to process the inner
payload. The structure of the analyzer borrows heavily from the VXLAN
analyzer.
2021-04-27 11:21:29 +02:00
Vlad Grigorescu
deae51891c Remove unneccessary @load 2021-04-26 23:02:57 -05:00
Vlad Grigorescu
ac720a1313 Add tests for e-mail actions, and cleanup the new logic a bit. 2021-04-26 23:01:34 -05:00
Vlad Grigorescu
6d0f48abfc Fixup priorities 2021-04-26 21:46:45 -05:00
Vlad Grigorescu
f8b06becd2 Add a new field to the notice, which defines where to send the e-mail 2021-04-26 21:40:27 -05:00
Vlad Grigorescu
3bb4a35200 Add page and email_admin to hostnames extension 2021-04-23 10:05:34 -05:00
Vern Paxson
0f18a01bf7 fixed out-of-range enum constant 2021-04-01 19:16:07 -07:00
Jon Siwek
7bf885b0b8 Merge remote-tracking branch 'origin/topic/vlad/expose_supervisor_rotation_func'
- Restored a deprecated version of 'supervisor_rotation_format_func'
  during merge.

* origin/topic/vlad/expose_supervisor_rotation_func:
  Rename supervisor_rotation_format_func to archiver_rotation_format_func, and expose it for non-supervised setups
2021-03-26 17:18:52 -07:00
Vlad Grigorescu
acfb21c5a6 Rename supervisor_rotation_format_func to archiver_rotation_format_func, and expose it for non-supervised setups
Closes #1463
2021-03-26 15:26:48 -05:00
Tim Wojtulewicz
4f24c02cd5 Merge remote-tracking branch 'origin/topic/vern/cpp-prep-fixes'
* origin/topic/vern/cpp-prep-fixes:
  fix for associating current scope with the name of enums; name tidying
  avoid infinite recursion in same_type() if it is analyzing recursive types
  remove iffy reliance on type punning that relies on interpreter's behavior
  fixes for propagating optimization options, and pruning script function analysis
2021-03-18 12:53:14 -07:00
Vern Paxson
b473bc48e1 remove iffy reliance on type punning that relies on interpreter's behavior 2021-03-18 08:46:18 -07:00
Vern Paxson
5a8ba8c3e5 lint fixes: ensuring functions return values, robustness to nil Val's 2021-03-18 08:21:19 -07:00
Vern Paxson
e21c0f1115 correcting some simple typos 2021-03-18 08:19:46 -07:00
Jon Siwek
6af436aad3 GH-1426: Improve handling of Broker data store creation failures
Broker::create_master() and Broker::create_clone() now return
a valid value even when there's a failure to open the backend database
(e.g. SQLite filesystem error).  In that case, the returned value can
still be passed into other data store operations, but they'll fail
immediately with an error.  Broker::is_closed() can now also be used to
determine whether the data store creation calls failed.
2021-03-06 02:32:29 -08:00
Jon Siwek
737d2c390b Support explicit disabling of file analyzers 2021-02-23 15:50:18 -08:00
Vern Paxson
f21f7b65d5 adding &is_used attribute for base scripts - not actually needed yet, but will be once optimization is added 2021-02-06 10:59:58 -08:00
Vern Paxson
4f2b138e72 removing unused assignments from base scripts 2021-02-06 10:59:37 -08:00
Jon Siwek
1ca85f0221 Merge remote-tracking branch 'origin/topic/timw/deprecation-cleanup'
Merge adjustments:
- Removed some stale str_split() references from docs
- Renumbered TypeTag enum comments
- Simplified test-case for @unload (don't need .bro files anymore)

* origin/topic/timw/deprecation-cleanup:
  Doc updates
  Fix language.init-in-anon-function btest due to changes to log filter predicates
  Remove deprecated log filter predicates for 4.1
  Remove Plugin::HookCallFunction and fix tests related to it
  Remove support for .bro script extension and BRO_ environment variables
  Remove deprecated ICMP events
  Remove some deprected methods/events from bif files
  Remove TYPE_COUNTER
  Remove all of the random single-file deprecations
  Remove all fully-deprecated files
  Update bifcl submodule to remove deprecations from generated code
2021-01-29 16:40:54 -08:00
Jon Siwek
e82824b638 Fix various broken links in script documentation 2021-01-28 17:46:58 -08:00
Christian Kreibich
4ce3bf3cd2 Remove deprecated log filter predicates for 4.1
Update the logging framework tests: since hooks operate
by name, they cannot be anonymous. I'm also dropping the &optional
attribute from the status field, since here know that the values are
actually defined, and access to an optional status field should
normally be guarded by the existence test operator.

Also includes baseline update for plugins.hooks, which picks up the
fact that the pred record field is now gone.
2021-01-27 10:52:40 -07:00
Johanna Amann
079d4164c0 Merge remote-tracking branch 'origin/topic/christian/fix-config-segfault'
* origin/topic/christian/fix-config-segfault:
  Btest tweak for improved type rendering in config framework errors and set types
  More precise type information in a config framework error message
  Explicitly don't support sets with multiple index types in input/config frameworks
2021-01-21 11:30:36 +00:00
Christian Kreibich
421639e7a7 Explicitly don't support sets with multiple index types in input/config frameworks
The input framework's Manager::IsCompatibleType() already rejected
sets with multiple index types that aren't all the same (i.e. that are
not pure). Pure ones (e.g. "set[addr,addr]") slipped through and could
cause Zeek to segfault elsewhere in the config framework due to type
comparison subtleties. Note that the ASCII reader can't read such sets
anyway, so this method now rejects sets with any kind of index-type
tuple.

In the config framework, the script-level change handler has a risky
conversion from any to set[bool], which can trigger segfaults when the
underlying set's index is a type tuple. We now prevent this code path
by ensuring it only applies to sets with a single index type.
2021-01-11 13:35:46 -08:00
Jon Siwek
321a027d07 Remove unusable/broken RocksDB code and options
The Broker RockSDB data store backend was previously unusable
and broken, so all code and options related to it are now removed.
2021-01-11 11:12:59 -08:00
Jon Siwek
7cf08d4e58 Merge remote-tracking branch 'origin/topic/neverlord/1336'
* origin/topic/neverlord/1336:
  Fix subtle race on data store initialization
2020-12-23 10:36:09 -08:00
Dominik Charousset
8d726ed07a Fix subtle race on data store initialization 2020-12-22 21:15:17 +01:00
Jon Siwek
6323b0a8c2 Remove an extraneous Zeekygen-style comment 2020-12-04 19:20:57 -08:00
Johanna Amann
47ceac2491 Merge remote-tracking branch 'origin/topic/johanna/GH-348'
* origin/topic/johanna/GH-348:
  Sumstats: allow users to manage epoch manually
  Sumstats: epoch_finished was not called under certain circumstances
2020-12-04 18:49:53 +00:00
Tim Wojtulewicz
e27008ef26 GH-1184: Add 'source' field to weird log denoting where the weird was reported 2020-12-01 09:34:37 -07:00
Johanna Amann
7362f30c3a Sumstats: allow users to manage epoch manually
This change allows users to specify an epoch length of 0, which means
that the user manually has to finish the epochs. A new next_epoch
function is introduced to allow users to manually end epochs.

Addresses GH-348
2020-11-16 15:48:46 +00:00
Johanna Amann
8ffbc69a4d Sumstats: epoch_finished was not called under certain circumstances
In non-clustered mode, epoch_finished was not called when there was no
data during the epoch.

This behavior does not fit the documentation, and also is different in
cluster-mode, where epoch_finished is, indeed, called after every epoch.

This small change fixes this behavior.
2020-11-16 15:36:50 +00:00
Jon Siwek
bd40a97a78 GH-1264: Implement "ssh_server_host_key" event
This event provides host key fingerprints for both SSH1 and SSH2.
2020-11-13 22:58:56 -08:00
Duffy O'Craven
b32eba579d updated to mention notice_alarm.log by name. That has taken the place of the file formerly named alarm.log 2020-10-27 13:56:30 -07:00
Johanna Amann
5f29f1dca1 Make event ordering deterministic
NetControl::init and filter_change_tracking could basically be raised in
random order.
2020-10-15 16:35:52 +00:00
Seth Hall
cd330c801d
Apply suggestions from code review
Co-authored-by: Jon Siwek <jsiwek@corelight.com>
2020-10-13 16:48:15 -04:00
Seth Hall
e78386d6e5
Update scripts/base/frameworks/cluster/main.zeek
Co-authored-by: Jon Siwek <jsiwek@corelight.com>
2020-10-13 16:46:26 -04:00
Seth Hall
cf8671d078 Make defining a port number for hosts in a cluster that only connect outbound optional 2020-10-12 10:46:28 -04:00
Robin Sommer
b0bf9f02c8 Merge remote-tracking branch 'origin/topic/christian/364-logfilter-hooks' into master
(Adding a NEWS entry.)

* origin/topic/christian/364-logfilter-hooks:
  Update testing/btest/scripts/base/frameworks/logging/hooks.zeek
  Btests for log filter policy hooks
  Btest baseline updates to reflect new logging policy hooks
  Migrate existing use of filter predicates to policy hooks
  Support for log filter policy hooks
2020-10-07 08:44:50 +00:00
Jon Siwek
506d961b27 Merge branch 'logging-ascii-enable-shadow-logs' of https://github.com/awelzel/zeek into master
- Improved documentation/comment for the new option

* 'logging-ascii-enable-shadow-logs' of https://github.com/awelzel/zeek:
  logging/ascii: Support leftover log rotation in non-supervisor setups
2020-10-06 10:15:34 -07:00
Arne Welzel
1f5ab4878b logging/ascii: Support leftover log rotation in non-supervisor setups
We have a use case to rotate leftover log files in a non-supervisor
setup. There doesn't seem to be a strict requirement on supervisor
functionality. Allow enabling leftover log rotation through
LogAscii::enable_leftover_log_rotation and redef this for the
logger node in a supervisor setup individually.
2020-10-02 20:38:48 +02:00
Christian Kreibich
1bd658da8f Support for log filter policy hooks
This adds a "policy" hook into the logging framework's streams and
filters to replace the existing log filter predicates. The hook
signature is as follows:

    hook(rec: any, id: Log::ID, filter: Log::Filter);

The logging manager invokes hooks on each log record. Hooks can veto
log records via a break, and modify them if necessary. Log filters
inherit the stream-level hook, but can override or remove the hook as
needed.

The distribution's existing log streams now come with pre-defined
hooks that users can add handlers to. Their name is standardized as
"log_policy" by convention, with additional suffixes when a module
provides multiple streams. The following adds a handler to the Conn
module's default log policy hook:

    hook Conn::log_policy(rec: Conn::Info, id: Log::ID, filter: Log::Filter)
            {
            if ( some_veto_reason(rec) )
                break;
            }

By default, this handler will get invoked for any log filter
associated with the Conn::LOG stream.

The existing predicates are deprecated for removal in 4.1 but continue
to work.
2020-09-30 12:32:45 -07:00