Commit graph

2298 commits

Author SHA1 Message Date
Jon Siwek
e82824b638 Fix various broken links in script documentation 2021-01-28 17:46:58 -08:00
Jon Siwek
3449bc81af Merge remote-tracking branch 'origin/topic/johanna/update-tls-constants-which-changed-a-lot-since-I-last-looked-which-just-shows-i-should-really-look-at-them-more-regularly-but-luckily-they-are-not-really-used-for-anything-so-it-does-not-really-matter-that-much'
* origin/topic/johanna/update-tls-constants-which-changed-a-lot-since-I-last-looked-which-just-shows-i-should-really-look-at-them-more-regularly-but-luckily-they-are-not-really-used-for-anything-so-it-does-not-really-matter-that-much:
  Update SSL consts
2021-01-21 17:33:36 -08:00
Jon Oakley
25de6f28e5 GH-1352: Added flag to stop processing SMTP headers in attached
messages
2021-01-21 14:55:10 -05:00
Johanna Amann
079d4164c0 Merge remote-tracking branch 'origin/topic/christian/fix-config-segfault'
* origin/topic/christian/fix-config-segfault:
  Btest tweak for improved type rendering in config framework errors and set types
  More precise type information in a config framework error message
  Explicitly don't support sets with multiple index types in input/config frameworks
2021-01-21 11:30:36 +00:00
Johanna Amann
8ff05bae1b Update SSL consts
There have been a bunch of mostly TLS 1.3 related changes.
2021-01-20 14:14:47 +00:00
Christian Kreibich
421639e7a7 Explicitly don't support sets with multiple index types in input/config frameworks
The input framework's Manager::IsCompatibleType() already rejected
sets with multiple index types that aren't all the same (i.e. that are
not pure). Pure ones (e.g. "set[addr,addr]") slipped through and could
cause Zeek to segfault elsewhere in the config framework due to type
comparison subtleties. Note that the ASCII reader can't read such sets
anyway, so this method now rejects sets with any kind of index-type
tuple.

In the config framework, the script-level change handler has a risky
conversion from any to set[bool], which can trigger segfaults when the
underlying set's index is a type tuple. We now prevent this code path
by ensuring it only applies to sets with a single index type.
2021-01-11 13:35:46 -08:00
Jon Siwek
321a027d07 Remove unusable/broken RocksDB code and options
The Broker RockSDB data store backend was previously unusable
and broken, so all code and options related to it are now removed.
2021-01-11 11:12:59 -08:00
Jon Siwek
7cf08d4e58 Merge remote-tracking branch 'origin/topic/neverlord/1336'
* origin/topic/neverlord/1336:
  Fix subtle race on data store initialization
2020-12-23 10:36:09 -08:00
Dominik Charousset
8d726ed07a Fix subtle race on data store initialization 2020-12-22 21:15:17 +01:00
Johanna Amann
886d7178ef TLS 1.3 changes: Address review feedback
Only minor changes, new consts, and documentation updates.

Part of GH-1335. Addresses GH-1323.
2020-12-18 10:51:36 +00:00
Johanna Amann
3c95c9a956 Fix TLS 1.3 session resumption detection.
Now we detect TLS 1.3 session resumption by looking if both sides have
the PSK extension set, which is much more exact than the previous
approach.
2020-12-15 16:34:47 +00:00
Johanna Amann
bea3075c1f TLS analyzer: change logic to track TLS 1.3 connection establishment
This commit changes the logic that is used to tracks connection
establishment - and moves it from scriptland into the core.

TLS 1.3 connection establishment is much more finnicky for us than the
establishment of earlier versions - since we cannot rely on the CCS
message anymore (which is meaningless and not sent in a lot of cases).

With this commit, the ssl_encrypted_data message gets raised for
encrypted TLS 1.3 handshake messages - which is much more correct than
the behavior before that just interpreted them as plaintext messages.

I will refine this a bit more - at the moment the connection established
event happens a bit too early - earlier than TLS 1.3 connections
actually can be estasblished.

Part of GH-1323
2020-12-14 19:51:05 +00:00
Johanna Amann
14a448cddc Update Mozilla Root Store 2020-12-09 21:46:32 +00:00
Jon Siwek
6323b0a8c2 Remove an extraneous Zeekygen-style comment 2020-12-04 19:20:57 -08:00
Johanna Amann
47ceac2491 Merge remote-tracking branch 'origin/topic/johanna/GH-348'
* origin/topic/johanna/GH-348:
  Sumstats: allow users to manage epoch manually
  Sumstats: epoch_finished was not called under certain circumstances
2020-12-04 18:49:53 +00:00
Tim Wojtulewicz
e27008ef26 GH-1184: Add 'source' field to weird log denoting where the weird was reported 2020-12-01 09:34:37 -07:00
Tim Wojtulewicz
01ec5ebdcd Reverts the regex change in dead3226a5.
The regex change broke some of the external tests. I added some more cases
to the regular email btest to hopefully cover all of the cases better.
2020-11-18 15:35:17 -07:00
Tim Wojtulewicz
a56fbe408f Merge remote-tracking branch 'christina23/master'
* christina23/master:
  Added unit tests for regex fix
  Improved regex for SMTP parsing
2020-11-17 15:01:21 -07:00
Jon Siwek
02c0b33b54 Merge remote-tracking branch 'origin/topic/jsiwek/gh-1264-ssh-host-key-fingerprints' into master
* origin/topic/jsiwek/gh-1264-ssh-host-key-fingerprints:
  Simply ssh/main.zeek by using "ssh_server_host_key" for fingerprinting
  Deprecate "ssh1_server_host_key" parameters *e* and *p*
  GH-1264: Implement "ssh_server_host_key" event
2020-11-16 11:22:37 -08:00
christina23
dead3226a5 Improved regex for SMTP parsing 2020-11-16 11:43:35 -05:00
Johanna Amann
7362f30c3a Sumstats: allow users to manage epoch manually
This change allows users to specify an epoch length of 0, which means
that the user manually has to finish the epochs. A new next_epoch
function is introduced to allow users to manually end epochs.

Addresses GH-348
2020-11-16 15:48:46 +00:00
Johanna Amann
8ffbc69a4d Sumstats: epoch_finished was not called under certain circumstances
In non-clustered mode, epoch_finished was not called when there was no
data during the epoch.

This behavior does not fit the documentation, and also is different in
cluster-mode, where epoch_finished is, indeed, called after every epoch.

This small change fixes this behavior.
2020-11-16 15:36:50 +00:00
Jon Siwek
331b94db39 Simply ssh/main.zeek by using "ssh_server_host_key" for fingerprinting 2020-11-14 08:40:27 -08:00
Brevet Vivien
3769ed6c66 [SSH] Handle SSH version 1.99
SSH can set in its identification a version 1.99 (SSH-1.99-xxx).
That means the client/server is compatible with SSHv1 and SSHv2.
So the version choice depends of the both side.

1.99 : 1.99 => 2.0
1.99 : 1.x  => 1.x
1.99 : 2.0  => 2.O

(see "Compatibility With Old SSH Versions" in RFC 4253)
2020-11-14 15:33:34 +01:00
Jon Siwek
45449dad72 Deprecate "ssh1_server_host_key" parameters *e* and *p*
They are named such that *e* is actually the modulus, not the exponent.
The replacement parameters are named *exponent* and *modulus* for
clarity.
2020-11-13 22:58:56 -08:00
Jon Siwek
bd40a97a78 GH-1264: Implement "ssh_server_host_key" event
This event provides host key fingerprints for both SSH1 and SSH2.
2020-11-13 22:58:56 -08:00
Jon Siwek
89af6f2004 Move UnknownProtocol options to init-bare.zeek
Otherwise the `unknown_protocol` event cannot be used independently
from `policy/mic/unknown-protocols.zeek`.
2020-11-11 12:58:38 -08:00
Tim Wojtulewicz
a6a4b976ec Support for additional DNS RR Type: LOC[29], SSHFP[44], NSEC3PARAM[51], custom BIND9 signaling[65534] 2020-11-11 13:35:51 -07:00
Tim Wojtulewicz
efe42bc67b Remove default_analyzer for Ethernet packet analzyer 2020-11-09 19:54:45 -07:00
Tim Wojtulewicz
cd06bf34c7 GH-1215: Remove dispatch_map from packet analysis, replace with BIF methods for registering dispatches 2020-11-02 19:03:25 +00:00
Duffy O'Craven
b32eba579d updated to mention notice_alarm.log by name. That has taken the place of the file formerly named alarm.log 2020-10-27 13:56:30 -07:00
Seth Hall
56e234a762 Changes in response to Johanna's code review. 2020-10-22 13:30:30 -04:00
Seth Hall
552a24e07c Add an option to ignore packets sourced from particular subnets.
It's implemented with a new set[subnet] option named ignore_checksums_nets.

If you populate this set with subnets, any packet with a src address within
that set of subnets will not have it's checksum validated.
2020-10-22 13:23:10 -04:00
Johanna Amann
65125121d8 Merge branch 'topic/johanna/GH-169'
* topic/johanna/GH-169:
  Make event ordering deterministic
  dump-events: try to make baseline work on all systems
  Introduce generate_all_events bif and add option to misc/dump-events

Fixes GH-169
2020-10-16 07:11:57 +00:00
Tim Wojtulewicz
1cf251d1ca Move IP and IP tunnel code from Sessions into packet analyzers 2020-10-15 12:18:30 -07:00
Johanna Amann
5f29f1dca1 Make event ordering deterministic
NetControl::init and filter_change_tracking could basically be raised in
random order.
2020-10-15 16:35:52 +00:00
Jon Siwek
6cb0cb9746 Merge remote-tracking branch 'origin/topic/seth/make-cluster-port-optional'
* origin/topic/seth/make-cluster-port-optional:
  Apply suggestions from code review
  Update scripts/base/frameworks/cluster/main.zeek
  Make defining a port number for hosts in a cluster that only connect outbound optional
2020-10-13 15:35:43 -07:00
Seth Hall
cd330c801d
Apply suggestions from code review
Co-authored-by: Jon Siwek <jsiwek@corelight.com>
2020-10-13 16:48:15 -04:00
Seth Hall
e78386d6e5
Update scripts/base/frameworks/cluster/main.zeek
Co-authored-by: Jon Siwek <jsiwek@corelight.com>
2020-10-13 16:46:26 -04:00
Jon Siwek
d62fb3ab9a Merge remote-tracking branch 'origin/topic/jsiwek/zeek-script-args'
* origin/topic/jsiwek/zeek-script-args:
  Improve zeek_script_args test case and documentation
  Apply suggestions from code review
  Add a test for script args.
  Fixed an option processing bug
  Make it possible to pass command line options through to scripts.
2020-10-13 13:34:18 -07:00
Jon Siwek
2d3b4dab74 Improve zeek_script_args test case and documentation 2020-10-13 12:40:53 -07:00
Jon Siwek
961532a8f7 Merge remote-tracking branch 'origin/topic/seth/pcap_findalldevs'
- Minor adjustments to whitespace/formatting

* origin/topic/seth/pcap_findalldevs:
  Finishing changes from code review.
  Update src/iosource/pcap/pcap.bif
  Update src/iosource/pcap/pcap.bif
  Update scripts/base/init-bare.zeek
  Update src/iosource/pcap/pcap.bif
  I accidentally missed a paren
  New bif to wrap pcap_findalldevs
2020-10-13 10:52:14 -07:00
Seth Hall
92eb7c10da Finishing changes from code review. 2020-10-13 08:35:45 -04:00
Seth Hall
dfa21d54c8
Update scripts/base/init-bare.zeek
Co-authored-by: Jon Siwek <jsiwek@corelight.com>
2020-10-13 08:12:20 -04:00
Seth Hall
7bcbc57401 New bif to wrap pcap_findalldevs 2020-10-12 12:47:23 -04:00
Seth Hall
cf8671d078 Make defining a port number for hosts in a cluster that only connect outbound optional 2020-10-12 10:46:28 -04:00
Seth Hall
2bdc56dfcd Make it possible to pass command line options through to scripts.
The feature is documented with the zeek_script_args variable in
init-bare.zeek.
2020-10-08 15:46:28 -04:00
Robin Sommer
b0bf9f02c8 Merge remote-tracking branch 'origin/topic/christian/364-logfilter-hooks' into master
(Adding a NEWS entry.)

* origin/topic/christian/364-logfilter-hooks:
  Update testing/btest/scripts/base/frameworks/logging/hooks.zeek
  Btests for log filter policy hooks
  Btest baseline updates to reflect new logging policy hooks
  Migrate existing use of filter predicates to policy hooks
  Support for log filter policy hooks
2020-10-07 08:44:50 +00:00
Jon Siwek
506d961b27 Merge branch 'logging-ascii-enable-shadow-logs' of https://github.com/awelzel/zeek into master
- Improved documentation/comment for the new option

* 'logging-ascii-enable-shadow-logs' of https://github.com/awelzel/zeek:
  logging/ascii: Support leftover log rotation in non-supervisor setups
2020-10-06 10:15:34 -07:00
Arne Welzel
1f5ab4878b logging/ascii: Support leftover log rotation in non-supervisor setups
We have a use case to rotate leftover log files in a non-supervisor
setup. There doesn't seem to be a strict requirement on supervisor
functionality. Allow enabling leftover log rotation through
LogAscii::enable_leftover_log_rotation and redef this for the
logger node in a supervisor setup individually.
2020-10-02 20:38:48 +02:00