Commit graph

3318 commits

Author SHA1 Message Date
Robin Sommer
3ebfcdf0ae Add test creating multiple plugins with load dependencies.
If we load plugins purely alphabetically, the 1st Zeek run in the test
will success while the 2nd will fail.
2020-11-24 16:59:11 +00:00
Vern Paxson
6894e4c5e1 updates from test suite 2020-11-19 17:13:36 -08:00
Vern Paxson
c42586af2c inlining of Zeek script functions 2020-11-19 16:16:59 -08:00
Tim Wojtulewicz
01ec5ebdcd Reverts the regex change in dead3226a5.
The regex change broke some of the external tests. I added some more cases
to the regular email btest to hopefully cover all of the cases better.
2020-11-18 15:35:17 -07:00
Tim Wojtulewicz
a56fbe408f Merge remote-tracking branch 'christina23/master'
* christina23/master:
  Added unit tests for regex fix
  Improved regex for SMTP parsing
2020-11-17 15:01:21 -07:00
Jon Siwek
824ff81bf6 Merge remote-tracking branch 'origin/topic/johanna/1033'
- Fixed signed/unsigned comparison compiler warning
- Did other minor changes to address efficiency nitpicks

* origin/topic/johanna/1033:
  BrokerStore <-> Zeek Tables: support complex indices
2020-11-17 13:34:26 -08:00
Johanna Amann
44ae8f9e8f BrokerStore <-> Zeek Tables: support complex indices
This change adds support for complex indexes for sets and tables. With
this change, sets with indexes like:

set[string, count, count]

will function. Before this change, Zeek raised an error message in these
cases.

Addresses GH-1033
2020-11-17 14:48:10 +00:00
Jon Siwek
6ed4850a26 Merge remote-tracking branch 'origin/topic/jsiwek/gh-352-improve-sql-injection-regex' into master
* origin/topic/jsiwek/gh-352-improve-sql-injection-regex:
  GH-352: Improve HTTP::match_sql_injection_uri regex
2020-11-16 11:51:04 -08:00
Jon Siwek
02c0b33b54 Merge remote-tracking branch 'origin/topic/jsiwek/gh-1264-ssh-host-key-fingerprints' into master
* origin/topic/jsiwek/gh-1264-ssh-host-key-fingerprints:
  Simply ssh/main.zeek by using "ssh_server_host_key" for fingerprinting
  Deprecate "ssh1_server_host_key" parameters *e* and *p*
  GH-1264: Implement "ssh_server_host_key" event
2020-11-16 11:22:37 -08:00
christina23
bb7af9a823 Added unit tests for regex fix 2020-11-16 11:43:35 -05:00
Johanna Amann
7362f30c3a Sumstats: allow users to manage epoch manually
This change allows users to specify an epoch length of 0, which means
that the user manually has to finish the epochs. A new next_epoch
function is introduced to allow users to manually end epochs.

Addresses GH-348
2020-11-16 15:48:46 +00:00
Jon Siwek
331b94db39 Simply ssh/main.zeek by using "ssh_server_host_key" for fingerprinting 2020-11-14 08:40:27 -08:00
Brevet Vivien
3769ed6c66 [SSH] Handle SSH version 1.99
SSH can set in its identification a version 1.99 (SSH-1.99-xxx).
That means the client/server is compatible with SSHv1 and SSHv2.
So the version choice depends of the both side.

1.99 : 1.99 => 2.0
1.99 : 1.x  => 1.x
1.99 : 2.0  => 2.O

(see "Compatibility With Old SSH Versions" in RFC 4253)
2020-11-14 15:33:34 +01:00
Jon Siwek
45449dad72 Deprecate "ssh1_server_host_key" parameters *e* and *p*
They are named such that *e* is actually the modulus, not the exponent.
The replacement parameters are named *exponent* and *modulus* for
clarity.
2020-11-13 22:58:56 -08:00
Jon Siwek
bd40a97a78 GH-1264: Implement "ssh_server_host_key" event
This event provides host key fingerprints for both SSH1 and SSH2.
2020-11-13 22:58:56 -08:00
Jon Siwek
ae923106f1 GH-352: Improve HTTP::match_sql_injection_uri regex
Changes \x00-\x37 ranges to \x00-\x1f with assumption that the former
was attempting to match ASCII control characters, but mistook an octal
range for hex.  This change reduces some false positives.
2020-11-12 16:19:35 -08:00
Jon Siwek
fc114069b0 Merge remote-tracking branch 'origin/topic/jsiwek/unknown-protocol-options'
* origin/topic/jsiwek/unknown-protocol-options:
  Move UnknownProtocol options to init-bare.zeek
  Coverity 1436183: Initialize packet_analysis::Manager fields
2020-11-12 14:35:01 -08:00
Jon Siwek
4618df762d Merge remote-tracking branch 'origin/topic/jsiwek/gh-1273-size-expr-any'
* origin/topic/jsiwek/gh-1273-size-expr-any:
  GH-1273: Change SizeExpr to yield "any" type when operating on "any"
2020-11-12 14:30:52 -08:00
Jon Siwek
aab99b743d Merge remote-tracking branch 'origin/topic/jsiwek/type-alias-introspection'
* origin/topic/jsiwek/type-alias-introspection:
  Add enum_names() BIF to return names of an enum type's values
  Add type_aliases() BIF for introspecting type-names of types/values
  Change Type::type_aliases map to store IntrusivePtr
  Fix lookup_ID() BIF to return enum values
2020-11-12 14:30:32 -08:00
Robin Sommer
78fc5615a5 Merge remote-tracking branch 'origin/topic/christian/fix-extfunc-includes'
* origin/topic/christian/fix-extfunc-includes:
  Fix type clash fatal error with log filters that use $ext_func and $include/$exclude
2020-11-12 10:33:48 +00:00
Jon Siwek
89af6f2004 Move UnknownProtocol options to init-bare.zeek
Otherwise the `unknown_protocol` event cannot be used independently
from `policy/mic/unknown-protocols.zeek`.
2020-11-11 12:58:38 -08:00
Tim Wojtulewicz
a6a4b976ec Support for additional DNS RR Type: LOC[29], SSHFP[44], NSEC3PARAM[51], custom BIND9 signaling[65534] 2020-11-11 13:35:51 -07:00
Christian Kreibich
f97a33e14d Fix type clash fatal error with log filters that use $ext_func and $include/$exclude
The logging manager's Manager::TraverseRecord(), called when adding a
log filter to a stream, skipped any fields intoduced by a filter's
$ext_func when such fields weren't mentioned in a $include restriction
or mentioned in an $exclude restriction. This was inconsistent with
Manager::RecordToFilterVals, used when actually writing log entries,
which does include those values.

The result was that the record indices descent in Manager::RecordToFilterVals
expects to find only record values, when in fact only the record
provided by ext_func is present. This leads to type mismatches and
hard Zeek exits like this one:

1300475173.475401 fatal error in zeek/share/zeek//base/init-bare.zeek, line 4810: Val::CONVERTER (string/record) (zeek)

The fix makes ext_func's field additions decisive, meaning the
filter's include/exclude lists don't apply to it. If a user really
wants to override this, they can reset the filter's ext_func back to
our no-op default.

The included btest produces the above error when the fix is not present.
2020-11-10 18:13:16 -08:00
Tim Wojtulewicz
c3cf36e135 GH-1221: Add unknown_protocols.log for logging packet analyzer lookup failures 2020-11-09 20:37:26 -07:00
Jon Siwek
d4528162d1 GH-1273: Change SizeExpr to yield "any" type when operating on "any" 2020-11-09 19:03:42 -08:00
Jon Siwek
f147c47271 Add enum_names() BIF to return names of an enum type's values 2020-11-06 18:31:41 -08:00
Jon Siwek
77514f234f Add type_aliases() BIF for introspecting type-names of types/values 2020-11-06 17:18:44 -08:00
Jon Siwek
bfb7afc600 Fix lookup_ID() BIF to return enum values
Looking up an enum value from a string equal to its name previously
returned "<no ID value>".
2020-11-06 15:37:54 -08:00
Jon Siwek
b7bd87d9cf GH-1252: rename files with colons for Windows compatibility
A short-term measure so that the Zeek source tree can simply exist on a
Windows filesystem.  For true support, the logic that decides/generates
the filename format will need to change.
2020-11-04 10:29:52 -08:00
Tim Wojtulewicz
cd06bf34c7 GH-1215: Remove dispatch_map from packet analysis, replace with BIF methods for registering dispatches 2020-11-02 19:03:25 +00:00
Tim Wojtulewicz
008cdef712 Merge remote-tracking branch 'origin/topic/jsiwek/better-zeekygen-redef-info'
* origin/topic/jsiwek/better-zeekygen-redef-info:
  Improve how Zeekygen generated record/enum redefinition docs
2020-10-26 11:04:17 -07:00
Jon Siwek
fb85a19fa8 GH-1245: require TLD of hostname literals to start with a letter 2020-10-23 14:17:10 -07:00
Jon Siwek
73c1af838c GH-251 (revert): remove coercion-to-signed-integer for |x| expressions
For `|x|`, where `x` is an expression with an integral result, an
implicit coercion of that result into signed `int` type no longer takes
place.

This was actually the behavior before Zeek 3.0 as well, but the attempt
to prevent mistakes that easily result from integer literals in Zeek
being unsigned like `|5 - 9|` causing an overflow/wraparound and
yielding a very large number is not generally consistent since overflows
are still generally able to happen in other ways and also in other
contexts besides just absolute-values.  So the preference was to revert
to a behavior that favors consistency.  For reference, see
https://github.com/zeek/zeek/pull/251#issuecomment-713956976
2020-10-22 17:41:53 -07:00
Seth Hall
fe94249ad3 Fixing a test 2020-10-22 13:30:30 -04:00
Seth Hall
552a24e07c Add an option to ignore packets sourced from particular subnets.
It's implemented with a new set[subnet] option named ignore_checksums_nets.

If you populate this set with subnets, any packet with a src address within
that set of subnets will not have it's checksum validated.
2020-10-22 13:23:10 -04:00
Robin Sommer
df48b7ea57 Update BTest to 0.63.
New baselines will now be store canonified.

Also setting new MinVersion option in btest.cfg.
2020-10-20 09:01:11 +00:00
Jon Siwek
487d67ac50 Improve how Zeekygen generated record/enum redefinition docs
It now provides a summary of the new fields/enums added by any given
redefinition along with associated commentary.
2020-10-19 18:33:22 -07:00
Jon Siwek
5e164469f3 Merge remote-tracking branch 'origin/topic/vlad/gh-1225'
* origin/topic/vlad/gh-1225:
  Extract length as a uint8
  Change ICMP ND length to a uint16
2020-10-16 10:37:08 -07:00
Johanna Amann
65125121d8 Merge branch 'topic/johanna/GH-169'
* topic/johanna/GH-169:
  Make event ordering deterministic
  dump-events: try to make baseline work on all systems
  Introduce generate_all_events bif and add option to misc/dump-events

Fixes GH-169
2020-10-16 07:11:57 +00:00
Vlad Grigorescu
b5d11d1ace Change ICMP ND length to a uint16 2020-10-15 16:56:05 -05:00
Tim Wojtulewicz
a99b540e46 Rework Sessions::Weird 2020-10-15 13:03:11 -07:00
Tim Wojtulewicz
a7d4364334 Review cleanup 2020-10-15 12:44:45 -07:00
Tim Wojtulewicz
1cf251d1ca Move IP and IP tunnel code from Sessions into packet analyzers 2020-10-15 12:18:30 -07:00
Johanna Amann
5f29f1dca1 Make event ordering deterministic
NetControl::init and filter_change_tracking could basically be raised in
random order.
2020-10-15 16:35:52 +00:00
Johanna Amann
e6805258dc dump-events: try to make baseline work on all systems 2020-10-15 15:35:51 +00:00
Johanna Amann
e95e9b8a5d Merge remote-tracking branch 'origin/master' into topic/johanna/GH-169 2020-10-15 15:31:23 +00:00
Jon Siwek
5f1ee35d31 Merge remote-tracking branch 'origin/topic/jsiwek/gh-1211-improve-already-defined-error' into master
* origin/topic/jsiwek/gh-1211-improve-already-defined-error:
  GH-1211: Improve error message for already-defined functions
2020-10-14 10:48:05 -07:00
Jon Siwek
d62fb3ab9a Merge remote-tracking branch 'origin/topic/jsiwek/zeek-script-args'
* origin/topic/jsiwek/zeek-script-args:
  Improve zeek_script_args test case and documentation
  Apply suggestions from code review
  Add a test for script args.
  Fixed an option processing bug
  Make it possible to pass command line options through to scripts.
2020-10-13 13:34:18 -07:00
Jon Siwek
2d3b4dab74 Improve zeek_script_args test case and documentation 2020-10-13 12:40:53 -07:00
Jon Siwek
86e10bfb7e Merge remote-tracking branch 'origin/topic/vlad/caploss_no_traffic'
- Tweaked the Too_Little_Traffic notice message to avoid
  cluster-specific terminology.

* origin/topic/vlad/caploss_no_traffic:
  Fix scheduling due to network_time being 0 in zeek_init
  Add test for CaptureLoss::Too_Little_Traffic
  Add CaptureLoss::Too_Little_Traffic
  Add CaptureLoss::initial_watch_interval for a quick read on cluster health after startup.
  Documentation update, reference the threshold variable. [nomail] [skip ci]
  Whitespace fixes only [nomail] [skip ci]
2020-10-12 17:05:05 -07:00