Commit graph

2270 commits

Author SHA1 Message Date
Tim Wojtulewicz
efe42bc67b Remove default_analyzer for Ethernet packet analzyer 2020-11-09 19:54:45 -07:00
Tim Wojtulewicz
cd06bf34c7 GH-1215: Remove dispatch_map from packet analysis, replace with BIF methods for registering dispatches 2020-11-02 19:03:25 +00:00
Duffy O'Craven
b32eba579d updated to mention notice_alarm.log by name. That has taken the place of the file formerly named alarm.log 2020-10-27 13:56:30 -07:00
Seth Hall
56e234a762 Changes in response to Johanna's code review. 2020-10-22 13:30:30 -04:00
Seth Hall
552a24e07c Add an option to ignore packets sourced from particular subnets.
It's implemented with a new set[subnet] option named ignore_checksums_nets.

If you populate this set with subnets, any packet with a src address within
that set of subnets will not have it's checksum validated.
2020-10-22 13:23:10 -04:00
Johanna Amann
65125121d8 Merge branch 'topic/johanna/GH-169'
* topic/johanna/GH-169:
  Make event ordering deterministic
  dump-events: try to make baseline work on all systems
  Introduce generate_all_events bif and add option to misc/dump-events

Fixes GH-169
2020-10-16 07:11:57 +00:00
Tim Wojtulewicz
1cf251d1ca Move IP and IP tunnel code from Sessions into packet analyzers 2020-10-15 12:18:30 -07:00
Johanna Amann
5f29f1dca1 Make event ordering deterministic
NetControl::init and filter_change_tracking could basically be raised in
random order.
2020-10-15 16:35:52 +00:00
Jon Siwek
6cb0cb9746 Merge remote-tracking branch 'origin/topic/seth/make-cluster-port-optional'
* origin/topic/seth/make-cluster-port-optional:
  Apply suggestions from code review
  Update scripts/base/frameworks/cluster/main.zeek
  Make defining a port number for hosts in a cluster that only connect outbound optional
2020-10-13 15:35:43 -07:00
Seth Hall
cd330c801d
Apply suggestions from code review
Co-authored-by: Jon Siwek <jsiwek@corelight.com>
2020-10-13 16:48:15 -04:00
Seth Hall
e78386d6e5
Update scripts/base/frameworks/cluster/main.zeek
Co-authored-by: Jon Siwek <jsiwek@corelight.com>
2020-10-13 16:46:26 -04:00
Jon Siwek
d62fb3ab9a Merge remote-tracking branch 'origin/topic/jsiwek/zeek-script-args'
* origin/topic/jsiwek/zeek-script-args:
  Improve zeek_script_args test case and documentation
  Apply suggestions from code review
  Add a test for script args.
  Fixed an option processing bug
  Make it possible to pass command line options through to scripts.
2020-10-13 13:34:18 -07:00
Jon Siwek
2d3b4dab74 Improve zeek_script_args test case and documentation 2020-10-13 12:40:53 -07:00
Jon Siwek
961532a8f7 Merge remote-tracking branch 'origin/topic/seth/pcap_findalldevs'
- Minor adjustments to whitespace/formatting

* origin/topic/seth/pcap_findalldevs:
  Finishing changes from code review.
  Update src/iosource/pcap/pcap.bif
  Update src/iosource/pcap/pcap.bif
  Update scripts/base/init-bare.zeek
  Update src/iosource/pcap/pcap.bif
  I accidentally missed a paren
  New bif to wrap pcap_findalldevs
2020-10-13 10:52:14 -07:00
Seth Hall
92eb7c10da Finishing changes from code review. 2020-10-13 08:35:45 -04:00
Seth Hall
dfa21d54c8
Update scripts/base/init-bare.zeek
Co-authored-by: Jon Siwek <jsiwek@corelight.com>
2020-10-13 08:12:20 -04:00
Seth Hall
7bcbc57401 New bif to wrap pcap_findalldevs 2020-10-12 12:47:23 -04:00
Seth Hall
cf8671d078 Make defining a port number for hosts in a cluster that only connect outbound optional 2020-10-12 10:46:28 -04:00
Seth Hall
2bdc56dfcd Make it possible to pass command line options through to scripts.
The feature is documented with the zeek_script_args variable in
init-bare.zeek.
2020-10-08 15:46:28 -04:00
Robin Sommer
b0bf9f02c8 Merge remote-tracking branch 'origin/topic/christian/364-logfilter-hooks' into master
(Adding a NEWS entry.)

* origin/topic/christian/364-logfilter-hooks:
  Update testing/btest/scripts/base/frameworks/logging/hooks.zeek
  Btests for log filter policy hooks
  Btest baseline updates to reflect new logging policy hooks
  Migrate existing use of filter predicates to policy hooks
  Support for log filter policy hooks
2020-10-07 08:44:50 +00:00
Jon Siwek
506d961b27 Merge branch 'logging-ascii-enable-shadow-logs' of https://github.com/awelzel/zeek into master
- Improved documentation/comment for the new option

* 'logging-ascii-enable-shadow-logs' of https://github.com/awelzel/zeek:
  logging/ascii: Support leftover log rotation in non-supervisor setups
2020-10-06 10:15:34 -07:00
Arne Welzel
1f5ab4878b logging/ascii: Support leftover log rotation in non-supervisor setups
We have a use case to rotate leftover log files in a non-supervisor
setup. There doesn't seem to be a strict requirement on supervisor
functionality. Allow enabling leftover log rotation through
LogAscii::enable_leftover_log_rotation and redef this for the
logger node in a supervisor setup individually.
2020-10-02 20:38:48 +02:00
Christian Kreibich
1bd658da8f Support for log filter policy hooks
This adds a "policy" hook into the logging framework's streams and
filters to replace the existing log filter predicates. The hook
signature is as follows:

    hook(rec: any, id: Log::ID, filter: Log::Filter);

The logging manager invokes hooks on each log record. Hooks can veto
log records via a break, and modify them if necessary. Log filters
inherit the stream-level hook, but can override or remove the hook as
needed.

The distribution's existing log streams now come with pre-defined
hooks that users can add handlers to. Their name is standardized as
"log_policy" by convention, with additional suffixes when a module
provides multiple streams. The following adds a handler to the Conn
module's default log policy hook:

    hook Conn::log_policy(rec: Conn::Info, id: Log::ID, filter: Log::Filter)
            {
            if ( some_veto_reason(rec) )
                break;
            }

By default, this handler will get invoked for any log filter
associated with the Conn::LOG stream.

The existing predicates are deprecated for removal in 4.1 but continue
to work.
2020-09-30 12:32:45 -07:00
Robin Sommer
1f6fc4415f Tweak find-filtered-trace to not flag traces if they have non-TCP
traffic.

Closes #160.
2020-09-25 11:29:44 +00:00
Jon Siwek
cee10b5dc6 Fix a Sphinx warning about misformatted packet analyzer comment 2020-09-23 22:52:35 -07:00
Jan Grashoefer
7ede4f48bd Simplify packet analyzer config. 2020-09-23 11:13:29 -07:00
Jan Grashoefer
efa262a229 Make default packet analyzer definition explicit. 2020-09-23 11:13:29 -07:00
Jan Grashoefer
8f951574d7 Add explicit root analyzer for packet analysis. 2020-09-23 11:13:29 -07:00
Jan Grashoefer
3f3f00030d Simplify MPLS analysis. 2020-09-23 11:13:29 -07:00
Jan Grashoefer
d5ca0f9da5 Rename DefaultAnalyzer to IP. 2020-09-23 11:13:28 -07:00
Jan Grashoefer
0ec7516602 Small cleanup of packet analysis. 2020-09-23 11:13:28 -07:00
Jan Grashoefer
0925b3bbec Remove encap_hdr_size (replaced by skip analyzer). 2020-09-23 11:13:28 -07:00
Jan Grashoefer
fc814bd7e2 Add SkipAnalyzer.
This is WIP: The test case would require a new pcap or the possibility
to overwrite analyzer mappings. The CustomEncapsulationSkip method and
the corresponding options need to be removed.
2020-09-23 11:13:28 -07:00
Jan Grashoefer
4aeab7402d Improve naming in packet analysis. 2020-09-23 11:13:28 -07:00
Jan Grashoefer
6365fa6d80 Migrate all packet analyzers to new API. 2020-09-23 11:13:28 -07:00
Jan Grashoefer
cbdaa53f85 Remove magic identifiers from Ethernet analyzer. 2020-09-23 11:13:28 -07:00
Jan Grashoefer
462b1fe3a2 Bring back default packet analysis.
Default analyzers can be configured per packet analyzer by omitting the
identifier in the ConfigEntry.
2020-09-23 11:13:28 -07:00
Jan Grashoefer
9feda100b9 Move dispatching into packet analyzers.
WIP that updates only the Ethernet analyzer.
2020-09-23 11:13:28 -07:00
Jan Grashoefer
e53ec46c23 Renamed LL-Analyzers to Packet Analyzers. 2020-09-23 11:13:28 -07:00
Peter Oettig
b2e6c9ac9a Initial implementation of Lower-Level analyzers 2020-09-23 11:13:25 -07:00
Jon Siwek
f744d4c070 Improve fa_file/fa_metadata documentation
Related to https://github.com/zeek/zeek-docs/pull/38
2020-09-22 16:04:00 -07:00
Tim Wojtulewicz
6b93020c6f Merge remote-tracking branch 'origin/topic/jsiwek/gh-1164-fix-rst-conn-state'
* origin/topic/jsiwek/gh-1164-fix-rst-conn-state:
  GH-1164: Fix incorrect RSTOS0 conn_state determinations
2020-09-17 13:40:06 -07:00
Jon Siwek
a16bd47bf7 GH-1164: Fix incorrect RSTOS0 conn_state determinations
The RSTOS0 `conn_state` label is documented as "Originator sent a SYN
followed by a RST, never saw SYN-ACK from responder", but was previously
applied to cases where no originator SYN exists, like a single RST-only
packet.
2020-09-11 16:14:41 -07:00
Jon Siwek
05cf511f18 GH-1119: add base/protcols/conn/removal-hooks.zeek
This adds two new functions: `Conn::register_removal_hook()` and
`Conn::unregister_removal_hook()` for registering a hook function to be
called back during `connection_state_remove`.  The benefit of using hook
callback approach is better scalability: the overhead of unrelated
protocols having to dispatch no-op `connection_state_remove` handlers is
avoided.
2020-09-11 12:12:10 -07:00
Jon Siwek
49e2047da0 Avoid run-time cost of find-filtered-trace.zeek for live traffic 2020-09-10 12:53:47 -07:00
Jon Siwek
5f435c2644 Remove connection_successful and successful_connection_remove events
Related to https://github.com/zeek/zeek/issues/1119
2020-09-10 12:06:50 -07:00
Jon Siwek
260895fcb1 Merge remote-tracking branch 'origin/topic/robin/gh-343-http-sigs'
* origin/topic/robin/gh-343-http-sigs:
  Change HTTP's DPD signatures so that each side can trigger the analyzer on its own.
2020-09-08 17:24:29 -07:00
Jon Siwek
991bbc961d Merge remote-tracking branch 'origin/topic/robin/gh-623-sampling'
- Merge adjustments:
  - Minor whitespace/style tweaks
  - Fixed portability of the btest due to differences in `uniq -c`
    output format

* origin/topic/robin/gh-623-sampling:
  Extend weird sampling with option to track selected weirds globally.
2020-09-08 17:06:40 -07:00
Robin Sommer
0af57d12b2 Change HTTP's DPD signatures so that each side can trigger the analyzer on its own.
This is to avoid missing large sessions where a single side exceeds
the DPD buffer size. It comes with the trade-off that now the analyzer
can be triggered by anybody controlling one of the endpoints (instead
of both).

Test suite changes are minor, and nothing in "external".

Closes #343.
2020-09-08 07:33:36 +00:00
Robin Sommer
5fa94973da Extend weird sampling with option to track selected weirds globally.
The new set "sampling_global_list" lists weirds to rate-limite
globally instead of per connection/flow.

Closes #623.
2020-09-07 07:31:22 +00:00