Commit graph

4030 commits

Author SHA1 Message Date
AmazingPP
970f45d686 Replace all the Warning() calls after IsFieldDeprecated() over to Warn() 2022-09-03 19:15:47 +08:00
AmazingPP
997d3ed126 Fix deprecation not flagged and incorrect line number in record 2022-09-03 10:36:01 +08:00
Tim Wojtulewicz
ed4521af60 Add type checking to string_cat arguments 2022-08-29 08:45:59 -07:00
Tim Wojtulewicz
686eb54f95 Merge remote-tracking branch 'origin/topic/awelzel/gtpv1-no-protocol-detection'
* origin/topic/awelzel/gtpv1-no-protocol-detection:
  gtpv1: Do not register for protocol detection
2022-08-26 14:49:54 -07:00
Dominik Charousset
456efd8082 Revert to regular peering fn for ssl_auth_failure
With the fixed Broker version, the `ssl_auth_failure` btest can use the
regular `peer` function again.
2022-08-26 17:00:45 +02:00
Arne Welzel
42be2444a7 gtpv1: Do not register for protocol detection
While reviewing/understanding the analyzer setup, it didn't seem like
GTPv1 implements packet_analysis::Analyzer::DetectProtocol(), so
should not register it for protocol_detection either.

Alternatively, maybe DetectProtocol() should've been implemented in
which case maybe this should be an issue?
2022-08-26 10:47:38 +02:00
Tim Wojtulewicz
c88a362d1b Avoid race conditions in broker.ssl_auth_failure btest 2022-08-24 12:10:42 -07:00
Tim Wojtulewicz
76b6cf7afa Merge remote-tracking branch 'simeonmiteff/master'
* simeonmiteff/master:
  Pull changes from zeek/cmake fork
  Skip test based on preprocessor flag set by cmake
  Set flag for libpcap without DLT_LINUX_SLL2
  Force event order in core/init-error btest
  Update some coverage baselines
  Update plugins/hooks baseline
  Add support for DLT_LINUX_SLL2 PCAP link-type
2022-08-24 09:23:01 -07:00
Johanna Amann
0968322c5f Merge remote-tracking branch 'origin/topic/neverlord/gh-2343'
* origin/topic/neverlord/gh-2343:
  Add missing bits for Broker::metrics_import_topics
  Try adding Broker::metrics_import_topics, stuck
2022-08-24 13:26:20 +01:00
Tim Wojtulewicz
8829490045 Merge remote-tracking branch 'origin/topic/awelzel/2075-if-conditional-parsing'
* origin/topic/awelzel/2075-if-conditional-parsing:
  parse.y/anonymous_function: Allow conditionals between begin_lambda and lambda_body
2022-08-23 20:28:17 -07:00
Simeon Miteff
74cc5dcd6b Skip test based on preprocessor flag set by cmake
Relies on change in d42dcb2d55029975a6a6b2e6378fc49a268631ec
2022-08-24 12:47:32 +10:00
Simeon Miteff
bfcc457093 Force event order in core/init-error btest
See https://github.com/zeek/zeek/pull/2340#issuecomment-1218131444
2022-08-24 12:47:25 +10:00
Simeon Miteff
2a22eb4078 Update some coverage baselines 2022-08-24 10:38:51 +10:00
Simeon Miteff
0bfec34732 Update plugins/hooks baseline 2022-08-24 10:38:51 +10:00
Simeon Miteff
b8f0acb5f1 Add support for DLT_LINUX_SLL2 PCAP link-type 2022-08-24 10:38:31 +10:00
Christian Kreibich
c887bcb517 Merge remote-tracking branch 'origin/topic/awelzel/simple-scan-over-scan-zeek'
* origin/topic/awelzel/simple-scan-over-scan-zeek:
  Deprecate misc/scan.zeek
2022-08-23 10:19:47 -07:00
Christian Kreibich
4ed15857b8 Merge remote-tracking branch 'origin/topic/awelzel/vector-value-iteration'
* origin/topic/awelzel/vector-value-iteration:
  stmt: Support iterating over vector values
2022-08-23 09:40:12 -07:00
Arne Welzel
6dc585bd8c Deprecate misc/scan.zeek
Update bare-mode-errors test to ignore that specific message.
2022-08-23 09:10:53 +02:00
Dominik Charousset
6565b4862d Add missing bits for Broker::metrics_import_topics 2022-08-22 17:10:07 +02:00
Arne Welzel
f334df3b79 stmt: Support iterating over vector values
I ran into wanting to iterate over just the values of a vector and wondering
whether that could just work.

This adds support for the following, where v will be value of vec[i].

    local vec = vector("zero", "one", "two");

    for ( i, v in vec )
        print i, v;
2022-08-22 16:57:37 +02:00
Christian Kreibich
f1a9108199 Remove long-unused BRO_DISABLE_BROXYGEN env var from btests 2022-08-19 15:31:55 -07:00
Christian Kreibich
0f8e675a49 Merge branch 'topic/awelzel/unified2-drop'
* topic/awelzel/unified2-drop:
  Remove unified2 file analyzer
2022-08-19 15:23:34 -07:00
Arne Welzel
8d19fa23ef Remove unified2 file analyzer 2022-08-19 14:05:00 +02:00
Arne Welzel
78beecf37d Remove barnyard2 integration scripts 2022-08-19 14:04:27 +02:00
Tim Wojtulewicz
90f0e7a6ea Support other byte lengths in bytestring_to_count 2022-08-17 15:45:30 -07:00
Tim Wojtulewicz
687dd05ddb GH-163: Use ID name (including module name) to create EventExpr when possible
The change to the capture-loss test is actually a fix for a bug exposed by the
code change. Previously it wasn't firing the scheduled event because of a failed
name lookup. Now that the lookup has been fixed, the event happens twice.
2022-08-17 13:15:01 -07:00
Arne Welzel
c6ca89a590 parse.y/anonymous_function: Allow conditionals between begin_lambda and lambda_body
This is to allow conditional statements following a lambda header
(begin_lambda) just as is done with func_hdr conditional_list func_body.
2022-08-17 13:40:25 +02:00
Robin Sommer
598cef21bd
Merge ssh://github.com/fatemabw/zeek
* ssh://github.com/fatemabw/zeek:
  Update options.zeek
  Create out-27
  Add files via upload
  Update src/packet_analysis/protocol/tcp/TCPSessionAdapter.cc
  Updating the weird names to use all lower case
  Fixing whitespaces..
  Fixing clang pre-commit error
  Add check for option 27
  Add the parsed fields for TCP option 27
  Add TCP options bad length check
2022-08-17 09:09:58 +02:00
Robin Sommer
39148ef706
Merge remote-tracking branch 'origin/topic/awelzel/files-log-unrolling'
I removed `deprecated-txhosts-rxhosts-connuids.zeek` from
`local.zeek`, seems preferable not to have a script-to-go-away in the
standard configuration for new users. Also tweaked `NEWS` just a tiny
bit.

* origin/topic/awelzel/files-log-unrolling:
  files.log: Unroll and introduce uid and id fields
2022-08-17 09:01:43 +02:00
Arne Welzel
d2314d2666 files.log: Unroll and introduce uid and id fields
This is a script-only change that unrolls File::Info records into
multiple files.log entries if the same file was seen over different
connections by single worker. Consequently, the File::Info record
gets the commonly used uid and id fields added. These fields are
optional for File::Info - a file may be analyzed without relation
to a network connection (e.g by using Input::add_analysis()).

The existing tx_hosts, rx_hosts and conn_uids fields of Files::Info
are not meaningful after this change and removed by default. Therefore,
files.log will have them removed, too.

The tx_hosts, rx_hosts and conn_uids fields can be revived by using the
policy script frameworks/files/deprecated-txhosts-rxhosts-connuids.zeek
included in the distribution. However, with v6.1 this script will be
removed.
2022-08-16 17:22:20 +02:00
Tim Wojtulewicz
b5b022449e Update zeek-testing-private commit hash after email regex changes
Related to https://github.com/zeek/zeek/pull/2221
2022-08-12 16:44:25 -07:00
Fatema BW
61244738c8
Update options.zeek 2022-08-11 18:29:12 -07:00
Fatema BW
e92c252b3d
Create out-27 2022-08-11 18:27:28 -07:00
Fatema BW
4609429aa4
Add files via upload 2022-08-11 18:23:02 -07:00
Fatema BW
d9632631ce
Merge branch 'zeek:master' into master 2022-08-11 18:20:31 -07:00
Tim Wojtulewicz
7fe6290974 Merge remote-tracking branch 'micrictor/master'
* micrictor/master:
  Add a field to Modbus/TCP log to indicate the Modbus PDU type
  Add modbus transaction and unit ids to logs
  Enable modbus logging for requests
2022-08-11 11:57:10 -07:00
Tim Wojtulewicz
e618be094a Merge remote-tracking branch 'theavgjojo/master'
* theavgjojo/master:
  UPDATED: improving email address splitting for common comma case
2022-08-11 10:41:18 -07:00
Tim Wojtulewicz
313e303fda Merge remote-tracking branch 'AmazingPP/topic/amazingpp/table-values-and-keys'
* AmazingPP/topic/amazingpp/table-values-and-keys:
  Add more test cases
  Add table_keys function
  Add table_values function
2022-08-11 08:49:34 -07:00
Robin Sommer
6d27df548e
Merge remote-tracking branch 'origin/topic/awelzel/1678-disabling-analyzer-hook'
* origin/topic/awelzel/1678-disabling-analyzer-hook:
  Add NEWS entry and zeekygen-smithing for disabling_analyzer()
  Introduce global disabling_analyzer() hook to veto disable_analyzer()
  ssl: Only delete c$ssl$analyzer_id when disabling the analyzer was successful
2022-08-11 16:02:37 +02:00
Robin Sommer
ee7288b1e6
Merge remote-tracking branch 'origin/topic/awelzel/record-field-redef'
* origin/topic/awelzel/record-field-redef:
  Support redef'ing the &log attribute of record fields
2022-08-11 11:59:26 +02:00
Robin Sommer
9de2eceb2a
Merge remote-tracking branch 'origin/topic/awelzel/2262-telemetry-ditch-singleton-metrics'
* origin/topic/awelzel/2262-telemetry-ditch-singleton-metrics:
  telemetry: Remove singleton BIFs and the C++ pieces
2022-08-11 11:54:52 +02:00
Arne Welzel
abb7f9a509 Introduce global disabling_analyzer() hook to veto disable_analyzer()
This hook can be used to coordinate disabling an analyzer for a given
connection. The contract is simple: Any script can veto a disable_analyzer()
call by breaking from this hook. The decision is local to the script taking
into account any state attached to the connection object or script specific
state stored elsewhere.
A script breaking from the hook takes over the responsibility to call
disable_analyzer() at a later point when it finds the condition due to which
it vetoed fulfilled (which may be never).

Signature:

    disabling_analyzer: hook(c: connection, atype: AllAnalyzers::Tag, aid: count);

Example use-cases are keeping the SSL analyzer enabled for finger-printing
until a certain amount of bytes or packets have been transferred or
similarly the connection duration exceed a certain threshold.

Other example use-cases might be keeping analyzers for SSH, RDP or SSL
enabled for connections from specific subnets.

It's a bit quirky as it makes disable_analyzer() a maybe operation. While log
policy hooks and/or the notice hook have similar semantics, they are not as
stateful. It still seems like a quite powerful primitive.

The disable_analyzer() call in dpd/main.zeek may motivate the addition of a
force flag as a follow-up for situations where the caller "knows better" or
absolutely wants to override.

Closes #1678 #1593.
2022-08-11 09:40:36 +02:00
AmazingPP
206e6d28a4 Add more test cases 2022-08-11 13:35:27 +08:00
Christian Kreibich
3a0a702723 Merge branch 'topic/christian/management-bugfixes'
* topic/christian/management-bugfixes:
  Management framework: bump cluster testsuite
  Management framework: log node set in dispatch requests cleanly
  Management framework: log additional node events
  Management framework: upon deployment, make agent log multiple node results
  Management framework: fix early return condition for get-id-value
2022-08-10 14:18:38 -07:00
Christian Kreibich
20e08f0d38 Management framework: bump cluster testsuite 2022-08-10 11:24:10 -07:00
Arne Welzel
985bbe4e57 Support redef'ing the &log attribute of record fields
Add new syntax for adding and removing attributes from record fields:

    redef RecordType$field_name += { &log };
    redef RecordType$field_name -= { &log };

For now this only allowed for the &log attribute as the semantics are clear.
For &default and &optional the semantics aren't obvious and no use-cases have
been identified where those would make sense to change.

This enables a mechanism to add potentially interesting fields to the typical
Info records in base scripts, but letting users opt-into actually including
them into their log. At the same time, users that find specific fields in a
standard log uninteresting can opt-out without using `Log::Filter$exclude`
which can be difficult to use correctly. Patching or forking external packages
to remove columns from a log can also be avoided with this mechanism.

Closes #2000.
2022-08-10 17:27:05 +02:00
Tim Wojtulewicz
514df9f179 Fix module-scoped type definitions that conflict with existing global ones 2022-08-08 15:30:34 -07:00
Tim Wojtulewicz
cb3abccfb1 Merge remote-tracking branch 'ynadji/topic/yacin/2319-add-change-handler-to-site'
* ynadji/topic/yacin/2319-add-change-handler-to-site:
  update plugins.hooks baseline
  lower priority for change handlers
  split update_zones_regex into two functions
  GH-2319: Add change handlers to Site
2022-08-08 11:09:16 -07:00
Tim Wojtulewicz
3da1848279 Merge remote-tracking branch 'AmazingPP/topic/amazingpp/command-line-script-warning'
* AmazingPP/topic/amazingpp/command-line-script-warning:
  Squelch the zeekygen warnings for command line
2022-08-08 11:03:32 -07:00
Yacin Nadji
84610ed832 update plugins.hooks baseline 2022-08-08 11:52:06 +02:00