Commit graph

4024 commits

Author SHA1 Message Date
Tim Wojtulewicz
c88a362d1b Avoid race conditions in broker.ssl_auth_failure btest 2022-08-24 12:10:42 -07:00
Tim Wojtulewicz
76b6cf7afa Merge remote-tracking branch 'simeonmiteff/master'
* simeonmiteff/master:
  Pull changes from zeek/cmake fork
  Skip test based on preprocessor flag set by cmake
  Set flag for libpcap without DLT_LINUX_SLL2
  Force event order in core/init-error btest
  Update some coverage baselines
  Update plugins/hooks baseline
  Add support for DLT_LINUX_SLL2 PCAP link-type
2022-08-24 09:23:01 -07:00
Johanna Amann
0968322c5f Merge remote-tracking branch 'origin/topic/neverlord/gh-2343'
* origin/topic/neverlord/gh-2343:
  Add missing bits for Broker::metrics_import_topics
  Try adding Broker::metrics_import_topics, stuck
2022-08-24 13:26:20 +01:00
Tim Wojtulewicz
8829490045 Merge remote-tracking branch 'origin/topic/awelzel/2075-if-conditional-parsing'
* origin/topic/awelzel/2075-if-conditional-parsing:
  parse.y/anonymous_function: Allow conditionals between begin_lambda and lambda_body
2022-08-23 20:28:17 -07:00
Simeon Miteff
74cc5dcd6b Skip test based on preprocessor flag set by cmake
Relies on change in d42dcb2d55029975a6a6b2e6378fc49a268631ec
2022-08-24 12:47:32 +10:00
Simeon Miteff
bfcc457093 Force event order in core/init-error btest
See https://github.com/zeek/zeek/pull/2340#issuecomment-1218131444
2022-08-24 12:47:25 +10:00
Simeon Miteff
2a22eb4078 Update some coverage baselines 2022-08-24 10:38:51 +10:00
Simeon Miteff
0bfec34732 Update plugins/hooks baseline 2022-08-24 10:38:51 +10:00
Simeon Miteff
b8f0acb5f1 Add support for DLT_LINUX_SLL2 PCAP link-type 2022-08-24 10:38:31 +10:00
Christian Kreibich
c887bcb517 Merge remote-tracking branch 'origin/topic/awelzel/simple-scan-over-scan-zeek'
* origin/topic/awelzel/simple-scan-over-scan-zeek:
  Deprecate misc/scan.zeek
2022-08-23 10:19:47 -07:00
Christian Kreibich
4ed15857b8 Merge remote-tracking branch 'origin/topic/awelzel/vector-value-iteration'
* origin/topic/awelzel/vector-value-iteration:
  stmt: Support iterating over vector values
2022-08-23 09:40:12 -07:00
Arne Welzel
6dc585bd8c Deprecate misc/scan.zeek
Update bare-mode-errors test to ignore that specific message.
2022-08-23 09:10:53 +02:00
Dominik Charousset
6565b4862d Add missing bits for Broker::metrics_import_topics 2022-08-22 17:10:07 +02:00
Arne Welzel
f334df3b79 stmt: Support iterating over vector values
I ran into wanting to iterate over just the values of a vector and wondering
whether that could just work.

This adds support for the following, where v will be value of vec[i].

    local vec = vector("zero", "one", "two");

    for ( i, v in vec )
        print i, v;
2022-08-22 16:57:37 +02:00
Christian Kreibich
f1a9108199 Remove long-unused BRO_DISABLE_BROXYGEN env var from btests 2022-08-19 15:31:55 -07:00
Christian Kreibich
0f8e675a49 Merge branch 'topic/awelzel/unified2-drop'
* topic/awelzel/unified2-drop:
  Remove unified2 file analyzer
2022-08-19 15:23:34 -07:00
Arne Welzel
8d19fa23ef Remove unified2 file analyzer 2022-08-19 14:05:00 +02:00
Arne Welzel
78beecf37d Remove barnyard2 integration scripts 2022-08-19 14:04:27 +02:00
Tim Wojtulewicz
90f0e7a6ea Support other byte lengths in bytestring_to_count 2022-08-17 15:45:30 -07:00
Tim Wojtulewicz
687dd05ddb GH-163: Use ID name (including module name) to create EventExpr when possible
The change to the capture-loss test is actually a fix for a bug exposed by the
code change. Previously it wasn't firing the scheduled event because of a failed
name lookup. Now that the lookup has been fixed, the event happens twice.
2022-08-17 13:15:01 -07:00
Arne Welzel
c6ca89a590 parse.y/anonymous_function: Allow conditionals between begin_lambda and lambda_body
This is to allow conditional statements following a lambda header
(begin_lambda) just as is done with func_hdr conditional_list func_body.
2022-08-17 13:40:25 +02:00
Robin Sommer
598cef21bd
Merge ssh://github.com/fatemabw/zeek
* ssh://github.com/fatemabw/zeek:
  Update options.zeek
  Create out-27
  Add files via upload
  Update src/packet_analysis/protocol/tcp/TCPSessionAdapter.cc
  Updating the weird names to use all lower case
  Fixing whitespaces..
  Fixing clang pre-commit error
  Add check for option 27
  Add the parsed fields for TCP option 27
  Add TCP options bad length check
2022-08-17 09:09:58 +02:00
Robin Sommer
39148ef706
Merge remote-tracking branch 'origin/topic/awelzel/files-log-unrolling'
I removed `deprecated-txhosts-rxhosts-connuids.zeek` from
`local.zeek`, seems preferable not to have a script-to-go-away in the
standard configuration for new users. Also tweaked `NEWS` just a tiny
bit.

* origin/topic/awelzel/files-log-unrolling:
  files.log: Unroll and introduce uid and id fields
2022-08-17 09:01:43 +02:00
Arne Welzel
d2314d2666 files.log: Unroll and introduce uid and id fields
This is a script-only change that unrolls File::Info records into
multiple files.log entries if the same file was seen over different
connections by single worker. Consequently, the File::Info record
gets the commonly used uid and id fields added. These fields are
optional for File::Info - a file may be analyzed without relation
to a network connection (e.g by using Input::add_analysis()).

The existing tx_hosts, rx_hosts and conn_uids fields of Files::Info
are not meaningful after this change and removed by default. Therefore,
files.log will have them removed, too.

The tx_hosts, rx_hosts and conn_uids fields can be revived by using the
policy script frameworks/files/deprecated-txhosts-rxhosts-connuids.zeek
included in the distribution. However, with v6.1 this script will be
removed.
2022-08-16 17:22:20 +02:00
Tim Wojtulewicz
b5b022449e Update zeek-testing-private commit hash after email regex changes
Related to https://github.com/zeek/zeek/pull/2221
2022-08-12 16:44:25 -07:00
Fatema BW
61244738c8
Update options.zeek 2022-08-11 18:29:12 -07:00
Fatema BW
e92c252b3d
Create out-27 2022-08-11 18:27:28 -07:00
Fatema BW
4609429aa4
Add files via upload 2022-08-11 18:23:02 -07:00
Fatema BW
d9632631ce
Merge branch 'zeek:master' into master 2022-08-11 18:20:31 -07:00
Tim Wojtulewicz
7fe6290974 Merge remote-tracking branch 'micrictor/master'
* micrictor/master:
  Add a field to Modbus/TCP log to indicate the Modbus PDU type
  Add modbus transaction and unit ids to logs
  Enable modbus logging for requests
2022-08-11 11:57:10 -07:00
Tim Wojtulewicz
e618be094a Merge remote-tracking branch 'theavgjojo/master'
* theavgjojo/master:
  UPDATED: improving email address splitting for common comma case
2022-08-11 10:41:18 -07:00
Tim Wojtulewicz
313e303fda Merge remote-tracking branch 'AmazingPP/topic/amazingpp/table-values-and-keys'
* AmazingPP/topic/amazingpp/table-values-and-keys:
  Add more test cases
  Add table_keys function
  Add table_values function
2022-08-11 08:49:34 -07:00
Robin Sommer
6d27df548e
Merge remote-tracking branch 'origin/topic/awelzel/1678-disabling-analyzer-hook'
* origin/topic/awelzel/1678-disabling-analyzer-hook:
  Add NEWS entry and zeekygen-smithing for disabling_analyzer()
  Introduce global disabling_analyzer() hook to veto disable_analyzer()
  ssl: Only delete c$ssl$analyzer_id when disabling the analyzer was successful
2022-08-11 16:02:37 +02:00
Robin Sommer
ee7288b1e6
Merge remote-tracking branch 'origin/topic/awelzel/record-field-redef'
* origin/topic/awelzel/record-field-redef:
  Support redef'ing the &log attribute of record fields
2022-08-11 11:59:26 +02:00
Robin Sommer
9de2eceb2a
Merge remote-tracking branch 'origin/topic/awelzel/2262-telemetry-ditch-singleton-metrics'
* origin/topic/awelzel/2262-telemetry-ditch-singleton-metrics:
  telemetry: Remove singleton BIFs and the C++ pieces
2022-08-11 11:54:52 +02:00
Arne Welzel
abb7f9a509 Introduce global disabling_analyzer() hook to veto disable_analyzer()
This hook can be used to coordinate disabling an analyzer for a given
connection. The contract is simple: Any script can veto a disable_analyzer()
call by breaking from this hook. The decision is local to the script taking
into account any state attached to the connection object or script specific
state stored elsewhere.
A script breaking from the hook takes over the responsibility to call
disable_analyzer() at a later point when it finds the condition due to which
it vetoed fulfilled (which may be never).

Signature:

    disabling_analyzer: hook(c: connection, atype: AllAnalyzers::Tag, aid: count);

Example use-cases are keeping the SSL analyzer enabled for finger-printing
until a certain amount of bytes or packets have been transferred or
similarly the connection duration exceed a certain threshold.

Other example use-cases might be keeping analyzers for SSH, RDP or SSL
enabled for connections from specific subnets.

It's a bit quirky as it makes disable_analyzer() a maybe operation. While log
policy hooks and/or the notice hook have similar semantics, they are not as
stateful. It still seems like a quite powerful primitive.

The disable_analyzer() call in dpd/main.zeek may motivate the addition of a
force flag as a follow-up for situations where the caller "knows better" or
absolutely wants to override.

Closes #1678 #1593.
2022-08-11 09:40:36 +02:00
AmazingPP
206e6d28a4 Add more test cases 2022-08-11 13:35:27 +08:00
Christian Kreibich
3a0a702723 Merge branch 'topic/christian/management-bugfixes'
* topic/christian/management-bugfixes:
  Management framework: bump cluster testsuite
  Management framework: log node set in dispatch requests cleanly
  Management framework: log additional node events
  Management framework: upon deployment, make agent log multiple node results
  Management framework: fix early return condition for get-id-value
2022-08-10 14:18:38 -07:00
Christian Kreibich
20e08f0d38 Management framework: bump cluster testsuite 2022-08-10 11:24:10 -07:00
Arne Welzel
985bbe4e57 Support redef'ing the &log attribute of record fields
Add new syntax for adding and removing attributes from record fields:

    redef RecordType$field_name += { &log };
    redef RecordType$field_name -= { &log };

For now this only allowed for the &log attribute as the semantics are clear.
For &default and &optional the semantics aren't obvious and no use-cases have
been identified where those would make sense to change.

This enables a mechanism to add potentially interesting fields to the typical
Info records in base scripts, but letting users opt-into actually including
them into their log. At the same time, users that find specific fields in a
standard log uninteresting can opt-out without using `Log::Filter$exclude`
which can be difficult to use correctly. Patching or forking external packages
to remove columns from a log can also be avoided with this mechanism.

Closes #2000.
2022-08-10 17:27:05 +02:00
Tim Wojtulewicz
514df9f179 Fix module-scoped type definitions that conflict with existing global ones 2022-08-08 15:30:34 -07:00
Tim Wojtulewicz
cb3abccfb1 Merge remote-tracking branch 'ynadji/topic/yacin/2319-add-change-handler-to-site'
* ynadji/topic/yacin/2319-add-change-handler-to-site:
  update plugins.hooks baseline
  lower priority for change handlers
  split update_zones_regex into two functions
  GH-2319: Add change handlers to Site
2022-08-08 11:09:16 -07:00
Tim Wojtulewicz
3da1848279 Merge remote-tracking branch 'AmazingPP/topic/amazingpp/command-line-script-warning'
* AmazingPP/topic/amazingpp/command-line-script-warning:
  Squelch the zeekygen warnings for command line
2022-08-08 11:03:32 -07:00
Yacin Nadji
84610ed832 update plugins.hooks baseline 2022-08-08 11:52:06 +02:00
AmazingPP
9869226e1e Squelch the zeekygen warnings for command line 2022-08-06 11:35:05 +08:00
TheAvgJojo
1e37e91eda UPDATED: improving email address splitting for common comma case 2022-08-05 17:38:54 -04:00
Arne Welzel
acacc6b6c2 telemetry: Remove singleton BIFs and the C++ pieces
The low-level singleton Telemetry BIFs have been removed with the that there
haven't been any users. Singleton metrics can be instantiated by providing
an empty label vector instead and aren't in any way a special concept.

Closes #2262.
2022-08-05 19:33:49 +02:00
Tim Wojtulewicz
9524963da6 Merge remote-tracking branch 'anniebryan/notice-suppression-bug-fix'
* anniebryan/notice-suppression-bug-fix:
  Added test case with back-to-back notices
  Fix notice suppression atomicity bug
2022-08-05 16:21:29 +00:00
Tim Wojtulewicz
1870d26684 GH-2034: Store module names and use them in lookups for ifdef 2022-08-05 15:36:21 +00:00
Robin Sommer
686e740bbe
Merge remote-tracking branch 'origin/topic/awelzel/telemetry-script-land-v0'
* origin/topic/awelzel/telemetry-script-land-v0:
  Introduce telemetry framework
  Bump broker submodule to master.
2022-08-05 17:14:23 +02:00