Commit graph

2745 commits

Author SHA1 Message Date
Robin Sommer
369e42a6e4 Fix SMB tests on Apple M1.
Due to different double precision on M1, file IDs for SMB could end up
changing on M1 because the access time of a file goes into their
computation. The real solution for this would be changing Zeek's
internal "time" representation to uint64; that's planned, but requires
major surgery. For now, this PR changes the SMB code to also pass SMB's
original time representation (which is a uint64) into script-land, and
then use that for computing the file ID.

Closes #1406
2021-06-29 20:17:02 +02:00
Tim Wojtulewicz
30973af94a Merge remote-tracking branch 'origin/topic/johanna/remove-stepping-stone'
* origin/topic/johanna/remove-stepping-stone:
  Remove the Stepping Stone analyzer
2021-06-21 16:14:25 -07:00
Robin Sommer
6559d56dbf Merge remote-tracking branch 'origin/topic/vlad/page_hostnames'
* origin/topic/vlad/page_hostnames:
  Expand the tests, and have email_admin email src contacts in one e-mail, and dst contacts in another.
  Simplify the logic a big by making Notice$email_dest have a default rather than be optional
  Missed some baselines
  Update hashes in external testing repos
  Update notice.log baselines
  Remove unneccessary @load
  Add tests for e-mail actions, and cleanup the new logic a bit.
  Fixup priorities
  Add a new field to the notice, which defines where to send the e-mail
  Add page and email_admin to hostnames extension
2021-06-21 11:21:17 +02:00
Vlad Grigorescu
9d1e9a20e4 Expand the tests, and have email_admin email src contacts in one e-mail, and dst contacts in another. 2021-06-17 14:49:24 -05:00
Vlad Grigorescu
9d0bd96e1f Simplify the logic a big by making Notice$email_dest have a default rather than be optional 2021-06-17 10:31:04 -05:00
Johanna Amann
a995d73fdf Merge remote-tracking branch 'origin/topic/bbannier/issue-1590'
* origin/topic/bbannier/issue-1590:
  Change SSH version field to be `&optional`.
  Reformat function in SSH base script.

Changes during merge:
  * rename weirds to fit into our naming scheme
  * add NEWS entry

Closes GH-1590
2021-06-17 10:52:49 +01:00
Benjamin Bannier
daa9537f92 Change SSH version field to be &optional.
In 3769ed6c66 we added handling for SSH version 1.99 which unsed a SSH
version of 0 to indicate weird cases where no version could be
determined.

This patch is a fixup for that patch. Instead of using a magic version
of 0 we now use an `&optional` version value. If no SSH version can be
extracted the version will be unset; additionally a `conn_weird` event
will be raised.

Closes #1590.
2021-06-17 09:24:46 +02:00
Benjamin Bannier
e6e30240ec Reformat function in SSH base script. 2021-06-16 15:03:43 +02:00
Vlad Grigorescu
e579497247 Merge branch 'master' into topic/vlad/page_hostnames 2021-06-11 10:18:15 -05:00
Johanna Amann
e0d284ec9f Merge branch 'logging/script-logdir' of https://github.com/kramse/zeek
* 'logging/script-logdir' of https://github.com/kramse/zeek:
  Copy of ascii-empty test, just changed path in the beginning
  Logdir: Change requested by 0xxon, no problem
  Introduce script-land variable that can be used to set logdir.

Closes GH-772
2021-06-10 12:19:15 +01:00
Henrik Kramselund Jereminsen
74561e922f Logdir: Change requested by 0xxon, no problem
Suggestion from 0xxon to look at Input Framework inspired this change
2021-06-10 10:33:26 +01:00
Henrik Kramselund Jereminsen
6bde33aca7 Introduce script-land variable that can be used to set logdir.
Addresses GH-772
2021-06-10 10:32:11 +01:00
Arne Welzel
4327c5a022 init-bare/ProcStats: mem is in bytes, not KB.
zeek::util::get_memory_usage() does `r.ru_maxrss * 1024` and the
task_info() seems to return bytes in `resident_size`.
2021-06-05 21:25:27 +02:00
Johanna Amann
011ac442a1 Remove the Stepping Stone analyzer
This commit removes the stepping stone analyzer. It has been deactivated
by default since at least Zeek 2.0, is dysfunctional in cluster settings
and has a bunch of other issued.

Relates to GH-1573
2021-06-03 14:28:12 +01:00
Tim Wojtulewicz
ff79a58f59 Merge remote-tracking branch 'origin/topic/neverlord/telemetry-scraper'
* origin/topic/neverlord/telemetry-scraper:
  Integrate review feedback
  Sync new broker options, fix name inconsistencies
  Integrate new Broker metric exporter parameters
2021-06-01 10:19:46 -07:00
Chris C
ada482c0e3
Update detect-MHR.zeek
Update Virustotal URL to current
option match_sub_url = "https://www.virustotal.com/gui/search/%s";
2021-05-28 14:46:19 -05:00
Dominik Charousset
44ca01eb18 Merge branch master into topic/neverlord/telemetry-scraper 2021-05-26 08:06:13 +02:00
Dominik Charousset
7767c3d36c Sync new broker options, fix name inconsistencies 2021-05-25 17:22:45 +02:00
Dominik Charousset
f9cd05f00b Integrate new Broker metric exporter parameters 2021-05-24 17:20:48 +02:00
Tim Wojtulewicz
c1f0d312b5 Add base class for IP-based packet analyzers 2021-05-18 11:52:03 -07:00
Tim Wojtulewicz
0c3e3069d0 Added skeletons for TCP/UDP/ICMP packet analysis plugins.
This includes integration into the IP plugin and calling of the sessions code from each plugin.
2021-05-18 11:52:03 -07:00
Vern Paxson
24e92fa54a hooks for activating the compiler & associated BiF 2021-05-05 16:55:04 -07:00
Vern Paxson
66fb3e0f56 low-level tidying / documentation tweaks 2021-05-05 16:50:29 -07:00
Tim Wojtulewicz
ad67d810be Merge remote-tracking branch 'origin/topic/jsiwek/fix-netbios-decode-bifs'
* origin/topic/jsiwek/fix-netbios-decode-bifs:
  Fixes to `decode_netbios_name` and `decode_netbios_name_type` BIFs
2021-04-30 09:40:46 -07:00
Jon Siwek
76fb1e7fd0 Fixes to decode_netbios_name and decode_netbios_name_type BIFs
Fixes to `decode_netbios_name`:

* Improve validation that input string is a NetBIOS encoding
  (32 bytes, with characters ranging from 'A' to 'P').  This helps
  prevent Undefined Behavior of left-shifting negative values.
  Invalid encodings now cause a return-value of an empty string.

* More liberal in what decoded characters are allowed.  Namely,
  spaces are now allowed (but any trailing null-bytes and spaces
  are trimmed, similar to before).

Fixes to `decode_netbios_name_type`:

* Improve validation that input string is a NetBIOS encoding
  (32 bytes, with characters ranging from 'A' to 'P').  This helps
  prevent Undefined Behavior of left-shifting negative values and
  a heap-buffer-overread when the input string is too small.
  Invalid encodings now cause a return-value of 256.
2021-04-27 15:27:04 -07:00
Benjamin Bannier
5b2bf374fd GH-1517: Add Geneve decap support
This patch adds the ability to decap Geneve packets to process the inner
payload. The structure of the analyzer borrows heavily from the VXLAN
analyzer.
2021-04-27 11:21:29 +02:00
Vlad Grigorescu
deae51891c Remove unneccessary @load 2021-04-26 23:02:57 -05:00
Vlad Grigorescu
ac720a1313 Add tests for e-mail actions, and cleanup the new logic a bit. 2021-04-26 23:01:34 -05:00
Vlad Grigorescu
6d0f48abfc Fixup priorities 2021-04-26 21:46:45 -05:00
Vlad Grigorescu
f8b06becd2 Add a new field to the notice, which defines where to send the e-mail 2021-04-26 21:40:27 -05:00
Vlad Grigorescu
3bb4a35200 Add page and email_admin to hostnames extension 2021-04-23 10:05:34 -05:00
Christian Kreibich
768b885791 Reference the manual's explanation of originator and responder from conn_id 2021-04-09 10:31:56 -07:00
Jon Siwek
cd8f19328a Merge remote-tracking branch 'origin/topic/neverlord/gh-1408'
- Minor adjustments to header includes, whitespace, and a
  compiler warning fix during merge

* origin/topic/neverlord/gh-1408:
  Add new Timer class to the telemetry API
  Fix build with latest VectorVal API
  Update baselines
  Prefix telemetry BIFs with __
  Expose telemetry API to scripting land
  Add handle types for histogram metrics
  Move duplicated code to detail header
  Adhere to Zeek coding style
  Apply suggestions from code review
  Add telemetry Manager to Zeek setup / teardown
  Add missing whitespaces for Zeek coding style
  Add gauge metric types
  Add scaffold for new metrics API with counters
2021-04-05 14:21:44 -07:00
Vern Paxson
0f18a01bf7 fixed out-of-range enum constant 2021-04-01 19:16:07 -07:00
Dominik Charousset
2bac354e43 Expose telemetry API to scripting land 2021-03-29 10:47:32 +02:00
Jon Siwek
7bf885b0b8 Merge remote-tracking branch 'origin/topic/vlad/expose_supervisor_rotation_func'
- Restored a deprecated version of 'supervisor_rotation_format_func'
  during merge.

* origin/topic/vlad/expose_supervisor_rotation_func:
  Rename supervisor_rotation_format_func to archiver_rotation_format_func, and expose it for non-supervised setups
2021-03-26 17:18:52 -07:00
Vlad Grigorescu
acfb21c5a6 Rename supervisor_rotation_format_func to archiver_rotation_format_func, and expose it for non-supervised setups
Closes #1463
2021-03-26 15:26:48 -05:00
Johanna Amann
c23e3ca105 Heartbleed: fix substraction order.
The larger number was substracted from the smaller one leading to an
integer overflow. However, no information was lost due to everything
also being present in the notice message.

Fixes GH-1454
2021-03-25 13:41:12 +00:00
Michael Dopheide
83d5b44462 Sometimes user_agent is missing 2021-03-24 15:48:09 -05:00
Tim Wojtulewicz
f45df63cd0 Merge remote-tracking branch 'origin/topic/vern/zval'
* origin/topic/vern/zval: (42 commits)
  whitespace tweaks
  resolved some TODO comments
  remove unnecessary casts, and change necessary ones to use static_cast<>
  explain cmp_func default
  change functions for ZVal type management to static members
  fix some unsigned/signed integer warnings
  address lint concern about uninitialized variable
  Remove use of obsolete forward-declaration macros
  fix #include's that lack zeek/ prefixes
  explicitly populate holes created in vectors
  fixes for now-incorrect assumption that GetField always returns an existing ValPtr
  memory management for assignment to vector elements
  memory management for assignment to record fields
  destructor cleanup from ZAM_vector/ZAM_record
  fix #include's that lack zeek/ prefixes
  overlooked another way in which vector holes can be created
  initialize vector holes to the correct corresponding type
  explicitly populate holes created in vectors
  fix other instances of GetField().get() assuming long-lived ValPtr's
  fix for now-incorrect assumption that GetField always returns an existing ValPtr
  ...
2021-03-23 20:44:19 -07:00
Tim Wojtulewicz
4f24c02cd5 Merge remote-tracking branch 'origin/topic/vern/cpp-prep-fixes'
* origin/topic/vern/cpp-prep-fixes:
  fix for associating current scope with the name of enums; name tidying
  avoid infinite recursion in same_type() if it is analyzing recursive types
  remove iffy reliance on type punning that relies on interpreter's behavior
  fixes for propagating optimization options, and pruning script function analysis
2021-03-18 12:53:14 -07:00
Vern Paxson
b473bc48e1 remove iffy reliance on type punning that relies on interpreter's behavior 2021-03-18 08:46:18 -07:00
Vern Paxson
5a8ba8c3e5 lint fixes: ensuring functions return values, robustness to nil Val's 2021-03-18 08:21:19 -07:00
Vern Paxson
e21c0f1115 correcting some simple typos 2021-03-18 08:19:46 -07:00
Seth Hall
db7aba0835 ts fields in SMB logs now default to network_time()
This avoids a problem identified by amanbansal2709 in pull
request #1288. I fixed it in a different way than that pull request
by making sure the ts field is always set so that this isssue doesn't
return in the future.
2021-03-10 13:52:44 +00:00
Jon Siwek
6af436aad3 GH-1426: Improve handling of Broker data store creation failures
Broker::create_master() and Broker::create_clone() now return
a valid value even when there's a failure to open the backend database
(e.g. SQLite filesystem error).  In that case, the returned value can
still be passed into other data store operations, but they'll fail
immediately with an error.  Broker::is_closed() can now also be used to
determine whether the data store creation calls failed.
2021-03-06 02:32:29 -08:00
Vern Paxson
b065582319 fix mis-typed record fields that happened to work in original RecordVal impl. 2021-02-24 16:33:17 -08:00
Jon Siwek
737d2c390b Support explicit disabling of file analyzers 2021-02-23 15:50:18 -08:00
Vern Paxson
f21f7b65d5 adding &is_used attribute for base scripts - not actually needed yet, but will be once optimization is added 2021-02-06 10:59:58 -08:00
Vern Paxson
4f2b138e72 removing unused assignments from base scripts 2021-02-06 10:59:37 -08:00