Commit graph

1081 commits

Author SHA1 Message Date
Jon Siwek
e50a8848ae Improve an input framework test
A race condition could cause unstable output: if the thread reading the
file is fast, often you see both "pred" functions execute and then both
"line" events execute with both entries already in the table, but if the
thread reading the file is slow, you see pred, event, pred, event, with
only one entry available in the first event.
2020-02-07 20:03:10 -08:00
Jon Siwek
5b74bbf99a Fix a btest's race condition by handling Pcap::file_done 2020-02-06 17:50:17 -08:00
Jon Siwek
4fbcca04e8 Improve btest timeouts
* Generally increase timeouts for tests that have recent transient
  failures

* Change any test that relied on `btest-bg-wait -k` since that's never
  going to play with with CI systems.  Instead, we always need to have
  a well-defined termination condition in the test itself (and most
  already did, so didn't really need the `-k` flag anyway).
2020-02-06 17:50:17 -08:00
Johanna Amann
50ebbc90ad Resolve race condition in test 2020-02-05 09:34:29 -08:00
Johanna Amann
deaab42187 Change threading formatter errors to warnings.
This change standardizes threading formatter error handling and moves
the remaining error calls to be warnings instead.

This is in line with already existing code - in most cases warnings were
raised, only a few cases raised errors. These cases do not differ
significantly from other cases in which warnings are raised.

This also fixes GH-692, in which misformatted lines prevent future file
parsing.

This commit also moves the FailWarn method that is used by both the
config and the ascii reader up to the ReaderBackend. Furthermore it
makes the Warning method of ReaderBackend respect the warning
suppression that is introduced by the FailWarn method.
2020-02-04 16:31:29 -08:00
Tim Wojtulewicz
2dcc936787 Test changes caused by minor order-of-operation changes related to the new loop architecture 2020-01-31 10:13:09 -07:00
Frerich Raabe
440468a0dc Adding test for decompose_uri parsing URI with empty port
I'd expect the portnum component to remain uninitialized in case the
given URI specifies no port number -- however, decompose_uri() raises an
error instead. Will address this in a subsequent commit.
2020-01-24 22:23:50 +01:00
Jon Siwek
1db7a222a0 Handle invalid Base64 encodings in FTP ADAT analyzer 2020-01-15 12:44:10 -08:00
Jon Siwek
6218f99b96 Update timeouts for a btest 2020-01-09 19:05:23 -08:00
Jon Siwek
1f3ec434a8 Extend ASAN_OPTIONS in btests instead of overwrite 2020-01-08 21:04:18 -08:00
Jon Siwek
66ca20b737 Increase timeout for a few btests
Saw these start being sensitive running an "ASan -O0" build locally
2020-01-08 10:31:16 -08:00
Jon Siwek
a1377e1d42 Increase timeout for a btest
May be more timing-sensitive with slower ASAN/LSAN configuration in
addition to already-slower CI environment.
2020-01-02 23:04:22 -08:00
Jon Siwek
e6f4e01041 Remove redundant memory leak btests
Or otherwise convert into a regular btest if it didn't already seem to
be covered.

There's no need for a separate memory leak test group since compiling
with LeakSanitizer now covers leak checking for the full btest suite.
2020-01-02 23:04:22 -08:00
Jon Siwek
0fe2a14d98 Disable LeakSanitizer for btests that have known leaks
E.g. ones that throw interpreter exceptions,  as those are currently
known to potentially cause leaks.  Fixing the underlying leaks involves
the larger task of more IntrusivePtr usage.

Reference cycles may also cause leaks.
2020-01-02 12:05:49 -08:00
Jon Siwek
d936607cc9 Fix scripting error in a DHCP btest 2020-01-02 12:05:49 -08:00
Jon Siwek
10473d58a9 Fix scripting error in an ftp btest 2020-01-01 12:47:39 -08:00
Jon Siwek
cd9fec7bdb Add Cirrus CI config 2019-12-17 15:28:25 -08:00
Jon Siwek
30d0b21ecc Merge remote-tracking branch 'origin/topic/dev/print-to-log'
Adjustments during merge:

- kept the UNKNOWN Log::ID as placeholder value
- changed the coverage.find-bro-logs test to check for arbitrary $path
  field values instead of just string literals
- don't force EnumVal to unsigned integer since the relevant union member
  is the signed integer and added the relevant enum values/types to
  .bif files for easier access
- compare FILE* versus file name to check for stdout equality (don't
  think it matters much, just a bit more efficient)
- minor whitespace/style tweaks

* origin/topic/dev/print-to-log:
  Added a non boolean configuration and other changes as suggested by Jon
  Allow Print Statements to be redirected to a Log# This is a combination of 3 commits.
2019-12-02 13:47:09 -08:00
Dev Bali
4be6871df2 Added a non boolean configuration and other changes as suggested by Jon 2019-11-26 21:53:21 -08:00
Johanna Amann
e5db1f085c Merge remote-tracking branch 'origin/topic/jsiwek/gh-684-fix-rpc-parsing'
* origin/topic/jsiwek/gh-684-fix-rpc-parsing:
  GH-684: Fix parsing of RPC calls with non-AUTH_UNIX flavors

Addresses GH-684
2019-11-21 08:50:28 -08:00
Dev Bali
c9016f1397 Allow Print Statements to be redirected to a Log# This is a combination of 3 commits. 2019-11-13 19:59:42 -08:00
Jon Siwek
37a478ae99 GH-684: Fix parsing of RPC calls with non-AUTH_UNIX flavors
The parsing logic that should be specific to the AUTH_UNIX credential
flavor was previously applied unconditionally to other flavors.
2019-11-13 13:14:14 -08:00
Jon Siwek
0c71715cf4 Adjust btests for OpenBSD portability 2019-11-08 17:32:44 -08:00
Jon Siwek
7965dcd041 Convert pcapng test suite files to pcap format
The former isn't supported by default on OpenBSD.
2019-11-08 13:08:06 -08:00
Robin Sommer
9d7c9f37d4 Merge branch 'Fix_http_build_url' of ssh://github.com/Olerdrive/zeek
I changed the format string of the port to %d and added a test.
2019-10-28 09:32:32 +00:00
Johanna Amann
66effde97d Merge remote-tracking branch 'origin/topic/seth/624'
* origin/topic/seth/624:
  Support whitespace at end of line for config reader.

This merge fixes a failing test; it also sprinkles a few more spaces
into another test file.

The main change is that this now also works with configuration lines
that don't have a value.
2019-10-14 21:47:59 -07:00
Seth Hall
cd06cb24cc Support whitespace at end of line for config reader.
This commit addresses issue #624
2019-10-14 11:43:16 -04:00
Jon Siwek
39b06e3cee GH-591: allow Config::set_value() to use empty/unspecified table/sets 2019-10-02 22:13:43 -07:00
Jon Siwek
fd66e7b9f6 GH-591: fix reading set[enum] values from input files 2019-10-02 22:09:16 -07:00
Jon Siwek
a5d71ed2d2 Merge remote-tracking branch 'origin/topic/timw/595-json-perf'
* origin/topic/timw/595-json-perf:
  Update COPYING.3rdparty
  Use json::emplace to avoid some extra calls to operator[]
  Use tessil/unordered-map instead of nlohmann/fifo-map to mitigate performance issues when logging JSON
2019-10-01 16:42:59 -07:00
Jon Siwek
97519e4350 Merge remote-tracking branch 'origin/topic/timw/606-json-nulls'
* origin/topic/timw/606-json-nulls:
  GH-606: Output nulls into json data if a field isn't set
2019-09-30 17:40:27 -07:00
Tim Wojtulewicz
d4c394b72a GH-606: Output nulls into json data if a field isn't set 2019-09-27 14:12:48 -07:00
Tim Wojtulewicz
756f363185 GH-598: handle multi-key sets correctly when outputting json 2019-09-27 12:01:47 -07:00
Tim Wojtulewicz
d4d1009b5c Use json::emplace to avoid some extra calls to operator[] 2019-09-27 09:58:24 -07:00
Jon Siwek
aeef4bf030 Merge branch 'topic/jgras/dpd-late-match' of https://github.com/J-Gras/zeek
* 'topic/jgras/dpd-late-match' of https://github.com/J-Gras/zeek:
  Improve dpd_late_match event generation.
  Improve logging of speculative service.
  Update test-all-policy script.
  Add speculative service script.
  Allow to handle late DPD matches.
2019-09-17 11:17:41 -07:00
Robin Sommer
6f9d1ec72d Merge remote-tracking branch 'origin/topic/jsiwek/gh-566-fix-ssh-encrypted-packet'
* origin/topic/jsiwek/gh-566-fix-ssh-encrypted-packet:
  GH-566: fix cases where ssh_encrypted_packet event wasn't raised
2019-09-17 17:22:27 +00:00
Jon Siwek
9c8db5f6ca Stop loading scan detection in local.zeek by default 2019-09-16 10:51:50 -07:00
Jon Siwek
30da2f83d0 GH-566: fix cases where ssh_encrypted_packet event wasn't raised
When encrypted data was bundled within the same segment as the NewKeys
message, it wasn't not reported via a ssh_encrypted_package event as
it should have been.
2019-09-03 17:34:24 -07:00
Jan Grashoefer
81b2b21211 Improve logging of speculative service. 2019-08-30 15:16:37 +02:00
Jan Grashoefer
788b56a652 Add speculative service script.
The speculative service script handles dpd_late_match events to extend
conn.log with infos about potential protocol identifications.
2019-08-29 11:47:04 +02:00
Johanna Amann
bb98559c0d Merge remote-tracking branch 'origin/topic/jsiwek/gh-545-weird-addl'
* origin/topic/jsiwek/gh-545-weird-addl:
  GH-545: add "addl" parameter to flow_weird and net_weird events
2019-08-28 14:27:53 -07:00
Johanna Amann
ec57894a85 Merge remote-tracking branch 'origin/topic/jsiwek/gh-541-ntlm-fix'
* origin/topic/jsiwek/gh-541-ntlm-fix:
  GH-541: add test cases for NTLM AV Pair sequence handling
  GH-541: fix handling of NTLM AV Pair sequences
2019-08-28 11:33:49 -07:00
Jon Siwek
289a1e2e8e Merge branch '555-smb3-negotiate-context-fix' of https://github.com/mad/zeek
- Fixed the context list padding to only be used for dialect 0x0311.
  The new test case includes an example where parsing the optional
  padding would fail for another dialect.

* '555-smb3-negotiate-context-fix' of https://github.com/mad/zeek:
  Fix for smb3 negotiate context
2019-08-27 10:08:42 -07:00
Pavel Ershov
de4a83206d Fix for smb3 negotiate context 2019-08-27 12:21:03 +03:00
Jon Siwek
b954767488 GH-541: add test cases for NTLM AV Pair sequence handling 2019-08-26 10:28:46 -07:00
Jon Siwek
2d7c926291 GH-545: add "addl" parameter to flow_weird and net_weird events 2019-08-20 22:45:22 -04:00
Jon Siwek
9ccf3549fd Merge remote-tracking branch 'origin/topic/timw/360-log-gzip'
* origin/topic/timw/360-log-gzip:
  Add new LogAscii::gzip_file_extension option.

Adjusted documentation during merge.
2019-08-13 13:49:45 -07:00
Jon Siwek
c91b12863e Add tests for {http,mime}_all_headers events
And remove unnecessary check for mime_all_headers in HTTP entities
(they ony raise the http_all_headers event, never mime_all_headers).
2019-08-13 11:28:00 -07:00
Jon Siwek
bf9b983f00 Merge branch 'known_services_multiprotocols' of https://github.com/mauropalumbo75/zeek
* 'known_services_multiprotocols' of https://github.com/mauropalumbo75/zeek:
  improve logging with broker store
  drop services starting with -
  remove service from key for Cluster::publish_hrw
  remove check for empty services
  update tests
  order list of services in store key
  remove repeated services in logs if already seen
  add multiprotocol known_services when Known::use_service_store = T
  remove hyphen in front of some services (for example -HTTP, -SSL)   In some cases, there is an hyphen before the protocol name in the field   connection$service. This can cause problems in known_services and   is removed here. It originates probably in some analyzer where it   would be better removed in the future.
  add multiprotocol known_services when Known::use_service_store = F

Changes during merge:
  * whitespace
  * add unit test
2019-08-09 10:47:34 -07:00
Jon Siwek
ba56cf62a7 Change version to 3.1.0-dev 2019-08-08 16:08:50 -07:00