Commit graph

11634 commits

Author SHA1 Message Date
Tim Wojtulewicz
54752ef9a1 Deprecate the internal int/uint types in favor of the cstdint types they were based on 2019-08-12 13:50:07 -07:00
Jon Siwek
31d30bb47e GH-535: fix typo of "C_HESIOD" in DNS::classes
Fixes GH-535
2019-08-12 13:32:28 -07:00
Jon Siwek
18e4976c6c Merge remote-tracking branch 'origin/topic/timw/travis-leaks'
* origin/topic/timw/travis-leaks:
  fixup! Add new distro to Travis CI configuration for running leak tests
  Add new distro to Travis CI configuration for running leak tests
2019-08-12 13:01:55 -07:00
Jon Siwek
a1c5eddb95 GH-532: improve disable_analyzer BIF
- Add an extra "prevent" parameter (default value of false), which
  helps prevent the same analyzer type from being attached in the
  future.  It's useful in situations where you want to disable early
  on, but a DPD signature may still trigger later and re-attach
  the same analyzer.  E.g. when not using this flag, but calling
  disable_analyzer() inside an http_request event, will remove the
  HTTP analyzer that was attached due to well-known-port, but a later
  DPD signature match from upon seeing the HTTP reply will end up
  attaching another HTTP analyzer.  More surprising is that upon
  re-attaching that analyzer, you'll get the same http_request as
  before since the DPD buffer will get replayed into the new analyzer.

- Fixes disable_analyzer() to work when called even earlier, like
  within the protocol_confirmation event.  At that time, the
  Analyzer tree may have not properly added the new analyzer into
  Analyzer::children yet, but rather the temporary waiting list,
  Analyzer::new_children.  Analyzer::RemoveChildAnalyzer previously
  did not inspect the later list.

- Fixes disable_analyzer() when called on an analyzer added to the
  tree via TCP_Analyzer::AddChildPacketAnalyzer.  TCP_Analyzer
  keeps track of such children in its own list,
  TCP_Analyzer::packet_children, which the previous
  Analyzer::RemoveChildAnalyzer implementation didn't inspect.
2019-08-09 20:03:26 -07:00
Tim Wojtulewicz
b9f6992c0e fixup! Add new distro to Travis CI configuration for running leak tests 2019-08-09 11:01:16 -07:00
Jon Siwek
bf9b983f00 Merge branch 'known_services_multiprotocols' of https://github.com/mauropalumbo75/zeek
* 'known_services_multiprotocols' of https://github.com/mauropalumbo75/zeek:
  improve logging with broker store
  drop services starting with -
  remove service from key for Cluster::publish_hrw
  remove check for empty services
  update tests
  order list of services in store key
  remove repeated services in logs if already seen
  add multiprotocol known_services when Known::use_service_store = T
  remove hyphen in front of some services (for example -HTTP, -SSL)   In some cases, there is an hyphen before the protocol name in the field   connection$service. This can cause problems in known_services and   is removed here. It originates probably in some analyzer where it   would be better removed in the future.
  add multiprotocol known_services when Known::use_service_store = F

Changes during merge:
  * whitespace
  * add unit test
2019-08-09 10:47:34 -07:00
Jon Siwek
a68c9f6b71 Merge branch 'empty_services' of https://github.com/mauropalumbo75/zeek
* 'empty_services' of https://github.com/mauropalumbo75/zeek:
  remove empty services and include udp active connections when logging in connection_state_remove
2019-08-09 09:59:50 -07:00
Jon Siwek
acd1be6e7b Merge branch 'export_intel_events' of https://github.com/mauropalumbo75/zeek
* 'export_intel_events' of https://github.com/mauropalumbo75/zeek:
  minor restyle and add comments
  add an empty read_error event to the intel framework (in the export block, so that users can implement further checks with it)
  move event Intel::read_entry to export block

Adjusted whitespace in merge.
2019-08-09 09:36:19 -07:00
Jon Siwek
d726e44e0e Updating CHANGES and VERSION. 2019-08-09 09:15:22 -07:00
Tim Wojtulewicz
80e3c42526 Add new distro to Travis CI configuration for running leak tests 2019-08-08 16:36:17 -07:00
Jon Siwek
ba56cf62a7 Change version to 3.1.0-dev 2019-08-08 16:08:50 -07:00
Jon Siwek
d236f377e7 Add memory leak test of closure (un)serialization 2019-08-07 12:41:22 -07:00
Jon Siwek
662c416ad9 Updating submodule(s).
[nomail]
2019-08-07 10:29:16 -07:00
Jon Siwek
6af1bf18ac Replace use of deprecated pcap_lookupdev()
libpcap 1.9.0 has started deprecating this function in favor
of pcap_findalldevs()
2019-08-07 09:46:41 -07:00
Johanna Amann
51f565d235 Merge remote-tracking branch 'origin/topic/jsiwek/gh-527-coverity-issues'
* origin/topic/jsiwek/gh-527-coverity-issues:
  GH-527: fix parsing of MQTT Remaining Length field
  GH-527: fix LambdaExpr::Traverse
  GH-527: fix ref-counting issues in Frame unserialization
2019-08-07 08:10:29 -07:00
Mauro Palumbo
f93e41fd3f minor restyle and add comments 2019-08-07 10:32:36 +02:00
Jon Siwek
dc5aca6854 GH-527: fix parsing of MQTT Remaining Length field
Packet length is encoded in up to four bytes, with MSB (0x80)
indicating if there's more bytes in the representation still to follow.
The comparison/bitwise-mask wasn't correctly testing the MSB.

Coverity CID 1403964
2019-08-06 14:47:03 -07:00
Jon Siwek
f6f4e3e8bc GH-527: fix LambdaExpr::Traverse
Coverity CID 1403966
2019-08-06 14:47:03 -07:00
Jon Siwek
05bc680d3f GH-527: fix ref-counting issues in Frame unserialization
Coverity CIDs 1403968, 1403967
2019-08-06 14:47:03 -07:00
Johanna Amann
7521fec5b3 Update submodule
[nomail]
2019-08-06 11:49:12 -07:00
Mauro Palumbo
8e418d3c7b add an empty read_error event to the intel framework (in the export block, so that users can implement further checks with it) 2019-08-06 12:20:47 +02:00
Mauro Palumbo
1011abd5e0 move event Intel::read_entry to export block 2019-08-06 11:17:34 +02:00
Johanna Amann
3080290a5e Merge remote-tracking branch 'origin/topic/jsiwek/gh-474-mqtt-option'
* origin/topic/jsiwek/gh-474-mqtt-option:
  GH-474: change MQTT::max_payload_size to be a runtime option

Relates to GH-474
2019-08-05 21:06:50 -07:00
Jon Siwek
cc91ab5d9e Fix malformed SMB documentation 2019-08-05 19:00:43 -07:00
Jon Siwek
1eb1771c25 Fix documentation warnings for MQTT identifiers
I.e. a type was not in the export section, but a field was added
to connection record via a redef that uses the "hidden" type.
That generally doesn't help to hide it that way since a user comes
to rely on it indirectly anyway, and it also causes problems with
the Zeekygen documentation not being able to find it.
2019-08-05 18:55:48 -07:00
Jon Siwek
35c42b4b09 GH-474: change MQTT::max_payload_size to be a runtime option 2019-08-05 18:11:54 -07:00
Jon Siwek
704969ddd6 Merge remote-tracking branch 'origin/topic/johanna/disable-mqq-by-default'
* origin/topic/johanna/disable-mqq-by-default:
  Disable MQTT by default
2019-08-05 17:33:25 -07:00
Johanna Amann
0f96a9dedf Disable MQTT by default
To enable MQTT, one has to load policy/scripts/mqtt. Like with smb in
2.5, the consts are loaded by default.
2019-08-05 17:04:39 -07:00
Tim Wojtulewicz
337da50da6 Add new LogAscii::gzip_file_extension option.
This can be used with the LogAscii::gzip_level option to set the file extension of log files when they are compressed at creation time.
2019-08-05 14:36:34 -07:00
Johanna Amann
39b9468f9d Merge remote-tracking branch 'origin/topic/jsiwek/gh-474-mqtt-improvements'
* origin/topic/jsiwek/gh-474-mqtt-improvements:
  GH-474: add MQTT::max_payload_size option
  GH-474: use topic vectors for MQTT (un)subscribe events/logs

Relates to #520
2019-08-05 10:26:40 -07:00
Jon Siwek
5f4c04c900 Updating submodule(s).
[nomail]
2019-08-02 18:24:51 -07:00
Jon Siwek
6bc947a48e GH-474: add MQTT::max_payload_size option
This caps size of payload strings within mqtt_publish events and
mqtt_publish.log files.  A new "payload_len" field in the log file
shows the real payload size in cases where it may have been truncated.
2019-08-02 14:28:55 -07:00
Jon Siwek
c43e809a69 GH-474: use topic vectors for MQTT (un)subscribe events/logs 2019-08-02 13:48:43 -07:00
Johanna Amann
649d9f502b Update Certificate Transparency list 2019-08-02 12:43:43 -07:00
Johanna Amann
6f25125443 Update CA store to NSS 3.45
This also required updating a test that required a root-certificate that
was removed from the Mozilla store - the test now directly includes that
specific root-cert.
2019-08-02 12:36:54 -07:00
Jon Siwek
d2eed166bd GH-517: fix MQTT suback/unsuback accessing non-existent index 2019-08-02 11:57:46 -07:00
Jon Siwek
b3884de2e5 Fix how Broker/CAF sleep duration options are set
With CAF 0.17.0, these are now reporting an incorrect option name
was being used.
2019-08-02 11:33:00 -07:00
Tim Wojtulewicz
bbf49406c1 DFA: remove uses of PDict 2019-08-02 09:45:50 -07:00
Tim Wojtulewicz
acff8d5a2b EventRegistry: remove uses of PDict 2019-08-02 09:45:50 -07:00
Johanna Amann
6fa0f4ac49 Merge remote-tracking branch 'origin/topic/johanna/conn-duration-thresholds'
* origin/topic/johanna/conn-duration-thresholds:
  Add duration thresholding to the conn-size analyzer.
2019-08-01 14:20:49 -07:00
Johanna Amann
f9ee0079a5 Merge remote-tracking branch 'origin/topic/jsiwek/lambda-name-fixes'
* origin/topic/jsiwek/lambda-name-fixes:
  Guarantee unique internal name for each lambda function
  Use consistent hashing method for internal lambda function names
2019-08-01 13:37:45 -07:00
Johanna Amann
ca36728a4e Merge remote-tracking branch 'origin/topic/jsiwek/gh-514-improve-addr-conversion-errors'
* origin/topic/jsiwek/gh-514-improve-addr-conversion-errors:
  Improve error messages from to_addr and to_subnet BIFs
2019-08-01 13:15:43 -07:00
Jon Siwek
70359c703f Fix a test that used a hardcoded Broker port 2019-08-01 12:28:50 -07:00
Jon Siwek
21bc1b6703 Updating submodule(s).
[nomail]
2019-08-01 12:07:26 -07:00
Johanna Amann
9d489cde20 Add duration thresholding to the conn-size analyzer.
Now, in addition to setting thresholds for bytes and packet, one can set
a threshold for connection duration. Note that the threshold event is
only raised once the next packet in the connection is seen.

This also fixes a small pre-existing bug, in which a bunch of warnings
were raised if someone just used the lower-level functions without going
through the higher-level scripting API.
2019-08-01 11:57:40 -07:00
Jon Siwek
11f90bc9f5 GH-512: add --mandir configure option 2019-08-01 11:26:18 -07:00
Jon Siwek
68b0e1d54d Updating submodule(s).
[nomail]
2019-08-01 11:22:26 -07:00
Jon Siwek
ec4df80c67 Improve error messages from to_addr and to_subnet BIFs
Related to GH-514
2019-08-01 10:49:03 -07:00
Jon Siwek
ac7daf8456 Merge branch 'master' of https://github.com/zeek/zeek 2019-07-31 21:31:55 -07:00
Jon Siwek
ee28e9e9f3 Merge remote-tracking branch 'origin/topic/seth/mqtt'
* origin/topic/seth/mqtt:
  Bug fixes and test baseline updates
  Fix an issue with bro_init -> zeek_init
  MQTT Analyzer heavily updated and ported from the analyzer originally by Supriya Kumar

Adjustments during merge:

* Minor whitespace cleanups
* Some bro to zeek renaming
* Fixed the parsing of unsubscribe messages to generate an event for each topic
2019-07-31 21:29:38 -07:00