Commit graph

3074 commits

Author SHA1 Message Date
Jon Siwek
b776f17cea Merge remote-tracking branch 'origin/topic/vern/any-typetype-when-fix'
- Minor whitespace adjutment in merge

* origin/topic/vern/any-typetype-when-fix:
  bug fixes for using "when" in functions that have a local of type "any"
2020-04-27 13:33:40 -07:00
Johanna Amann
faa8a38578 Merge remote-tracking branch 'origin/topic/jsiwek/gh-854-preserve-header-name'
* origin/topic/jsiwek/gh-854-preserve-header-name:
  GH-854: provide access to original HTTP/MIME header names
2020-04-27 19:31:49 +00:00
Johanna Amann
0136648342 Merge remote-tracking branch 'origin/topic/jsiwek/empty-bloomfilter-lookup'
* origin/topic/jsiwek/empty-bloomfilter-lookup:
  Remove error message from empty bloomfilter lookups
2020-04-27 19:23:16 +00:00
Vern Paxson
b86d5b4424 bug fixes for using "when" in functions that have a local of type "any" 2020-04-25 16:56:24 -07:00
Jon Siwek
f452f26d11 Remove error message from empty bloomfilter lookups
If a bloomfilter doesn't have a type, that just means no
bloomfilter_add() has been called yet, so seems undesirable to emit an
error for a lookup against something that's known to be empty.
2020-04-24 10:15:57 -07:00
Jon Siwek
5032993b94 GH-854: provide access to original HTTP/MIME header names
The "http_header" event now has an "original_name" parameter that allows
access to the original header name (the "name" parameter reamins the
same as before: it's the uppercased header name).

The "mime_header_rec" record type now also includes an "original_name"
field to similarly provide access to original header name in the
following events: "http_all_headers", "mime_one_header", and
"mime_all_headers".
2020-04-20 16:56:41 -07:00
Jon Siwek
81517bd703 Update deprecated BifEvent::generate_* usages 2020-04-17 18:42:58 -07:00
Tim Wojtulewicz
ba1c03188f Merge remote-tracking branch 'origin/topic/jsiwek/alternate-hook-event-prototypes'
* origin/topic/jsiwek/alternate-hook-event-prototypes:
  Add warning for ineffective &default arguments in handlers
  Fix frame size allocation of alternate event/hook handlers
  Emit error for alternate event/hook prototype args with attributes
  Improve alternate event/hook prototype matching
  Allow alternate event/hook prototype declarations
2020-04-13 15:00:25 -07:00
Jon Siwek
ce9183a2ed Fix Broker topics used to uniquely identify cluster nodes
Node-specific topic prefix subscriptions/publications now add a trailing
slash like "zeek/cluster/node/<name>/".  Without the trailing slash,
messages attempting to target "proxy-10" may also be sent to "proxy-1"
since subscription matching is prefix-based.
2020-04-10 14:36:00 -07:00
Jon Siwek
640dbea57c Add warning for ineffective &default arguments in handlers
For event/hook handlers that had a previous declaration, any &default
arguments are ineffective.  Only &default uses in the initial
prototype's arguments have an effect (that includes if the handler
is actually the site at which the declaration occurs).
2020-04-09 22:51:01 -07:00
Jon Siwek
9243341e8c Fix frame size allocation of alternate event/hook handlers
When the only existing handler doesn't use the full argument list, still
have to ensure the full frame is allocated because all arguments get set
regardless.
2020-04-09 22:05:34 -07:00
Jon Siwek
070b28ac05 Emit error for alternate event/hook prototype args with attributes
Argument attributes are only allowed in the canonical prototype.
2020-04-09 21:00:09 -07:00
Jon Siwek
9b6934eab8 Improve alternate event/hook prototype matching
This fixes it to again allow the old behavior of matching a handler
against the canonical prototype as long as all argument types, but not
necessarily names, match.
2020-04-09 20:20:38 -07:00
Johanna Amann
876c803d75 Merge remote-tracking branch 'origin/topic/timw/776-using-statements'
* origin/topic/timw/776-using-statements:
  Remove 'using namespace std' from SerialTypes.h
  Remove other using statements from headers
  GH-776: Remove using statements added by PR 770

Includes small fixes in files that changed since the merge request was
made.

Also includes a few small indentation fixes.
2020-04-09 13:31:07 -07:00
Tim Wojtulewicz
d53c1454c0 Remove 'using namespace std' from SerialTypes.h
This unfortunately cuases a ton of flow-down changes because a lot of other
code was depending on that definition existing. This has a fairly large chance
to break builds of external plugins, considering how many internal ones it broke.
2020-04-07 15:59:59 -07:00
Johanna Amann
4ee84b69f2 Merge remote-tracking branch 'origin/topic/jsiwek/udp_content_delivery_ports_use_resp'
* origin/topic/jsiwek/udp_content_delivery_ports_use_resp:
  Add "udp_content_ports" option
  Add new "udp_content_delivery_ports_use_resp" option
2020-04-07 14:28:49 -07:00
Johanna Amann
8e4656bc61 Merge remote-tracking branch 'origin/topic/jsiwek/fix-duration-threshold'
* origin/topic/jsiwek/fix-duration-threshold:
  Fix connection duration thresholding
2020-04-07 13:44:56 -07:00
Jon Siwek
c1b3c9593b Add "udp_content_ports" option
Any port in that set found as either source or destination port
of a UDP packet will cause the "udp_contents" event to be raised.
2020-04-07 13:02:29 -07:00
Tim Wojtulewicz
08fbdb1418 Merge remote-tracking branch 'origin/topic/jsiwek/fix-patterns-as-table-index'
* origin/topic/jsiwek/fix-patterns-as-table-index:
  Fix using patterns stored as table/set indices
2020-04-07 12:05:44 -07:00
Jon Siwek
da56cd44d9 Fix using patterns stored as table/set indices
Pattern values used as table/set indices are stored in serialized form
using just the pattern text, but re-creating the value from that didn't
fully initialize/compile the regex matcher after (re-)creating it from
that pattern text.
2020-04-06 17:12:33 -07:00
Jon Siwek
2da84020cf Add new "udp_content_delivery_ports_use_resp" option
This controls whether ports given by "udp_content_delivery_ports_orig" and
"udp_content_delivery_ports_orig" are in terms of the UDP packet's
destination port or by the Connection's "responder" port (the former is
the unchanged default behavior).
2020-04-06 14:51:34 -07:00
Jon Siwek
bc5844b24c Fix connection duration thresholding
Connection durations were being compared "less than" instead of "greater
than" the desired duration threshold as provided to
set_current_conn_bytes_threshold() or
ConnThreshold::set_duration_threshold()
2020-04-06 13:32:32 -07:00
Anthony Kasza
19bb5210f4 change: -C flag to btest 2020-04-06 10:17:37 -06:00
Anthony Kasza
980e6299d3 add: ignore checksum added to btest script 2020-04-06 10:17:06 -06:00
Anthony Kasza
d10c9ab667 change: update btest baselines for rdpeudp-handshake-success 2020-04-03 16:11:18 -06:00
Anthony Kasza
243095fd9b add: rdpeudp2-handshake-success btest baseline 2020-04-03 16:09:53 -06:00
Anthony Kasza
d38599ecb4 add: btest for rdpeudp2 2020-04-03 11:48:26 -06:00
Anthony Kasza
9b1b8ae292 add: test trace for when RDPEUDP negotiates RDPEUDP2 2020-04-03 09:59:39 -06:00
Jon Siwek
e66148a13a Merge branch 'topic/ak/rdpeudp'
- Squashed the original commit set
- Cleaned up formatting
- Fixed register_for_ports() for right RDPEUDP analyzer

* topic/ak/rdpeudp:
  Add RDP over UDP analyzer
2020-04-02 18:31:40 -07:00
Anthony Kasza
60644bc85f Add RDP over UDP analyzer 2020-04-02 17:53:47 -07:00
Jon Siwek
8c0e8ecd28 Allow alternate event/hook prototype declarations
The alternates must be some subset of the canonical prototype (the one
that's first declared) and allows users to define handlers for any
such prototype.  Example:

    # Prototype declarations
    global my_event: event(s: string, c: count);
    global my_event: event(c: count);
    global my_event: event();

    # Handler definitions
    event my_event(s: string, c: count) { print s, c; }
    event my_event(c: count) { print c; }
    event my_event() { }

This allows handlers to consume a subset of the arguments or even
re-order them.  This makes it easier to either extend an existing
event/hook's arguments and/or deprecate usages of certain prototypes.
2020-04-01 17:25:28 -07:00
Johanna Amann
3ce1c9ffd6 Merge remote-tracking branch 'origin/topic/jsiwek/misc-lambda-fixes'
* origin/topic/jsiwek/misc-lambda-fixes:
  GH-725: fix logic for finding a lambda's usage of outer IDs
  Change record field anonymous functions to use lambda expressions
2020-03-31 15:46:54 -07:00
Johanna Amann
3c470ffe13 Merge remote-tracking branch 'origin/topic/jsiwek/gh-887-fix-gre-erspan-dot1q'
* origin/topic/jsiwek/gh-887-fix-gre-erspan-dot1q:
  GH-887: improve GRE/ERSPAN parsing of non-IPv4/IPv6 inner payload
  Fix ERSPAN III optional sub-header parsing
2020-03-30 14:30:00 -07:00
Johanna Amann
2e6eb7795c Merge remote-tracking branch 'origin/topic/jsiwek/tcp-hdr-reserved-bits'
* origin/topic/jsiwek/tcp-hdr-reserved-bits:
  Add a "reserved" field to tcp_hdr record
2020-03-30 13:23:51 -07:00
Jon Siwek
b7dee712d5 GH-887: improve GRE/ERSPAN parsing of non-IPv4/IPv6 inner payload
This changes the decapsulation logic for GRE/ERSPAN payloads to re-use
existing Layer 2 parsing logic that already handles things like 802.1Q
tags correctly before going on to process the inner IPv4/IPv6 payload.
2020-03-27 15:22:00 -07:00
Tim Wojtulewicz
85c6c2b9ee Merge remote-tracking branch 'origin/topic/jsiwek/runtime-exception-leak-cleanup'
* origin/topic/jsiwek/runtime-exception-leak-cleanup:
  Func::DescribeDebug: move a NumFields() call out of loop
  Use const-ref parameter for zeek::val_list_to_args()
  Fix missing IntrusivePtr.h include and ambiguous ODesc::Add call
  Remove TimerMgr arg from event queuing/scheduling methods
  Deprecate Analyzer::ConnectionEvent()
  Deprecate file_analysis::File::FileEvent methods using val_list args
  Deprecate Connection::ConnectionEvent methods
  Deprecate EventMgr::QueueEventFast() and update usages to Enqueue()
  Deprecate EventMgr::QueueEvent() and update usages to Enqueue()
  Deprecate Func::Call(val_list*, ...)
  Use vector<IntrusivePtr<Val>> for Func::Call and Event queuing args
  Fix memory leak in Zeek when-statement bodies with runtime errors
  Change TableVal::RecoverIndex() to return IntrusivePtr
  Use IntrusivePtr in TableVal::CallExpireFunc
  Fix memory leak when runtime error occurs in a Zeek for-loop
  Enable leak checks for btests that produce runtime exceptions
2020-03-27 11:49:45 -07:00
Jon Siwek
1ca11f11c7 GH-725: fix logic for finding a lambda's usage of outer IDs 2020-03-26 17:05:59 -07:00
Jon Siwek
f032885085 Change record field anonymous functions to use lambda expressions
There was an alternate syntax to assign anonymous functions to record
fields that was never migrated to use the new lambda expression
machinery (and so didn't allow referencing variables in outer scope):

    type myrec: record {
        foo: function(a: string);
    };

    local o = "o";
    local mr = myrec($foo(a: string) = { print a + o; });
2020-03-26 15:48:18 -07:00
Jon Siwek
13391806ba Add a "reserved" field to tcp_hdr record
This record is available by handling either raw_packet or new_packet
events.
2020-03-26 11:48:53 -07:00
Johanna Amann
8a4ccabc3e Merge remote-tracking branch 'origin/topic/jsiwek/fix-greedy-flowbuffer-bounds-check'
* origin/topic/jsiwek/fix-greedy-flowbuffer-bounds-check:
  Add test case for binpac flowbuffer frame length parsing bug
2020-03-24 13:49:58 -07:00
Jon Siwek
98d94ec785 Enable leak checks for btests that produce runtime exceptions
These were previously reporting leaks due to various allocations not
getting cleaned up during the stack unwind, but at the current state of
the transition toward IntrusivePtr usage, theses tests no longer leak.
2020-03-23 21:51:12 -07:00
Jon Siwek
427150b27a Merge remote-tracking branch 'origin/topic/jsiwek/gh-857-rebuild-tables-on-record-redef'
* origin/topic/jsiwek/gh-857-rebuild-tables-on-record-redef:
  Initialize RecordVal default fields when redef'd
  GH-857: fix redefining record types used to index tables
  Change RecordVals to get resized at time of RecordType redef
2020-03-20 11:31:11 -07:00
Tim Wojtulewicz
535c70b087 Merge remote-tracking branch 'origin/topic/jsiwek/gh-865-smb-negotiate-context-list'
* origin/topic/jsiwek/gh-865-smb-negotiate-context-list:
  GH-865: fix parsing of SMB NegotiateContextList
2020-03-20 10:05:34 -07:00
Jon Siwek
7e57f0788c Add test case for binpac flowbuffer frame length parsing bug 2020-03-19 22:09:23 -07:00
Jon Siwek
b62727a7fa Merge branch 'intrusive_ptr' of https://github.com/MaxKellermann/zeek
* 'intrusive_ptr' of https://github.com/MaxKellermann/zeek: (32 commits)
  Scope: store IntrusivePtr in `local`
  Scope: pass IntrusivePtr to AddInit()
  DNS_Mgr: use class IntrusivePtr
  Scope: use class IntrusivePtr
  Attr: use class IntrusivePtr
  Expr: check_and_promote_expr() returns IntrusivePtr
  Frame: use class IntrusivePtr
  Val: RecordVal::LookupWithDefault() returns IntrusivePtr
  Type: RecordType::FieldDefault() returns IntrusivePtr
  Val: TableVal::Delete() returns IntrusivePtr
  Type: base_type() returns IntrusivePtr
  Type: init_type() returns IntrusivePtr
  Type: merge_types() returns IntrusivePtr
  Type: use class IntrusivePtr in VectorType
  Type: use class IntrusivePtr in EnumType
  Type: use class IntrusivePtr in FileType
  Type: use class IntrusivePtr in TypeDecl
  Type: make TypeDecl `final` and the dtor non-`virtual`
  Type: use class IntrusivePtr in TypeType
  Type: use class IntrusivePtr in FuncType
  ...
2020-03-17 22:51:46 -07:00
Jon Siwek
9c70bcecbc GH-865: fix parsing of SMB NegotiateContextList
* The compression capability was incorrectly set to 0x0004 instead of 0x0003

* The padding was 4-byte instead of 8-byte aligned and also the spec.
  does not strictly require the padding for the last item in the list.

* Add a default case to handle parsing of unknown context types.
2020-03-16 19:00:01 -07:00
Jon Siwek
7a397d2746 Initialize RecordVal default fields when redef'd
If a RecordVal had been created, but later its RecordType redef'd to
contain fields with &default, those fields were incorrectly left
uninitialized.
2020-03-12 18:46:37 -07:00
Jon Siwek
71b82595ba GH-857: fix redefining record types used to index tables
This change tracks all TableVals created at parse-time whose index
depends on a given RecordType.  Should that RecordType be redef'd, those
TableVals are immediately rebuilt such that they are valid to
subsequently use in either parse-time initializations or eventually in
any arbitrary run-time expression.
2020-03-12 18:25:48 -07:00
Johanna Amann
da5fca7163 Merge branch 'topic/johanna/gh-744-cache-the-heck-out-of-these-certs'
* topic/johanna/gh-744-cache-the-heck-out-of-these-certs:
  X509 caching API change: callback gets entry directly
  Address feedback
  Small default updates & external baselines.
  X509 caching: small API changes, tests & test updates
  Certificate caching - now working in principle.
  First (sadly not completely working) approach at caching X509 certificates
2020-03-12 13:44:07 -07:00
Jon Siwek
860f851a66 Update pointer to external test repo 2020-03-12 12:04:24 -07:00