* Variables of `string` type can now be set to an empty string
* Trying to set a variable with non-`string` type to an empty value
now emits an error instead of silently doing nothing
* Providing an invalid identifier now emits an "unknown identifier"
error instead of silently doing nothing
* origin/master: (33 commits)
Fix location where CI places build.tgz
Update submodule(s)
Disable some deprecation diagnostics for GCC
Compare pcap_next_ex() result to PCAP_ERROR/PCAP_ERROR_BREAK
Optimize Connection::RemovalEvent() for bare-mode usage
Rename BroType to Type
Update NEWS
Review cleanup
Move Type types to zeek namespace
Review cleanup
Restrict Cirrus CI to only zeek repo's branches
GH-977: Improve pcap error handling
Remove not-useful code in iosource::Manager::OpenPktSrc
GH-999: Stop formatting DHCP Client ID Hardware Type 0 as MAC
Remove inline from some static KeyedHash members
Improve Func.h inclusion
Fix NVT analyzer memory leak from multiple telnet authn name options
Rename aux/ to auxil/
Move Flare/Pipe from the bro namespace to zeek::detail
Move Attr to the zeek::detail namespace
...
This introduces a new sampling state-map for expired connections to fix
segfaults that previously occured when passing in a `connection` record
to `Reporter::conn_weird()` for which the internal `Connection` object
had already been expired and deleted. This also introduces a new event
called `expired_conn_weird`, which is similar to `conn_weird`, except
the full `connection` record is no longer available, just the `conn_id`
and UID string.
For `DHCP::ClientID$hwtype` fields equal to 0, the `hwaddr` field is
no longer misformatted as a MAC and instead just contains the raw bytes
seen in the DHCP Client ID Option.
This commit fixes a few more loose ends to actually make the
Zeek Table<->brokerstore syncing work. This mostly slightly changes the
TableVal assign/remove operators to prevent loops when a remote change
arrives.
The tests inserts a value into a table on the manager, and it pops out
in a table on a clone - which is the easiest case.
Timeouts are still not handled at all; the behavior when inserting into
a clone is untested.
This also updates all usages of the deprecated Val ctor to use
either IntervalVal, TimeVal, or DoubleVal ctors. The reason for
doing away with the old constructor is that using it with TYPE_INTERVAL
isn't strictly correct since there exists a more specific subclass,
IntervalVal, with overriden ValDescribe() method that ought to be used
to print such values in a more descriptive way.
The BIF was not returning an IntervalVal which has an overriden
ValDescribe() method that allows for prettier printing like "6.0 secs"
instead of just "6.0".
* origin/topic/jsiwek/gh-893-intrusive-ptr-migration: (151 commits)
Integrate review feedback
Switch Broker Val converter visitor to return IntrusivePtr
Change BroFunc ctor to take const-ref IntrusivePtr<ID>
Add version of Frame::SetElement() taking IntrusivePtr<ID>
Change Scope/Func inits from id_list* to vector<IntrusivePtr<ID>>
Change Scope::GenerateTemporary() to return IntrusivePtr
Deprecate Scope::ReturnType(), replace with GetReturnType()
Deprecate Scope::ScopeID(), replace with GetID()
Switch parsing to use vector<IntrusivePtr<Attr>> from attr_list
Deprecate TableVal::FindAttr(), replace with GetAttr()
Deprecate TypeDecl::FindAttr(), replace with GetAttr()
Deprecate ID::FindAttr(), replace with GetAttr()
Deprecate Attributes::FindAttr(), replace with Find()
Deprecate Attributes::AddAttrs(Attributes*)
Add Attributes ctor that takes IntrusivePtrs
Change Attributes to store std:vector<IntrusivePtr<Attr>>
Change Attr::SetAttrExpr() to non-template
Deprecate Attr::AttrExpr(), replace with GetExpr()
Deprecate ID::Attrs(), replace with GetAttrs()
Remove weak_ref param from ID::SetVal()
...
- Updated the logic significantly: still filters out ICMP from being
considered an active service (like before) and adds a new
"Known::service_udp_requires_response" option (defaults to true) for
whether to require UDP server response before being considered an
active service.
* 'topic/dopheide/known-services' of https://github.com/dopheide-esnet/zeek:
Log services with unknown protocols
This also changes the argument type of Func::operator() to zeek::Args*
to allow plugins to be able to alter function arguments in place as
was previously documented.
This commit refactors the SendEvent call and moves it from the Input
ReaderBackend to to MsgThread. This allows all other types of threads
to access this functionality.
This necessitated a few more changes. Most importantly, one of the
ValueToVal methods was moved over to SerialTypes. Whereit arguably
belongs - there was nothing that was input-framework specific in
that method - and the functionality could come in useful in a number
of cases.
* origin/topic/timw/906-find-all-urls-regex:
Restore previous url scheme capture group
GH-906: Fix the regex in url.zeek to better match for find_all_urls
- Minor whitespace/format adjustments
* 'network-time-init' of https://github.com/J-Gras/zeek:
Fix documentation for network_time_init.
Improve network_time_init() test and add baseline.
Add network_time_init() event.
Add test for network_time behavior.
- Changed the logic significantly to just guarantee there's only ever a
single inactivity timer per connection
- Updated language.expire_subnet btest which is unduly sensitive to
timer-related changes
* origin/topic/jazoff/avoid-redundant-inactivity-timers:
avoid scheduling redundant inactivity timers
This commit switches UID hashing from md5 to a highway hash. It also
moves the salt value out of the file plugin - and makes it
installation-specific instead - it is moved to the global namespace.
There now are digest hash functions to make "static"
installation-specific hashes that are stable over workers available to
everyone; hashes can be 64, 128 or 256 bits in size.
Due to the fact that we switch the file hashing algorithm, all file
hashes change.
The underlyigng algorithm that is used for hashing is highwayhash-128,
which is significantly faster than md5.
* origin/topic/seth/more-file-type-ident:
Adds a note in NEWS about the signature addition and restructuring
Fixes a small bug in one signature with a duplicate name.
Organized and added to the shipped file identification signatures.
The places that used md5 basically already used it as a digest
algorithm. Switching to a digest just means that the internal values
used to not change between runs - which is actually wanted in these
cases.
This commit also removes our special cmake subdirectory. We don't expose
highwayhash in headers anymore - so we can just treat it as an internal
implementation choice that is not directly exposed to plugins.
- Added test case and adjusted whitespace in merge
* 'stats-logging-fix' of https://github.com/brittanydonowho/zeek:
Fixed stats.zeek to log all data before zeek terminates rather than return too soon
- Minor whitespace adjutment in merge
* origin/topic/vern/any-typetype-when-fix:
bug fixes for using "when" in functions that have a local of type "any"
Currently, siphash is used for strings up to 36 bytes. hmac-md5 is used
for longer strings.
This switch-over is a remnant of the previous hash-function that was
used, which apparently was slower with longer input strings.
This change serves no purpose anymore. I performed a few performance tests
on strings of varying sizes:
For a 40 byte string with 10 million iterations:
siphash: 0.31 seconds
hmac-md5: 3.8 seconds
For a 1080 byte string with 10 million iterations:
siphash: 4.2 seconds
hmac-md5: 17 seconds
For a 18360 byte string with 10 million iterations:
siphash: 69 seconds
hmac-md5: 240 seconds
Hence, this commit removes the use of hmac-md5.
This change causes reordering of lines in a few logs.
This commit also changes the datastructure for the seed in probabilistic/Hasher
to get rid of a type-punning warning.
If a bloomfilter doesn't have a type, that just means no
bloomfilter_add() has been called yet, so seems undesirable to emit an
error for a lookup against something that's known to be empty.
The "http_header" event now has an "original_name" parameter that allows
access to the original header name (the "name" parameter reamins the
same as before: it's the uppercased header name).
The "mime_header_rec" record type now also includes an "original_name"
field to similarly provide access to original header name in the
following events: "http_all_headers", "mime_one_header", and
"mime_all_headers".
Node-specific topic prefix subscriptions/publications now add a trailing
slash like "zeek/cluster/node/<name>/". Without the trailing slash,
messages attempting to target "proxy-10" may also be sent to "proxy-1"
since subscription matching is prefix-based.
For event/hook handlers that had a previous declaration, any &default
arguments are ineffective. Only &default uses in the initial
prototype's arguments have an effect (that includes if the handler
is actually the site at which the declaration occurs).
When the only existing handler doesn't use the full argument list, still
have to ensure the full frame is allocated because all arguments get set
regardless.
This fixes it to again allow the old behavior of matching a handler
against the canonical prototype as long as all argument types, but not
necessarily names, match.
