* origin/topic/timw/906-find-all-urls-regex:
Restore previous url scheme capture group
GH-906: Fix the regex in url.zeek to better match for find_all_urls
- Minor whitespace/format adjustments
* 'network-time-init' of https://github.com/J-Gras/zeek:
Fix documentation for network_time_init.
Improve network_time_init() test and add baseline.
Add network_time_init() event.
Add test for network_time behavior.
- Changed the logic significantly to just guarantee there's only ever a
single inactivity timer per connection
- Updated language.expire_subnet btest which is unduly sensitive to
timer-related changes
* origin/topic/jazoff/avoid-redundant-inactivity-timers:
avoid scheduling redundant inactivity timers
This commit switches UID hashing from md5 to a highway hash. It also
moves the salt value out of the file plugin - and makes it
installation-specific instead - it is moved to the global namespace.
There now are digest hash functions to make "static"
installation-specific hashes that are stable over workers available to
everyone; hashes can be 64, 128 or 256 bits in size.
Due to the fact that we switch the file hashing algorithm, all file
hashes change.
The underlyigng algorithm that is used for hashing is highwayhash-128,
which is significantly faster than md5.
* origin/topic/seth/more-file-type-ident:
Adds a note in NEWS about the signature addition and restructuring
Fixes a small bug in one signature with a duplicate name.
Organized and added to the shipped file identification signatures.
The places that used md5 basically already used it as a digest
algorithm. Switching to a digest just means that the internal values
used to not change between runs - which is actually wanted in these
cases.
This commit also removes our special cmake subdirectory. We don't expose
highwayhash in headers anymore - so we can just treat it as an internal
implementation choice that is not directly exposed to plugins.
- Added test case and adjusted whitespace in merge
* 'stats-logging-fix' of https://github.com/brittanydonowho/zeek:
Fixed stats.zeek to log all data before zeek terminates rather than return too soon
- Minor whitespace adjutment in merge
* origin/topic/vern/any-typetype-when-fix:
bug fixes for using "when" in functions that have a local of type "any"
Currently, siphash is used for strings up to 36 bytes. hmac-md5 is used
for longer strings.
This switch-over is a remnant of the previous hash-function that was
used, which apparently was slower with longer input strings.
This change serves no purpose anymore. I performed a few performance tests
on strings of varying sizes:
For a 40 byte string with 10 million iterations:
siphash: 0.31 seconds
hmac-md5: 3.8 seconds
For a 1080 byte string with 10 million iterations:
siphash: 4.2 seconds
hmac-md5: 17 seconds
For a 18360 byte string with 10 million iterations:
siphash: 69 seconds
hmac-md5: 240 seconds
Hence, this commit removes the use of hmac-md5.
This change causes reordering of lines in a few logs.
This commit also changes the datastructure for the seed in probabilistic/Hasher
to get rid of a type-punning warning.
If a bloomfilter doesn't have a type, that just means no
bloomfilter_add() has been called yet, so seems undesirable to emit an
error for a lookup against something that's known to be empty.
The "http_header" event now has an "original_name" parameter that allows
access to the original header name (the "name" parameter reamins the
same as before: it's the uppercased header name).
The "mime_header_rec" record type now also includes an "original_name"
field to similarly provide access to original header name in the
following events: "http_all_headers", "mime_one_header", and
"mime_all_headers".
Node-specific topic prefix subscriptions/publications now add a trailing
slash like "zeek/cluster/node/<name>/". Without the trailing slash,
messages attempting to target "proxy-10" may also be sent to "proxy-1"
since subscription matching is prefix-based.
For event/hook handlers that had a previous declaration, any &default
arguments are ineffective. Only &default uses in the initial
prototype's arguments have an effect (that includes if the handler
is actually the site at which the declaration occurs).
When the only existing handler doesn't use the full argument list, still
have to ensure the full frame is allocated because all arguments get set
regardless.
This fixes it to again allow the old behavior of matching a handler
against the canonical prototype as long as all argument types, but not
necessarily names, match.
* origin/topic/timw/776-using-statements:
Remove 'using namespace std' from SerialTypes.h
Remove other using statements from headers
GH-776: Remove using statements added by PR 770
Includes small fixes in files that changed since the merge request was
made.
Also includes a few small indentation fixes.
This unfortunately cuases a ton of flow-down changes because a lot of other
code was depending on that definition existing. This has a fairly large chance
to break builds of external plugins, considering how many internal ones it broke.
Pattern values used as table/set indices are stored in serialized form
using just the pattern text, but re-creating the value from that didn't
fully initialize/compile the regex matcher after (re-)creating it from
that pattern text.
This controls whether ports given by "udp_content_delivery_ports_orig" and
"udp_content_delivery_ports_orig" are in terms of the UDP packet's
destination port or by the Connection's "responder" port (the former is
the unchanged default behavior).
Connection durations were being compared "less than" instead of "greater
than" the desired duration threshold as provided to
set_current_conn_bytes_threshold() or
ConnThreshold::set_duration_threshold()
