* origin/topic/jsiwek/gh-857-rebuild-tables-on-record-redef:
Initialize RecordVal default fields when redef'd
GH-857: fix redefining record types used to index tables
Change RecordVals to get resized at time of RecordType redef
* 'intrusive_ptr' of https://github.com/MaxKellermann/zeek: (32 commits)
Scope: store IntrusivePtr in `local`
Scope: pass IntrusivePtr to AddInit()
DNS_Mgr: use class IntrusivePtr
Scope: use class IntrusivePtr
Attr: use class IntrusivePtr
Expr: check_and_promote_expr() returns IntrusivePtr
Frame: use class IntrusivePtr
Val: RecordVal::LookupWithDefault() returns IntrusivePtr
Type: RecordType::FieldDefault() returns IntrusivePtr
Val: TableVal::Delete() returns IntrusivePtr
Type: base_type() returns IntrusivePtr
Type: init_type() returns IntrusivePtr
Type: merge_types() returns IntrusivePtr
Type: use class IntrusivePtr in VectorType
Type: use class IntrusivePtr in EnumType
Type: use class IntrusivePtr in FileType
Type: use class IntrusivePtr in TypeDecl
Type: make TypeDecl `final` and the dtor non-`virtual`
Type: use class IntrusivePtr in TypeType
Type: use class IntrusivePtr in FuncType
...
* The compression capability was incorrectly set to 0x0004 instead of 0x0003
* The padding was 4-byte instead of 8-byte aligned and also the spec.
does not strictly require the padding for the last item in the list.
* Add a default case to handle parsing of unknown context types.
This change tracks all TableVals created at parse-time whose index
depends on a given RecordType. Should that RecordType be redef'd, those
TableVals are immediately rebuilt such that they are valid to
subsequently use in either parse-time initializations or eventually in
any arbitrary run-time expression.
* topic/johanna/gh-744-cache-the-heck-out-of-these-certs:
X509 caching API change: callback gets entry directly
Address feedback
Small default updates & external baselines.
X509 caching: small API changes, tests & test updates
Certificate caching - now working in principle.
First (sadly not completely working) approach at caching X509 certificates
Changed some configuration defaults to potentially more same values.
The callback function is now a hook to allow costomization of the events
that are raised.
Tests now exist. Test baselines are updated.
Added a comment to clarify comparison of EOF packet length to 13.
* origin/topic/vladg/gh-843:
Update baselines
Made additional MySQL fixes.
Add support to MySQL for deprecation of EOF packets.
Whitespace cleanup & fixes
Fix EOF detection in the MySQL protocol analyzer.
* origin/topic/jsiwek/32bit-compat:
Improve formatting of doubles that are close to integers
Improve HTTP version number comparisons
Add a 32-bit task to Cirrus CI config
Replace va_list fmt() overload with vfmt()
Format tables indexed by patterns consistently across 32-bit/64-bit
Format interval values consistently across 32-bit/64-bit platforms
A race condition could cause unstable output: if the thread reading the
file is fast, often you see both "pred" functions execute and then both
"line" events execute with both entries already in the table, but if the
thread reading the file is slow, you see pred, event, pred, event, with
only one entry available in the first event.
The order in which &expire_func's get called isn't well-defined, so
separate the output from either to ensure diffs against the Baseline are
always consistent.
* Generally increase timeouts for tests that have recent transient
failures
* Change any test that relied on `btest-bg-wait -k` since that's never
going to play with with CI systems. Instead, we always need to have
a well-defined termination condition in the test itself (and most
already did, so didn't really need the `-k` flag anyway).
* origin/topic/johanna/table-on-change:
&on_change: Address feedback of Jon.
&on_change: incooperate feedback of Jon.
&on_change wrapup: documentation, tests, whitespacing
Table expiry now raises &on_change handlers
Prevent recursion of &on_change handlers.
Make bro_broker::val_to_data take a const Val* instead of a Val
&on_change: add insertion/change notifications.
&on_change working for removals from tables.
Change signature of &on_change handler
Beginning implementation of &on_change for tables.
Logs that got sent sparsely or burstily would get buffered for long
periods of time since the logic to flush them only does so on the next
log write. In the worst case, a subsequent log write could never happen
and cause a log entry to be indefinitely buffered.
This fix introduces a recurring event/timer to simply flush all pending
logs at frequency of Broker::log_batch_interval.
This change standardizes threading formatter error handling and moves
the remaining error calls to be warnings instead.
This is in line with already existing code - in most cases warnings were
raised, only a few cases raised errors. These cases do not differ
significantly from other cases in which warnings are raised.
This also fixes GH-692, in which misformatted lines prevent future file
parsing.
This commit also moves the FailWarn method that is used by both the
config and the ascii reader up to the ReaderBackend. Furthermore it
makes the Warning method of ReaderBackend respect the warning
suppression that is introduced by the FailWarn method.
The Zeek code base has very inconsistent #includes. Many sources
included a few headers, and those headers included other headers, and
in the end, nearly everything is included everywhere, so missing
#includes were never noticed. Another side effect was a lot of header
bloat which slows down the build.
First step to fix it: in each source file, its own header should be
included first to verify that each header's includes are correct, and
none is missing.
After adding the missing #includes, I replaced lots of #includes
inside headers with class forward declarations. In most headers,
object pointers are never referenced, so declaring the function
prototypes with forward-declared classes is just fine.
This patch speeds up the build by 19%, because each compilation unit
gets smaller. Here are the "time" numbers for a fresh build (with a
warm page cache but without ccache):
Before this patch:
3144.94user 161.63system 3:02.87elapsed 1808%CPU (0avgtext+0avgdata 2168608maxresident)k
760inputs+12008400outputs (1511major+57747204minor)pagefaults 0swaps
After this patch:
2565.17user 141.83system 2:25.46elapsed 1860%CPU (0avgtext+0avgdata 1489076maxresident)k
72576inputs+9130920outputs (1667major+49400430minor)pagefaults 0swaps
During merge I split the test for bro_init/bro_done/bro_script_loaded
event errors into individual tests since the other testing of the zeek
versions of those events seemed fine to otherwise keep.
* origin/topic/robin/631-deprecation-v2:
Update NEWS for naming changes.
Small cleanup and updating submodules.
Remove test for legacy plugin.
Remove legancy symlinks in aux/.
Add warnings when loading scripts ending in ".bro", or using legacy environment variables.
Fix missing rename.
No longer symlink local.zeek to local.bro.
Update notice user agent.
Remove old_comm_usage_is_ok.
Remove bro-config.h.in and bro-path-dev.in.
Change Bro wrapper script to now abort when old executable names are still used.
Remove APIs that were explicitly deprecated to be removed in 3.1.
* origin/topic/jsiwek/supervisor: (44 commits)
Add note that Supervisor script APIs are unstable until 4.0
Move command-line arg parsing functions to Options.{h,cc}
Add btests for supervisor stem/leaf process revival
Move supervisor control events into SupervisorControl namespace
Fix supervisor "destroy" call on nodes not currently alive
Move supervisor source files into supervisor/
Address supervisor code re-factoring feedback from Robin
Convert supervisor internals to rapidjson
Add Supervisor documentation
Add supervisor btests
Improve logging of supervised node errors
Fix supervised node inheritence of command-line script paths
Improve normalize_path() util function
Use a timer to check for death of supervised node's parent
Improve supervisor checks for parent process termination
Improve handling of premature supervisor stem exit
Improve supervisor signal handler safety
Remove unused supervisor config options
Cleanup minor Supervisor TODOs
Improve supervisor debug logging
...
I'd expect the portnum component to remain uninitialized in case the
given URI specifies no port number -- however, decompose_uri() raises an
error instead. Will address this in a subsequent commit.
