Commit graph

5909 commits

Author SHA1 Message Date
Jon Siwek
39f549ed68 Merge remote-tracking branch 'origin/topic/jsiwek/3.2-supervisor'
* origin/topic/jsiwek/3.2-supervisor:
  Integrate Supervisor code review suggestions
  Add Supervisor::{stdout,stderr}_hook
  Add Log::rotation_format_func and Log::default_rotation_dir options
  Allow Zeek function equality comparisons
  Implement leftover log rotation/archival for supervised nodes
  Move some supervisor structures to detail namespace
  Improve default supervisor stdout/stderr handling
  Factor Supervisor initialization data to separate struct from Stem
  Move ParentProcessCheckTimer to detail namespace
2020-07-09 14:13:26 -07:00
Johanna Amann
e1a45d33e0 Merge remote-tracking branch 'origin/master' into topic/johanna/table-changes
* origin/master: (47 commits)
  scan.l: Remove "constant" did_module_restore logic
  Fix FreeBSD CI script to install right SWIG package
  Update submodule(s)
  GH-928: use realpath() instead of inode to de-duplicate scripts
  Update submodule(s)
  GH-1040: Add zero-indexed version of str_split
  Fix WhileStmt to call Stmt(Tag) ctor
  GH-1041: Move compress_path to a bif that uses normalize_path
  Update submodule(s)
  Update submodule(s)
  Update submodule(s)
  Fix --enable-mobile-ipv6 build
  Fix namespace of GetCurrentLocation() to zeek::detail
  Add backtrace() and print_backtrace()
  Rename BroString files to ZeekString
  Update NEWS entry with note about class renames
  Rename BroObj to Obj
  Rename BroString to zeek::String
  Move Func up to zeek namespace, rename BroFunc to ScriptFunc
  Mark global val_mgr as deprecated and fix uses of it to use namespaced version
  ...
2020-07-09 14:07:03 -07:00
Jon Siwek
7669f560d1 Integrate Supervisor code review suggestions 2020-07-09 13:56:11 -07:00
Arne Welzel
93e7f40b70 scan.l: Remove "constant" did_module_restore logic
Doesn't seem like did_module_restore has any effect on the code flow.
2020-07-08 16:20:57 -07:00
Robin Sommer
1fe822a745 Merge remote-tracking branch 'origin/topic/jsiwek/fix-while-stmt-ctor'
* origin/topic/jsiwek/fix-while-stmt-ctor:
  Fix WhileStmt to call Stmt(Tag) ctor
2020-07-08 15:41:50 +00:00
Jon Siwek
10709c627b Add Supervisor::{stdout,stderr}_hook
These allow capturing/handling the stdout/stderr of child processes
via Zeek scripts.
2020-07-07 20:21:32 -07:00
Jon Siwek
a06ef66edc Add Log::rotation_format_func and Log::default_rotation_dir options
These may be redefined to customize log rotation path prefixes,
including use of a directory.  File extensions are still up to
individual log writers to add themselves during the actual rotation.

These new also allow for some simplication to the default
ASCII postprocessor function: it eliminates the need for it doing an
extra/awkward rename() operation that only changes the timestamp format.

This also teaches the supervisor framework to use these new options
to rotate ascii logs into a log-queue/ directory with a specific
file name format (intended for an external archiver process to
monitor separately).
2020-07-07 18:42:37 -07:00
Jon Siwek
6e67a40d24 Allow Zeek function equality comparisons
Equality between two functions acts like a pointer comparison.
2020-07-07 18:39:23 -07:00
Jon Siwek
11949ce37a Implement leftover log rotation/archival for supervised nodes
This helps prevent a node from being killed/crashing in the middle
of writing a log, restarting, and eventually clobbering that log
file that never underwent the rotation/archival process.

The old `archive-log` and `post-terminate` scripts as used by
ZeekControl previously implemented this behavior, but the new logic is
entirely in the ASCII writer.  It uses ".shadow" log files stored
alongside the real log to help detect such scenarios and rotate them
correctly upon the next startup of the Zeek process.
2020-07-07 18:39:23 -07:00
Jon Siwek
a46e24091a Move some supervisor structures to detail namespace
* StemHandle -> detail::SupervisorStemHandle
* SupervisedNode -> detail::SupervisedNode
* Supervisor::Node -> detail::SupervisorNode
2020-07-07 18:39:23 -07:00
Jon Siwek
9087621714 Improve default supervisor stdout/stderr handling
The stdout/stderr of child processes is now redirected over a pipe back
to the supervisor process so that it can prefix the output with
the name of the emitting node.
2020-07-07 18:39:23 -07:00
Jon Siwek
0acd5fea0c Factor Supervisor initialization data to separate struct from Stem 2020-07-07 18:38:38 -07:00
Jon Siwek
d00b3a8daa Move ParentProcessCheckTimer to detail namespace 2020-07-07 18:38:38 -07:00
Jon Siwek
320b14ff09 GH-928: use realpath() instead of inode to de-duplicate scripts
Duplicate script `@load` directives are now detected by comparing
against canonical paths formed by realpath().  This fixes the previous,
unexpected behavior of treating scripts that hardlink to same
inode as duplicates: such links will now be loaded as distinct scripts
since their canonical path differs.
2020-07-07 17:29:52 -07:00
Jon Siwek
7b15b82009 Merge remote-tracking branch 'origin/topic/timw/1040-str-split'
* origin/topic/timw/1040-str-split:
  GH-1040: Add zero-indexed version of str_split
2020-07-06 21:06:51 -07:00
ronwellman
b17ec6bf93 Adjust for zeek namespace. 2020-07-06 22:59:43 -04:00
Tim Wojtulewicz
e6871ed3e9 GH-1040: Add zero-indexed version of str_split 2020-07-06 17:05:40 -07:00
Jon Siwek
76a38caf34 Fix WhileStmt to call Stmt(Tag) ctor
Also removed Stmt() default ctor to help ensure derived classes
initialize the Stmt tag (and other members).
2020-07-06 12:22:23 -07:00
Ron Wellman
e7146c2a6b Implement EDNS Client Subnet Option 2020-07-06 15:09:03 -04:00
Tim Wojtulewicz
560ee0c05e GH-1041: Move compress_path to a bif that uses normalize_path 2020-07-06 11:43:44 -07:00
Robin Sommer
3d3d5e7eb4 Merge remote-tracking branch 'origin/topic/jsiwek/add-backtrace-bif'
(Added a Doxygen comment.)

* origin/topic/jsiwek/add-backtrace-bif:
  Add backtrace() and print_backtrace()
2020-07-06 14:56:28 +00:00
Jon Siwek
61b52b54ba Fix --enable-mobile-ipv6 build 2020-07-03 14:30:13 -07:00
Jon Siwek
3a59948238 Fix namespace of GetCurrentLocation() to zeek::detail 2020-07-03 14:18:02 -07:00
Jon Siwek
a1c19840ce Add backtrace() and print_backtrace() 2020-07-03 14:09:31 -07:00
Jon Siwek
76e67ff239 Merge remote-tracking branch 'origin/topic/timw/266-namespaces-part2'
* origin/topic/timw/266-namespaces-part2:
  Rename BroString files to ZeekString
  Update NEWS entry with note about class renames
  Rename BroObj to Obj
  Rename BroString to zeek::String
  Move Func up to zeek namespace, rename BroFunc to ScriptFunc
  Mark global val_mgr as deprecated and fix uses of it to use namespaced version
  Minor cleanup items from PR review
  Update binpac and bifcl submodules with review changes
  Move Location to zeek::detail and BroObj to zeek
  Move BroString to zeek namespace
  Move Dictionary/PDict, List/PList, and Queue/PQueue to zeek namespace
  Remove typedef that should have been removed in 3.1
  Move Func and associated classes into zeek::detail namespace
  Move Frame and Scope to zeek::detail namespace
  Move all Val classes to the zeek namespaces
  Use type aliases for IntrusivePtr definitions
  Move deprecation macro to zeek-config.h.in to avoid having to over-include util.h
  Move IntrusivePtr and utility methods to the zeek namespace
2020-07-02 19:25:13 -07:00
Tim Wojtulewicz
45d2c96643 Rename BroString files to ZeekString 2020-07-02 17:24:22 -07:00
Tim Wojtulewicz
02cef05f93 Rename BroObj to Obj 2020-07-02 16:15:01 -07:00
Tim Wojtulewicz
736a3f53d4 Rename BroString to zeek::String 2020-07-02 16:15:01 -07:00
Tim Wojtulewicz
a2bc42dd93 Move Func up to zeek namespace, rename BroFunc to ScriptFunc 2020-07-02 16:15:01 -07:00
Tim Wojtulewicz
86fdf0eaa9 Mark global val_mgr as deprecated and fix uses of it to use namespaced version 2020-07-02 16:15:00 -07:00
Jon Siwek
902173a358 GH-786: fix Val::As methods casting incomplete types 2020-07-02 14:55:04 -07:00
Jon Siwek
5cba12444a GH-786: fix Stmt::As methods casting incomplete types 2020-07-02 14:28:28 -07:00
Jon Siwek
c8b6badd54 GH-786: fix Expr::As methods casting incomplete types 2020-07-02 14:23:20 -07:00
Tim Wojtulewicz
3098dd6fbb Minor cleanup items from PR review 2020-07-02 12:55:21 -07:00
Jon Siwek
0ec2ff20c6 Merge remote-tracking branch 'origin/topic/timw/786-typecasting'
* origin/topic/timw/786-typecasting:
  GH-786: Move Type::As methods to cc file so they have type info for casting safely
2020-07-02 11:38:35 -07:00
Johanna Amann
f6251e62a0 BrokerStore<->Zeek tables: allow setting storage location & tests
With this, the basic functionality of &backend seems to be working.

It is not yet integrated with zeekctl, one has to manually specify the
storage location for the sqlite files somewhere when using sqlite.

Usage for memory stores:

global table_to_share: table[string] of count &backend=Broker::MEMORY;

Usage for sqlite stores:

redef Broker::auto_store_db_directory = "[path]";
global table_to_share: table[string] of count &backend=Broker::SQLITE;

In both cases, the cluster should automatically sync to changes done by
any node. When using sqlite, data should also be saved to disk and
re-loaded on startup.
2020-07-01 17:10:43 -07:00
Johanna Amann
a220b02722 BrokerStore<->Zeek tables: &backend works for in-memory stores.
Currently this requires using this with a normal cluster - or sending
messages by yourself.

It, in principle, should also work with SQLITE - but that is a bit
nonsensical without being able to change the storage location.
2020-07-01 16:38:10 -07:00
Jon Siwek
734af6b6d1 Fix a deprecation warning compiling with GCC 2020-07-01 09:58:18 -07:00
Tim Wojtulewicz
ae7a4f7735 GH-786: Move Type::As methods to cc file so they have type info for casting safely 2020-07-01 09:27:55 -07:00
Tim Wojtulewicz
40ecede4ea Move Location to zeek::detail and BroObj to zeek 2020-06-30 21:12:26 -07:00
Tim Wojtulewicz
58c6e10b62 Move BroString to zeek namespace 2020-06-30 21:12:26 -07:00
Tim Wojtulewicz
464efbe66a Move Dictionary/PDict, List/PList, and Queue/PQueue to zeek namespace 2020-06-30 21:12:26 -07:00
Tim Wojtulewicz
4a1e17f4e0 Remove typedef that should have been removed in 3.1 2020-06-30 20:51:58 -07:00
Tim Wojtulewicz
d6f1ea16ac Move Func and associated classes into zeek::detail namespace 2020-06-30 20:51:58 -07:00
Tim Wojtulewicz
937a462e70 Move Frame and Scope to zeek::detail namespace 2020-06-30 20:51:53 -07:00
Tim Wojtulewicz
64332ca22c Move all Val classes to the zeek namespaces 2020-06-30 20:48:09 -07:00
Tim Wojtulewicz
ec9eff0bd5 Use type aliases for IntrusivePtr definitions 2020-06-30 20:37:30 -07:00
Tim Wojtulewicz
f6a251cdac Move deprecation macro to zeek-config.h.in to avoid having to over-include util.h 2020-06-30 20:19:14 -07:00
Tim Wojtulewicz
9364e6a5b7 Move IntrusivePtr and utility methods to the zeek namespace 2020-06-30 20:19:12 -07:00
Jon Siwek
4668378d91 Merge remote-tracking branch 'origin/topic/timw/1032-namespaced-enums'
Merge adjustments:
- Added back in deprecation tag for base_type_no_ref()
- Added back the deprecated plugin::hook_name() function

* origin/topic/timw/1032-namespaced-enums:
  Deprecate plugin::HookType and plugin::component::Type in a different way
  Deprecate init_class and IDScope in another way.
  Deprecate TypeTag and friends in a different way
  Deprecate attr_tag in a different way, rename to AttrTag
2020-06-30 16:55:36 -07:00