Commit graph

6667 commits

Author SHA1 Message Date
Jon Siwek
7ddd311583 Improve supervisor checks for parent process termination
Comparing parent process ID to 1 to detect loss of parent process was
not necessarily portable, so now it stores parent PID pre-fork and then
monitors for any change.
2020-01-15 14:42:13 -08:00
Jon Siwek
1db7a222a0 Handle invalid Base64 encodings in FTP ADAT analyzer 2020-01-15 12:44:10 -08:00
Jon Siwek
899a987527 Improve handling of premature supervisor stem exit
i.e. if the stem process terminates before the supervisor registers a
SIGCHLD handler.
2020-01-14 20:25:27 -08:00
Jon Siwek
f5b3673890 Improve supervisor signal handler safety
Now should only be making async-signal-safe calls
2020-01-14 18:56:34 -08:00
Jon Siwek
3e1a9ebec3 Remove unused supervisor config options
Since those related to offline pcap reading are not implemented yet.
2020-01-14 17:33:37 -08:00
Tim Wojtulewicz
227d29db80 Use the list of files from clang-tidy when searching for unit tests
The previous method for searching for these files included everything from src/3rdparty, which breaks when rapidjson is included. We don't want to include that directory anyways. We already had a good list of files to scan from the previous clang-tidy and adding any that are missing is an easy task.
2020-01-14 15:44:59 -07:00
Tim Wojtulewicz
23f551876c Optimize json_escape_utf8 a bit by removing repeated calls to string methods 2020-01-14 15:43:25 -07:00
Tim Wojtulewicz
ee0619f999 Expand unit test for json_escape_utf8 to include all of the strings from the ascii-json-utf8 btest 2020-01-14 15:43:25 -07:00
Tim Wojtulewicz
46e7308422 GHI-595: Convert from nlohmann/json to rapidjson for performance reasons 2020-01-14 15:43:25 -07:00
Jon Siwek
4d712d6203 Cleanup minor Supervisor TODOs
e.g. Mainly making default parameter for restart/destroy/status API
calls to operate on all nodes.
2020-01-14 13:41:46 -08:00
Jon Siwek
80b3aef486 Improve supervisor debug logging
Mainly making stem process debug messages sent up to parent supervisor
process and included in its debug.log though option to print to stderr
remains in form of environment variable in case debugging breaking
change to the IPC mechanism itself.
2020-01-14 11:24:46 -08:00
Jon Siwek
0ff99c3df8 Separate supervisor node config and status data structures 2020-01-13 20:09:05 -08:00
Johanna Amann
382c4ddc2f &on_change: add insertion/change notifications.
Also - work with sets in addition to tables.
2020-01-13 19:39:40 -08:00
Johanna Amann
b343cf9bb7 &on_change working for removals from tables.
Insertions and changes are still noops.
2020-01-13 15:28:56 -08:00
Johanna Amann
5b5d36cd83 Change signature of &on_change handler
After thinking about this for quite a while (and starting
implementation) - for now I think it is much easier to not allow the
table &on_change handler to change the result of a change - but to only
be notified that a change happened.

I might re-think this lateron - the reasoning is that this makes
development easier, and that I am not quite sure about potential
side-effects for things that assume that a table change does take
effect.
2020-01-13 13:38:48 -08:00
Johanna Amann
68167d14aa Merge remote-tracking branch 'origin/master' into topic/johanna/table-changes 2020-01-13 10:43:35 -08:00
Jon Siwek
520c6e3ebf Merge branch 'master' into topic/jsiwek/supervisor 2020-01-13 10:27:34 -08:00
Robin Sommer
ae9e79969e Fix method returning a reference to a temporary.
Found by cppcheck, via by Arthur Wong.
2020-01-13 11:24:20 +00:00
Jon Siwek
5191e14eff Add stdout/stderr redirection option to supervised node config 2020-01-10 19:48:31 -08:00
Jon Siwek
263a5f404a Add cpu affinity option to supervised node config 2020-01-10 18:25:42 -08:00
Tim Wojtulewicz
50a0835b41 Convert type-checking macros to actual functions 2020-01-09 11:43:09 -08:00
Johanna Amann
96329e2172 Merge remote-tracking branch 'origin/master' into topic/johanna/table-changes 2020-01-09 12:48:20 -05:00
Johanna Amann
bf31587c2c Beginning implementation of &on_change for tables. 2020-01-09 12:47:16 -05:00
Jon Siwek
a4089bc659 Enable LeakSanitizer for unit tests run via doctest 2020-01-08 21:14:40 -08:00
Jon Siwek
00cd04b0ae Extend Supervisor Node config with list of custom scripts 2020-01-08 14:52:46 -08:00
Jon Siwek
297317b232 Organize command-line options for Supervisor filtering/inheritance
Also have stem process execv() with original command-line arguments so
that they're re-parsed and inherited correctly by supervised-nodes in
the event the stem process needs to be re-created.
2020-01-08 14:05:19 -08:00
Jon Siwek
0e19520e13 Fix Supervisor memory leak 2020-01-07 20:42:43 -08:00
Jon Siwek
6046da9993 Merge branch 'master' into topic/jsiwek/supervisor 2020-01-07 16:57:58 -08:00
Jon Siwek
22431dbecb Use string_view in Supervisor API where possible 2020-01-07 11:59:39 -08:00
Tim Wojtulewicz
0f8f53808e Use bools instead of single-bit bitfields in Ident and TCP protocol analyzers 2020-01-07 12:07:58 -07:00
Tim Wojtulewicz
9d38419e8a Bit of code-modernization cleanup in BroString
- Convert single-bit bit fields into bools
- Use bool for a number of function arguments that were previously ints
- Use delegated constructors to reduce repetition in the other constructors
2020-01-07 11:48:52 -07:00
Tim Wojtulewicz
a463c5763f Use fixed types in NetbiosSSN.h and Timer.h instead of bit fields 2020-01-07 11:48:52 -07:00
Jon Siwek
b114766205 Add Supervisor::is_supervised()
And use to to avoid redef'ing the log rotation postprocessor to
"archive-log" by default since it's unlikely PATH is configured to find
that script.
2020-01-07 10:27:07 -08:00
Jon Siwek
aaa702fb4d Add option to change supervised node's working directory 2020-01-06 18:39:14 -08:00
Tim Wojtulewicz
a20c3a195a Fix build warning in X509 plugin 2020-01-06 15:38:28 -07:00
Jon Siwek
d581aa76aa Merge remote-tracking branch 'origin/topic/dev/patterns-in-sets'
Fixes in merge:
- Memory leak in HashKey ctor
- Minor whitespace/style changes

* origin/topic/dev/patterns-in-sets:
  Enable Patterns as Table index in non singleton cases
  Added support to create a Hashkey for PatternVals using their Pattern Texts
2020-01-06 14:15:30 -08:00
Jon Siwek
99faaf811d Fix snprintf compiler warning in hexdump BIF 2020-01-06 12:29:03 -08:00
Tim Wojtulewicz
65a4e36a92 Merge remote-tracking branch 'origin/topic/jsiwek/asan'
* origin/topic/jsiwek/asan: (28 commits)
  Increase timeout for a btest
  Switch CI scripts to use ASAN/LSAN instead of gperftools
  Remove redundant memory leak btests
  Fix reference counting issues related to lambdas/closures
  Disable LeakSanitizer for btests that have known leaks
  Rewrite the btest for when-statement timeouts
  Fix memory leak when a logging plugin hook prevents a write
  Fix memory leaks in various input framework error-handling cases
  Fix memory leak in Reporter::get_weird_sampling_whitelist() BIF
  Fix reference counting of Log::Filter "config" field
  Fix memory leak in system_env() BIF
  Clean up triggers awaiting global state modification at shutdown
  Fix memory leak in initializing log writers with no local backend
  Fix packet filter memory leaks
  Skip sending thread heartbeat if it alread asked to be finished
  Fix memory leak of sqlite input reader prepared statement
  Prevent duplicate "finish" threading messages
  Fix memory leak when table-based input stream overwrites old entries
  Fix scripting error in a DHCP btest
  Fix memory leaks in Kerberos ticket decryption
  ...
2020-01-06 11:20:47 -07:00
Jon Siwek
a4fab5327a Merge remote-tracking branch 'origin/topic/timw/util-unit-tests'
* origin/topic/timw/util-unit-tests:
  fixup! Add unit tests to util.cc and module_util.cc
  Mark safe_snprintf and safe_vsnprintf as deprecated, remove uses of them
  Add unit tests to util.cc and module_util.cc
2020-01-06 09:44:43 -08:00
Jon Siwek
44d922c4b5 Fix reference counting issues related to lambdas/closures
For example, circular references between a lambda function the frame
it's stored within and/or its closure could cause memory leaks.

This also fixes other various reference-count ownership issues that
could lead to memory errors.

There may still be some potential/undiscovered issues because the "outer
ID" finding logic doesn't look quite right as the AST traversal descends
within nested lambdas and considers their locals as "outer", but
possibly the other logic for locating values in closures or cloning
closures just works around that behavior.
2020-01-02 23:04:22 -08:00
Tim Wojtulewicz
b3e72a342f fixup! Add unit tests to util.cc and module_util.cc 2020-01-02 16:30:18 -07:00
Tim Wojtulewicz
67fcc9b5af Mark safe_snprintf and safe_vsnprintf as deprecated, remove uses of them
safe_snprintf and safe_vsnprintf just exist to ensure that the resulting strings are always null-terminated. The documentation for snprintf/vsnprintf states that the output of those methods are always null-terminated, thus making the safe versions obsolete.
2020-01-02 15:36:39 -07:00
Tim Wojtulewicz
6a52857f8f Add unit tests to util.cc and module_util.cc 2020-01-02 15:36:39 -07:00
Jon Siwek
09578c6176 Fix memory leak when a logging plugin hook prevents a write 2020-01-02 12:05:49 -08:00
Jon Siwek
6f5f7df970 Fix memory leaks in various input framework error-handling cases 2020-01-02 12:05:49 -08:00
Jon Siwek
3b6a2a5f4e Fix memory leak in Reporter::get_weird_sampling_whitelist() BIF 2020-01-02 12:05:49 -08:00
Jon Siwek
bf05add542 Fix reference counting of Log::Filter "config" field
Which can potentially be a minor memory leak if there's a lot of dynamic
adding/removing of logging filters.
2020-01-02 12:05:49 -08:00
Jon Siwek
273eb19ff5 Fix memory leak in system_env() BIF 2020-01-02 12:05:49 -08:00
Jon Siwek
9e494452f1 Clean up triggers awaiting global state modification at shutdown
Otherwise they can be reported as memory leaks since no more global
state modifications will take place to notify the trigger to clean
itself up.
2020-01-02 12:05:49 -08:00
Jon Siwek
5e73949327 Fix memory leak in initializing log writers with no local backend 2020-01-02 12:05:49 -08:00