Commit graph

9501 commits

Author SHA1 Message Date
Frerich Raabe
440468a0dc Adding test for decompose_uri parsing URI with empty port
I'd expect the portnum component to remain uninitialized in case the
given URI specifies no port number -- however, decompose_uri() raises an
error instead. Will address this in a subsequent commit.
2020-01-24 22:23:50 +01:00
Jon Siwek
cc8d28089b Updating submodule(s).
[nomail]
2020-01-24 13:18:33 -08:00
Jon Siwek
f45c2cf06b Skip check for outdated docs in Cirrus CI for PRs 2020-01-23 18:10:17 -08:00
Johanna Amann
97bea7c4f1 Merge remote-tracking branch 'origin/topic/jsiwek/fix-create-stream-priorities'
* origin/topic/jsiwek/fix-create-stream-priorities:
  Use consistent zeek_init priority for Log::create_stream calls
2020-01-23 15:31:06 -08:00
Robin Sommer
fc1c95725b Update Broker submodule.
Includes a test baseline update for change in Broker error message.
2020-01-23 13:51:29 +00:00
Robin Sommer
01b7db5b46 Merge remote-tracking branch 'origin/topic/jsiwek/smb-transaction-strings'
* origin/topic/jsiwek/smb-transaction-strings:
  Improve creation of SMB transaction data strings
2020-01-23 13:19:11 +00:00
Robin Sommer
11850088ee Merge remote-tracking branch 'origin/topic/jsiwek/ftp-word'
* origin/topic/jsiwek/ftp-word:
  Improve FTP word/whitespace handling
2020-01-23 13:14:46 +00:00
Jon Siwek
fce4bb3f50 Improve FTP word/whitespace handling 2020-01-22 19:50:14 -08:00
Jon Siwek
f939bcad7e Skip file analysis for zero-length SSL/TLS data 2020-01-22 16:49:32 -08:00
Jon Siwek
c75519ca88 Improve creation of SMB transaction data strings 2020-01-22 15:41:50 -08:00
Jon Siwek
7a748526c0 Use consistent zeek_init priority for Log::create_stream calls
Typically in base scripts, Log::create_stream() is called in zeek_init()
handler with &priority=5 such that it will have already been created
in the default zeek_init() &priority=0.
2020-01-22 13:58:20 -08:00
Jon Siwek
bbdf5f8938 Move supervisor control events into SupervisorControl namespace 2020-01-22 13:28:20 -08:00
Jon Siwek
68b513a364 Fix supervisor "destroy" call on nodes not currently alive
This would mistakenly have the Stem process kill itself due to giving
PID 0 as argument to kill() where it really was being used to mean "that
node does not currently have any live process associated with it" and so
can just be removed without trying to kill/reap.
2020-01-22 13:17:38 -08:00
Jon Siwek
59e075acab Move supervisor source files into supervisor/ 2020-01-22 11:23:10 -08:00
Jon Siwek
718879735e Address supervisor code re-factoring feedback from Robin 2020-01-21 22:26:17 -08:00
Jon Siwek
172456fac0 Convert supervisor internals to rapidjson 2020-01-21 13:19:05 -08:00
Jon Siwek
9c0d252c2b Merge branch 'master' into topic/jsiwek/supervisor 2020-01-21 12:17:56 -08:00
Robin Sommer
270702cacb Updating CHANGES and VERSION. 2020-01-18 14:32:30 +00:00
Robin Sommer
8170baabef Merge remote-tracking branch 'origin/topic/timw/595-rapidjson'
Tweaks:
    - Small change to the logic for removing quotes around strings.
    - Updated NEWS & COPYING.3rdparty
    - Use of intrusive_ptr for stack-allocated StringVals
    - Little bit of refactoring (I would love to merge the two BuildJSON() functions, too, but that's a larger task)

* origin/topic/timw/595-rapidjson:
  Use the list of files from clang-tidy when searching for unit tests
  Optimize json_escape_utf8 a bit by removing repeated calls to string methods
  Expand unit test for json_escape_utf8 to include all of the strings from the ascii-json-utf8 btest
  GHI-595: Convert from nlohmann/json to rapidjson for performance reasons
  Convert type-checking macros to actual functions
2020-01-18 10:49:15 +00:00
Jon Siwek
8247c42368 Add Supervisor documentation
Minor additions/changes to improve API I noticed along the way
2020-01-17 18:36:32 -08:00
Robin Sommer
c8c6621a0e Merge remote-tracking branch 'origin/topic/timw/bit-fields'
* origin/topic/timw/bit-fields:
  Use bools instead of single-bit bitfields in Ident and TCP protocol analyzers
  Bit of code-modernization cleanup in BroString
  Use fixed types in NetbiosSSN.h and Timer.h instead of bit fields
2020-01-17 11:55:00 +00:00
Robin Sommer
eafd818505 Updating submodule(s).
[nomail]
2020-01-17 11:43:00 +00:00
Jon Siwek
1972190b89 Add supervisor btests 2020-01-16 19:21:53 -08:00
Jon Siwek
21c75b46eb Improve logging of supervised node errors
Now getting sent through standard Reporter framework in the Supervisor
process.
2020-01-16 14:23:08 -08:00
Jon Siwek
8a145ee1a2 Fix supervised node inheritence of command-line script paths
They're now converting to absolute paths in the argument parsing phase
such that if a supervised node switches working directory, it can still
load the referenced script.
2020-01-16 13:11:04 -08:00
Jon Siwek
38cd56a3db Improve normalize_path() util function
It didn't always properly handle ".." when the preceding path component
was also the first component.
2020-01-16 13:08:01 -08:00
Robin Sommer
0d29a80519 Merge remote-tracking branch 'origin/topic/jsiwek/ftp-adat-fix'
* origin/topic/jsiwek/ftp-adat-fix:
  Handle invalid Base64 encodings in FTP ADAT analyzer
2020-01-16 13:27:03 +00:00
Jon Siwek
dbca14e1fc Use a timer to check for death of supervised node's parent 2020-01-15 15:27:53 -08:00
Jon Siwek
7ddd311583 Improve supervisor checks for parent process termination
Comparing parent process ID to 1 to detect loss of parent process was
not necessarily portable, so now it stores parent PID pre-fork and then
monitors for any change.
2020-01-15 14:42:13 -08:00
Jon Siwek
1db7a222a0 Handle invalid Base64 encodings in FTP ADAT analyzer 2020-01-15 12:44:10 -08:00
Jon Siwek
899a987527 Improve handling of premature supervisor stem exit
i.e. if the stem process terminates before the supervisor registers a
SIGCHLD handler.
2020-01-14 20:25:27 -08:00
Jon Siwek
f5b3673890 Improve supervisor signal handler safety
Now should only be making async-signal-safe calls
2020-01-14 18:56:34 -08:00
Jon Siwek
3e1a9ebec3 Remove unused supervisor config options
Since those related to offline pcap reading are not implemented yet.
2020-01-14 17:33:37 -08:00
Tim Wojtulewicz
227d29db80 Use the list of files from clang-tidy when searching for unit tests
The previous method for searching for these files included everything from src/3rdparty, which breaks when rapidjson is included. We don't want to include that directory anyways. We already had a good list of files to scan from the previous clang-tidy and adding any that are missing is an easy task.
2020-01-14 15:44:59 -07:00
Tim Wojtulewicz
23f551876c Optimize json_escape_utf8 a bit by removing repeated calls to string methods 2020-01-14 15:43:25 -07:00
Tim Wojtulewicz
ee0619f999 Expand unit test for json_escape_utf8 to include all of the strings from the ascii-json-utf8 btest 2020-01-14 15:43:25 -07:00
Tim Wojtulewicz
46e7308422 GHI-595: Convert from nlohmann/json to rapidjson for performance reasons 2020-01-14 15:43:25 -07:00
Jon Siwek
4d712d6203 Cleanup minor Supervisor TODOs
e.g. Mainly making default parameter for restart/destroy/status API
calls to operate on all nodes.
2020-01-14 13:41:46 -08:00
Jon Siwek
80b3aef486 Improve supervisor debug logging
Mainly making stem process debug messages sent up to parent supervisor
process and included in its debug.log though option to print to stderr
remains in form of environment variable in case debugging breaking
change to the IPC mechanism itself.
2020-01-14 11:24:46 -08:00
Jon Siwek
f59e3da4f7 Merge remote-tracking branch 'origin/sethhall-fix-ssl-files-warning'
* origin/sethhall-fix-ssl-files-warning:
  Fix warning when reading files from non-network sources
2020-01-14 09:46:19 -08:00
Seth Hall
d9ed76c90a
Fix warning when reading files from non-network sources
If files are being read from non-network sources, there was a warning in the SSL base scripts about missing the f$conns field.
2020-01-14 10:53:02 -05:00
Jon Siwek
0ff99c3df8 Separate supervisor node config and status data structures 2020-01-13 20:09:05 -08:00
Jon Siwek
520c6e3ebf Merge branch 'master' into topic/jsiwek/supervisor 2020-01-13 10:27:34 -08:00
Jon Siwek
9c353f56f1 Updating submodule(s).
[nomail]
2020-01-13 10:08:21 -08:00
Jon Siwek
6cb73e505b Updating submodule(s).
[nomail]
2020-01-13 09:32:15 -08:00
Robin Sommer
bcf9dbac94 Update submodules. 2020-01-13 12:11:08 +00:00
Robin Sommer
ae9e79969e Fix method returning a reference to a temporary.
Found by cppcheck, via by Arthur Wong.
2020-01-13 11:24:20 +00:00
Jon Siwek
5191e14eff Add stdout/stderr redirection option to supervised node config 2020-01-10 19:48:31 -08:00
Jon Siwek
263a5f404a Add cpu affinity option to supervised node config 2020-01-10 18:25:42 -08:00
Jon Siwek
6218f99b96 Update timeouts for a btest 2020-01-09 19:05:23 -08:00