Commit graph

9501 commits

Author SHA1 Message Date
Tim Wojtulewicz
f16f0360ff Only allow a single trace file (-r) or interface (-i) option on the command-line 2020-01-31 09:34:54 -07:00
Tim Wojtulewicz
2b2121be60 Remove CQ_TimerMgr in favor of PQ_TimerMgr 2020-01-31 09:34:54 -07:00
Max Kellermann
26da10ca05 util: add a tokenize_string() overload which returns string_views
Additionally, it uses a single "char" as delimiter, which is also
faster.

This patch speeds up Zeek startup by 10%.
2020-01-31 13:46:45 +01:00
Max Kellermann
763afe6f5f util: store std::string_view in "final_components" vector
Don't copy those path segments - instead, use std::string_view to
store references into the existing std::strings.  This saves a good
amount of allocation overhead.
2020-01-31 13:46:45 +01:00
Max Kellermann
37cbd98e34 util: use "auto" in normalize_path() 2020-01-31 13:46:45 +01:00
Max Kellermann
53c4e30024 util: reserve space in normalize_path()
Pessimistic reservations to ensure that it does not need to be
reallocated.
2020-01-31 13:46:45 +01:00
Max Kellermann
5c0c336c6b util: skip "." completely in normalize_path()
Don't copy "." segments to the final_components list only to remove it
afterwards.
2020-01-31 13:46:45 +01:00
Max Kellermann
0589f295fa util: pass std::string_view to normalize_path()
Reduce overhead in some callers.
2020-01-31 13:46:44 +01:00
Max Kellermann
f1566bda14 util: pass std::string_view to tokenize_string()
This saves some overhead because some callers pass a plain C string
here which needed to be copied to a temporary std::string.
2020-01-31 13:46:42 +01:00
Max Kellermann
e068ad8a53 util: don't modify the input string in tokenize_string()
This saves one full copy of the input string and avoids moving memory
around at O(n^2) in the erase() call in each loop iteration.
2020-01-31 13:42:30 +01:00
Arne Welzel
800c0c7132 parse.y: Properly set location info for functions
When defining a function, remember the location where the function header
was and restore it before calling `end_func()`. Inside `end_func()`, a
`BroFunc` object is created using the current global location information.

This came up while experimenting with zeek script profiling and wondering
why the locations set for `BroFunc` were "somewhere" in the middle of
functions instead of spanning them.
2020-01-31 10:47:20 +01:00
Max Kellermann
0b3317b1c2 util: optimize expand_escape() by avoiding sscanf()
sscanf() is notoriously slow, and the default scripts have lots of hex
escapes.  This patch reduces Zeek's startup time by 9%.

Before:

            245.04 msec task-clock:u              #    1.002 CPUs utilized
                 0      context-switches:u        #    0.000 K/sec
                 0      cpu-migrations:u          #    0.000 K/sec
            16,411      page-faults:u             #    0.067 M/sec
       629,238,575      cycles:u                  #    2.568 GHz
     1,237,236,556      instructions:u            #    1.97  insn per cycle
       262,223,957      branches:u                # 1070.142 M/sec
         3,351,083      branch-misses:u           #    1.28% of all branches

After:

            220.99 msec task-clock:u              #    1.002 CPUs utilized
                 0      context-switches:u        #    0.000 K/sec
                 0      cpu-migrations:u          #    0.000 K/sec
            16,419      page-faults:u             #    0.074 M/sec
       544,603,653      cycles:u                  #    2.464 GHz
     1,065,862,324      instructions:u            #    1.96  insn per cycle
       229,207,957      branches:u                # 1037.181 M/sec
         3,045,270      branch-misses:u           #    1.33% of all branches
2020-01-31 10:32:37 +01:00
Jon Siwek
70b45d1aba Merge remote-tracking branch 'origin/topic/robin/631-deprecation-v2'
During merge I split the test for bro_init/bro_done/bro_script_loaded
event errors into individual tests since the other testing of the zeek
versions of those events seemed fine to otherwise keep.

* origin/topic/robin/631-deprecation-v2:
  Update NEWS for naming changes.
  Small cleanup and updating submodules.
  Remove test for legacy plugin.
  Remove legancy symlinks in aux/.
  Add warnings when loading scripts ending in ".bro", or using legacy environment variables.
  Fix missing rename.
  No longer symlink local.zeek to local.bro.
  Update notice user agent.
  Remove old_comm_usage_is_ok.
  Remove bro-config.h.in and bro-path-dev.in.
  Change Bro wrapper script to now abort when old executable names are still used.
  Remove APIs that were explicitly deprecated to be removed in 3.1.
2020-01-30 19:19:56 -08:00
Max Kellermann
662c3aab58 Desc: move realloc() call out of the loop 2020-01-30 19:53:23 +01:00
Max Kellermann
61c3be8e16 SerializationFormat: move realloc() call out of the loop
Reallocate and copy the data only once.
2020-01-30 19:53:23 +01:00
Max Kellermann
ef15467757 PacketDumper: remove unused types 2020-01-30 19:53:23 +01:00
Tim Wojtulewicz
c5b2659b6b Merge remote-tracking branch 'origin/topic/jsiwek/supervisor-coverity-fixes'
* origin/topic/jsiwek/supervisor-coverity-fixes:
  Fix leaked FDs in redirecting supervised node stdout/stderr
  Add checks for failed fcntl calls
  Set Pipe file descriptor flags correctly
2020-01-30 08:44:59 -07:00
Jon Siwek
948cc32844 Fix leaked FDs in redirecting supervised node stdout/stderr 2020-01-29 16:05:39 -08:00
Jon Siwek
fd2c6c56a5 Add checks for failed fcntl calls 2020-01-29 16:04:46 -08:00
Jon Siwek
aac7f6e8f2 Set Pipe file descriptor flags correctly 2020-01-29 16:03:12 -08:00
Jon Siwek
f3e5728bcb Merge branch 'leaks' of https://github.com/MaxKellermann/zeek
* 'leaks' of https://github.com/MaxKellermann/zeek:
  Scope: fix memory leak by removing duplicate copy_string() call
  util, nb_dns: fix off-by-one bugs in strncpy() calls
  Type, util: add `constexpr` to static variables
  Net: remove unused variable
2020-01-29 11:50:09 -08:00
Max Kellermann
a458827292 Scope: fix memory leak by removing duplicate copy_string() call
The `ID` constructor also calls copy_string().
2020-01-29 20:22:16 +01:00
Max Kellermann
32bb019e3a util, nb_dns: fix off-by-one bugs in strncpy() calls
Fortunately, these bugs had no effect because the following lines
overwrote the last character with a null byte.
2020-01-29 20:22:16 +01:00
Max Kellermann
aacf84e552 Type, util: add constexpr to static variables
This allows the compiler to move them to section `.rodata`.
2020-01-29 20:22:16 +01:00
Max Kellermann
2694b4e2c8 Net: remove unused variable 2020-01-29 20:22:16 +01:00
Johanna Amann
ad18014bed Merge remote-tracking branch 'origin/topic/jsiwek/ssl-empty-files'
* origin/topic/jsiwek/ssl-empty-files:
  Skip file analysis for zero-length SSL/TLS data
2020-01-29 11:16:35 -08:00
Robin Sommer
6bcd583836 Merge remote-tracking branch 'origin/topic/jsiwek/supervisor'
* origin/topic/jsiwek/supervisor: (44 commits)
  Add note that Supervisor script APIs are unstable until 4.0
  Move command-line arg parsing functions to Options.{h,cc}
  Add btests for supervisor stem/leaf process revival
  Move supervisor control events into SupervisorControl namespace
  Fix supervisor "destroy" call on nodes not currently alive
  Move supervisor source files into supervisor/
  Address supervisor code re-factoring feedback from Robin
  Convert supervisor internals to rapidjson
  Add Supervisor documentation
  Add supervisor btests
  Improve logging of supervised node errors
  Fix supervised node inheritence of command-line script paths
  Improve normalize_path() util function
  Use a timer to check for death of supervised node's parent
  Improve supervisor checks for parent process termination
  Improve handling of premature supervisor stem exit
  Improve supervisor signal handler safety
  Remove unused supervisor config options
  Cleanup minor Supervisor TODOs
  Improve supervisor debug logging
  ...
2020-01-29 13:11:04 +00:00
Robin Sommer
42a7629695 Update NEWS for naming changes. 2020-01-29 12:38:29 +00:00
Robin Sommer
e1458360e3 Small cleanup and updating submodules. 2020-01-29 12:38:23 +00:00
Robin Sommer
4eeabd5210 Remove test for legacy plugin.
We no longer support creating new plugins using the old naming
scheme.
2020-01-29 12:08:10 +00:00
Robin Sommer
e552a07249 Remove legancy symlinks in aux/.
That's aux/bro-aux and aux/broctl.
2020-01-29 12:08:10 +00:00
Robin Sommer
649301b667 Add warnings when loading scripts ending in ".bro", or using legacy environment variables. 2020-01-29 12:08:10 +00:00
Robin Sommer
bbc308cb02 Fix missing rename. 2020-01-29 12:08:10 +00:00
Robin Sommer
123cbba6be No longer symlink local.zeek to local.bro. 2020-01-29 12:08:10 +00:00
Robin Sommer
ad8e659943 Update notice user agent. 2020-01-29 12:08:10 +00:00
Robin Sommer
a941e3d942 Remove old_comm_usage_is_ok.
That was just a left-over without any functionality.
2020-01-29 12:08:10 +00:00
Robin Sommer
0063a81410 Remove bro-config.h.in and bro-path-dev.in. 2020-01-29 12:08:10 +00:00
Robin Sommer
0516f3f439 Change Bro wrapper script to now abort when old executable names are still used. 2020-01-29 12:08:10 +00:00
Robin Sommer
d0b206fa36 Remove APIs that were explicitly deprecated to be removed in 3.1.
Special handling for bro_{init,done,script_loaded} events: if still
used, they cause Zeek to abort at startup.
2020-01-29 12:08:09 +00:00
Jon Siwek
84b8ee3b06 Merge remote-tracking branch 'origin/topic/jsiwek/cirrus-catalina'
* origin/topic/jsiwek/cirrus-catalina:
  Update Cirrus CI config to use macOS Catalina
2020-01-28 17:56:22 -08:00
Jon Siwek
b9ed77420c Update Cirrus CI config to use macOS Catalina 2020-01-28 16:29:18 -08:00
Jon Siwek
83874fa5fa Merge branch 'getrandom' of https://github.com/MaxKellermann/zeek
- Removed the superfluous check for C++17 in the merge since that's
  a requirement enforced at the CMake-level.

* 'getrandom' of https://github.com/MaxKellermann/zeek:
  util: use getrandom() on Linux if available
2020-01-28 12:45:15 -08:00
Johanna Amann
f126296446 Update submodule
[nomail]
2020-01-28 12:07:03 -08:00
Max Kellermann
cb4258434c util: use getrandom() on Linux if available
Unlike /dev/urandom, getrandom() doesn't need a file descriptor and
works when there is no /dev.  It requires Linux 3.17 and glibc 2.25,
but there is a fallback to the old code.

For simplicity, this patch uses __has_include() to detect the
availability of this API, but maybe we should move that to cmake.

(It might be useful to refactor the whole random gathering code to a
separate function.)
2020-01-28 11:45:25 +01:00
Jon Siwek
fdfd729206 Add note that Supervisor script APIs are unstable until 4.0 2020-01-27 13:51:01 -08:00
Jon Siwek
53363a9bd3 Move command-line arg parsing functions to Options.{h,cc} 2020-01-27 13:50:44 -08:00
Jon Siwek
5fb01caee6 Add btests for supervisor stem/leaf process revival 2020-01-27 10:58:40 -08:00
Jon Siwek
0412cb3996 Merge branch 'topic/frerich/gh-750-uri-with-empty-portnum' of https://github.com/frerich/zeek
* 'topic/frerich/gh-750-uri-with-empty-portnum' of https://github.com/frerich/zeek:
  Document recent fix for decompose_uri in release notes
  Fixed decompose_uri choking on URIs with empty port component
  Adding test for decompose_uri parsing URI with empty port
2020-01-24 15:16:14 -08:00
Frerich Raabe
4955356a52 Document recent fix for decompose_uri in release notes 2020-01-24 22:24:05 +01:00
Frerich Raabe
bb879fc2aa Fixed decompose_uri choking on URIs with empty port component
A call such as

  decompose_uri("git://git.kernel.org:/pub/scm/linux/");

would raise an error along the lines of

  error in /usr/local/zeek-3.0.0/share/zeek/base/utils/urls.zeek, line 122: bad conversion to count (to_count(parts[1]) and )

This was because an empty string got passsed to the to_count()
function.

Let's improve the behaviour and rather consider the portnum component
of the URI to be uninitialized.
2020-01-24 22:24:00 +01:00