Commit graph

13201 commits

Author SHA1 Message Date
Christian Kreibich
8eee5bb3d2 Additional infrastructure for printing types
Also added convenience for instantiating (dummy) configuration records.
2021-12-21 14:52:28 -08:00
Christian Kreibich
2395301168 Bump zeek-client 2021-12-21 14:52:28 -08:00
Christian Kreibich
5cb44c2f69 Support on-demand peering with agents when receiving new cluster configuration
Prior to this, static configuration needed to be in place to configure the
controller/agent layout. The configuration update can now include new instances
that the controller will connect to, assuming they're instances with a listening
agent.
2021-12-21 14:52:28 -08:00
Christian Kreibich
484f79f599 Expand requests support in the controller
Request records for configuration updates now store the full configuration. The
ClusterController::Request module now provies a to_string() function for
rendering requests to a string.
2021-12-21 14:52:28 -08:00
Christian Kreibich
aceb05099a Whitespace tweaks in cluster controller and agent scripts 2021-12-21 14:52:28 -08:00
Christian Kreibich
e9bdaebc70 Add Github action job for cluster tests
This job runs in sequence after the image build one, using its resulting image.
The actual tests live in the external zeek-testing-cluster testsuite, which
the new job clones and runs.

To specify a version of the testsuite to use, testing/external/ has a new
commit-hash.zeek-testing-cluster file that tracks the testsuite's relevant
commit ref
2021-12-21 14:52:28 -08:00
Christian Kreibich
2a5690fd00 Tweak Docker image configure invocation to include zeek-client
We don't yet install zeek-client by default, but need it in the image for
testing the cluster.
2021-12-21 14:52:28 -08:00
Christian Kreibich
eafa77b2b6 Add Docker image artifact storage
This enables subsequent jobs/workflows to access the generated image without use
of an image registry.
2021-12-21 14:52:28 -08:00
Robin Sommer
07045ec254
Fix host header normalization in intel framework.
The way we were splitting off ports from host names could fail for IPv6
addresses.

Closes #1844.
2021-12-21 21:54:47 +01:00
Robin Sommer
a7427e95bf
Switch to recording unmodified HTTP header.
We used to attempt to remove any port specification before recording
HTTP host headers in logs. Doing so would (1) remove potentially useful
information, (2) not match what the documentation seemed to suggest, and
(3) fail for IP6 addresses containing colons.

We now record the original HOST header as is.

Addresses #1844.
2021-12-21 21:54:47 +01:00
Robin Sommer
0507f6005c
Adding test for BitTorrent tracker.
Our test trace is extracted from https://www.cloudshark.org/captures/b9089aac6eee.

There actually seems to be a bug in the existing code: the URI passed to
bt_tracker_request() includes a partial HTTP version. This commits
includes the baseline as the current code produces it, we'll fix that in
a subsequent comment.
2021-12-21 17:48:26 +01:00
Dominik Charousset
da2a5ef455 Fix GCC builds and string output for Broker errors 2021-12-21 11:02:26 +01:00
Christian Kreibich
1e60264548 Update auxil/broker submodule [nomail] [skip ci] 2021-12-20 19:33:46 -08:00
Dominik Charousset
56f30b500a Update to latest Broker without public CAF dep 2021-12-20 08:16:21 +01:00
zeek-bot
e93fcd3c64 Update doc submodule [nomail] [skip ci] 2021-12-15 00:36:00 +00:00
Tim Wojtulewicz
2376fe414c Merge remote-tracking branch 'origin/topic/vern/usage-usage'
* origin/topic/vern/usage-usage:
  fixes for double-delete and reducing '?' operator with constant alternatives
  additional test suite updates for "-u" usage issues
  test suite updates for "xform" and "usage" alternatives, plus test name change
  removed unused script variable
  correct usage info for -u flag; -uu no longer supported
  fix typo in btest filename
2021-12-14 16:54:26 -07:00
Vern Paxson
06ffd97749 fixes for double-delete and reducing '?' operator with constant alternatives 2021-12-14 15:36:03 -08:00
Vern Paxson
77f6a658e6 additional test suite updates for "-u" usage issues 2021-12-14 15:35:24 -08:00
Tim Wojtulewicz
ba0a7c0fed NEWS: Fix some formatting issues 2021-12-14 14:01:27 -07:00
Tim Wojtulewicz
bfaf891a99 NEWS: add updates for new ZAM/script_opt features 2021-12-14 14:01:10 -07:00
Vern Paxson
dff664dc1f test suite updates for "xform" and "usage" alternatives, plus test name change 2021-12-14 12:50:17 -08:00
Vern Paxson
72a59bf828 removed unused script variable 2021-12-14 12:49:27 -08:00
Vern Paxson
5e71e757fe correct usage info for -u flag; -uu no longer supported 2021-12-14 11:37:51 -08:00
Vern Paxson
deb518588a fix typo in btest filename 2021-12-14 11:33:21 -08:00
Tim Wojtulewicz
4920b6affb Merge remote-tracking branch 'origin/topic/vern/cpp-incr'
* origin/topic/vern/cpp-incr:
  tweak to keep clang-format happy
  minor note regarding improving performance of C++-generated code
  factoring of generating C++ initializations, no semantic changes
  restored support for incremental compilation of scripts to C++
  fixes for -O gen-standalone-C++
  new ZEEK_FILE_ONLY and ZEEK_FUNC_ONLY environment variables for debugging script optimization - replaces ZEEK_ONLY
  fix for compiling record constructors to C++
  fixes for compiling vector operations to C++
  fixed for profiling missing some profile elements
  removing vestigial methods
  minor efficiency tweak for ZAM record construction
2021-12-14 11:35:11 -07:00
Tim Wojtulewicz
1b1cf8142d Merge remote-tracking branch 'origin/topic/timw/1860-double-to-int'
* origin/topic/timw/1860-double-to-int:
  GH-1860: Add double_to_int() bif
2021-12-14 09:23:59 -07:00
Tim Wojtulewicz
7de9a7d76f GH-1860: Add double_to_int() bif 2021-12-13 10:46:14 -07:00
Tim Wojtulewicz
f2b74d21f7 Merge remote-tracking branch 'origin/topic/timw/null-checks'
* origin/topic/timw/null-checks:
  Check for sets before attempting to check for same Yield types
  Add early bail-outs to same_type()
  Fix types for Analyzer::register_for_port(s) to be the same
2021-12-13 09:41:48 -07:00
Vern Paxson
a10fafe398 tweak to keep clang-format happy 2021-12-12 12:43:30 -08:00
Vern Paxson
3fc58bdd9f minor note regarding improving performance of C++-generated code 2021-12-12 12:42:16 -08:00
Vern Paxson
86288426fb factoring of generating C++ initializations, no semantic changes 2021-12-12 12:41:16 -08:00
Vern Paxson
2f7137999f restored support for incremental compilation of scripts to C++ 2021-12-12 12:36:45 -08:00
Vern Paxson
3b3cea799b fixes for -O gen-standalone-C++ 2021-12-12 12:36:08 -08:00
Vern Paxson
d5e7db1070 new ZEEK_FILE_ONLY and ZEEK_FUNC_ONLY environment variables for debugging script optimization - replaces ZEEK_ONLY 2021-12-12 12:34:50 -08:00
Vern Paxson
96ed944571 fix for compiling record constructors to C++ 2021-12-12 12:34:23 -08:00
Vern Paxson
52ed9351a9 fixes for compiling vector operations to C++ 2021-12-12 12:33:38 -08:00
Vern Paxson
4ea5785908 fixed for profiling missing some profile elements 2021-12-12 12:33:06 -08:00
Vern Paxson
ce7f886077 removing vestigial methods 2021-12-12 12:32:27 -08:00
Vern Paxson
bbae2cf36c minor efficiency tweak for ZAM record construction 2021-12-12 12:31:28 -08:00
zeek-bot
9b0173172f Update doc submodule [nomail] [skip ci] 2021-12-11 00:39:42 +00:00
Tim Wojtulewicz
429b42ecb1 Update cmake submodule across all other submodules 2021-12-10 15:36:15 -07:00
Tim Wojtulewicz
05e7ddcfd9 Merge remote-tracking branch 'origin/topic/vern/which-opt2'
* origin/topic/vern/which-opt2:
  btests for new --optimize-funcs and --optimize-files
  update btest to no longer use (unsupported) %S formatting, no longer needed
  replace --optimize-only with --optimize-funcs and --optimize-files
2021-12-10 15:35:49 -07:00
Vern Paxson
a0e7feda6f btests for new --optimize-funcs and --optimize-files 2021-12-10 13:02:52 -08:00
Tim Wojtulewicz
29b0d32fef Check for sets before attempting to check for same Yield types 2021-12-10 20:50:03 +00:00
Vern Paxson
267f5461d8 update btest to no longer use (unsupported) %S formatting, no longer needed 2021-12-10 12:46:07 -08:00
Vern Paxson
9069e744f9 replace --optimize-only with --optimize-funcs and --optimize-files 2021-12-10 12:45:27 -08:00
Tim Wojtulewicz
aa91f72b34 Merge remote-tracking branch 'origin/topic/vern/cpp-add-option2'
* origin/topic/vern/cpp-add-option2:
  reintroduction of "-O add-C++" option
2021-12-10 13:13:26 -07:00
Tim Wojtulewicz
56b421db9f Merge remote-tracking branch 'origin/topic/vern/cpp-maint-dec21'
* origin/topic/vern/cpp-maint-dec21:
  fixes for vector operations
  flag globals initialized to opaque values as non-compilable
  skip type signatures for lambdas
  fix for translating filenames beginning with numbers to C++ variable names
  remove unnecessary includes
2021-12-10 13:12:58 -07:00
Tim Wojtulewicz
79326ba62c Merge remote-tracking branch 'origin/topic/vern/cpp-record-constructor'
* origin/topic/vern/cpp-record-constructor:
  update script-to-C++ compilation for new record constructor internals
2021-12-10 13:11:45 -07:00
Tim Wojtulewicz
c105cbad6e Merge remote-tracking branch 'origin/topic/vern/ZAM-maint'
* origin/topic/vern/ZAM-maint:
  updates to ZAM to track recent changes in script semantics
2021-12-10 13:11:20 -07:00