diff --git a/CHANGES b/CHANGES index 886190e990..04ab2f538a 100644 --- a/CHANGES +++ b/CHANGES @@ -1,3 +1,29 @@ +6.0.0-dev.353 | 2023-04-12 13:23:29 -0700 + + * iosource/Manager: Fix poll_interval updating using not-yet valid IsLive() (Arne Welzel, Corelight) + + Testing io_poll_interval_live tweaks with @dopheide-esnet on a Myricom based + system to reduce CPU usage showed no visible effect. + + It turns out, the pkt_src->IsLive() call used to update poll_interval is only + valid *after* calling ->Register() with the source. The conditional updating + of the poll_interval introduced in 4fa3e4b9b4d78e6f20f42b9dbd85216403be07db + never worked out how it was intended to. + + The fix ensures that + + * we actually use a poll_interval of 10 in the live case + * changing io_poll_interval_live does have an effect + + This is a bit of a major change due to lowering the default poll_interval + by a magnitude, but that seemed to have been the intention always. It's also + tunable via redef, so worst case it can be adapted via configuration. + + As reference, with the default a Pcap::non_fd_timeout of 20usec *and* a + poll_interval of 100, theoretically we'd be trying to ask a non-selectable + packet source 500000 per second for a new packet. This is not a likely packet + rate that a single worker would currently observe or manage to process. + 6.0.0-dev.350 | 2023-04-11 15:41:31 -0700 * Only use -Wno-changes-meaning if it exists (Tim Wojtulewicz, Corelight) diff --git a/NEWS b/NEWS index 64ffd4979f..1c7a1ba7d7 100644 --- a/NEWS +++ b/NEWS @@ -191,6 +191,10 @@ Changed Functionality processing input. Specifically in zeek -r scenarios, that is not the wanted behaviour. +- The IO loop's poll interval is now correctly reduced from 100 to 10 for + live packet sources. This should lower CPU usage for deployments with + non-selectable packet sources. + Removed Functionality --------------------- diff --git a/VERSION b/VERSION index 34edf7a4e0..1d500a708c 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -6.0.0-dev.350 +6.0.0-dev.353 diff --git a/src/iosource/Manager.cc b/src/iosource/Manager.cc index 1cf2d9e599..6b836897b3 100644 --- a/src/iosource/Manager.cc +++ b/src/iosource/Manager.cc @@ -402,12 +402,16 @@ void Manager::Register(PktSrc* src) { pkt_src = src; + Register(src, false); + + // Once we know if the source is live or not, adapt the + // poll_interval accordingly. + // + // Note that src->IsLive() is only valid after calling Register(). if ( src->IsLive() ) poll_interval = BifConst::io_poll_interval_live; else if ( run_state::pseudo_realtime ) poll_interval = 1; - - Register(src, false); } static std::pair split_prefix(std::string path)