http: Prevent request/response de-synchronization and unbounded state growth

When http_reply events are received before http_request events, either
through faking traffic or possible re-ordering, it is possible to trigger
unbounded state growth due to later http_requests never being matched
again with responses.

Prevent this by synchronizing request/response counters when late
requests come in.

Also forcefully flush pending requests when http_replies are never
observed either due to the analyzer having been disabled or because
half-duplex traffic.

Fixes #1705

testing/btest/scripts/base/protocols/http/http-desynched.zeek

@@ -0,0 +1,13 @@
+# @TEST-DOC: 5 HTTP requests, the first one is responded to with 3 HTTP responses.
+#
+# @TEST-EXEC: zeek -b -r $TRACES/http/http-desync-request-response-5.pcap %INPUT
+# @TEST-EXEC: btest-diff http.log
+
+@load base/protocols/http
+
+# mime type is irrelevant to this test, so filter it out
+event zeek_init()
+	{
+	Log::remove_default_filter(HTTP::LOG);
+	Log::add_filter(HTTP::LOG, [$name="less-mime-types", $exclude=set("mime_type")]);
+	}

testing/btest/scripts/base/protocols/http/http-pending-state-growth.zeek

@@ -0,0 +1,15 @@
+# @TEST-DOC: Pcap has a gap for the server side. This previously caused unbounded state growth in c$http_state$pending.
+#
+# @TEST-EXEC: zcat <$TRACES/http/1000-requests-one-dropped-response.pcap.gz | zeek -C -b -r - %INPUT >out
+# @TEST-EXEC: echo "total http.log lines" >>out
+# @TEST-EXEC: grep -v '^#' http.log | wc -l | sed 's/ //g' >>out
+# @TEST-EXEC: btest-diff out
+# @TEST-EXEC: btest-diff weird.log
+
+@load base/protocols/http
+
+event connection_state_remove(c: connection)
+	{
+	if ( c?$http_state )
+		print "http_state pending", |c$http_state$pending|;
+	}