Commit graph

2220 commits

Author SHA1 Message Date
Bernhard Amann
355b85fcd7 most of the stuff we should need for benchmarking. next: search memory leaks, after 1.5million simulated
inputs we are leaking about 1Gb of ram...
2012-03-30 09:08:08 -07:00
Bernhard Amann
ead30e423d change type of heart_beat_interval to interval (makes much more sese) 2012-03-30 08:40:38 -07:00
Bernhard Amann
ef77b1447f Merge remote-tracking branch 'origin/topic/robin/log-threads' into topic/bernhard/log-threads-configureable-heartbeat
Conflicts:
	src/threading/Manager.cc
2012-03-29 09:19:22 -07:00
Bernhard Amann
6a60f484f9 make heart beat interval for threading configureable from scripting layer 2012-03-29 09:03:33 -07:00
Seth Hall
47f58e6340 Cluster support for the metrics framework returns and all tests work again. 2012-03-28 14:00:59 -04:00
Seth Hall
6600e62ea3 Ported scripts using metrics framework and added a new smtp script.
- New script measures a couple of aspects of SMTP traffic.

- Existing metrics scripts had a small amount of work done
  to make them work with changes to metrics framework.
2012-03-28 11:39:27 -04:00
Daniel Thayer
bae6a4178e Merge remote-tracking branch 'origin/master' into topic/icmp6
Conflicts:
	scripts/base/init-bare.bro
	src/Sessions.cc
2012-03-26 17:07:32 -05:00
Jon Siwek
0ceca706f6 Change routing0_data_to_addrs BIF to return vector of addresses.
Because the order of addresses in type 0 routing headers is
interesting/important.
2012-03-26 14:35:01 -05:00
Robin Sommer
30014ac920 Cosmetics in preparation for merge. 2012-03-23 17:00:51 -07:00
Robin Sommer
4321f635ac Removing remaining comments. Looks fine. 2012-03-23 17:00:51 -07:00
Robin Sommer
d7c9471818 Extending queue statistics. 2012-03-23 15:57:25 -07:00
Seth Hall
5904043e85 Modifications to the metrics framework API and new features.
- Metrics:ID enum has been replaced with strings.

- Uniqueness can now be measured with the Metrics::add_unique function.

- Filters can change the index value with the $normalize_func field.
2012-03-22 16:46:37 -04:00
Jon Siwek
c765f43fe3 Refactor script-layer IPv6 ext. header chain (addresses #795)
This replaces the "ip6_hdr_chain" in the "ip6_hdr" record with a vector of
"ip6_ext_hdr" to make it easier to traverse the chain.
2012-03-21 10:34:38 -05:00
Jon Siwek
1c1d657039 Changes to IPv6 ext. header parsing (addresses #795).
In response to feedback from Robin:

  - rename "ip_hdr" to "ip4_hdr"

  - pkt_hdr$ip6 is now of type "ip6_hdr" instead of "ip6_hdr_chain"

  - "ip6_hdr_chain" no longer contains an "ip6_hdr" field, instead
    it's the other way around, "ip6_hdr" contains an "ip6_hdr_chain"

  - other internal refactoring
2012-03-20 15:50:17 -05:00
Jon Siwek
f11fca588e Merge branch 'master' into topic/jsiwek/ipv6-ext-headers 2012-03-19 14:26:59 -05:00
Bernhard Amann
3286d013c9 forgot two files. 2012-03-16 23:45:10 -07:00
Bernhard Amann
57ffe1be77 completely change interface again.
compiles, not really tested.

basic test works 70% of the time, coredumps in the other 30 - but was not easy to debug on a first glance (most interestingly the crash happens in the logging framework - I wonder how that works).
Other tests are not adjusted to the new interface yet.
2012-03-15 18:41:51 -07:00
Bernhard Amann
b4e6971aab Add regular debugging output for interesting operations (stream/filter operations) to input framework (this was way overdue) 2012-03-14 14:45:53 -07:00
Bernhard Amann
c3d2f1d5fc Merge remote-tracking branch 'origin/master' into topic/bernhard/input-threads 2012-03-14 12:28:53 -07:00
Daniel Thayer
cea52fbccb Merge remote-tracking branch 'origin/master' into topic/icmp6 2012-03-14 11:29:29 -05:00
Robin Sommer
79948c7974 Merge remote-tracking branch 'origin/topic/jsiwek/ipv6-ext-headers'
* origin/topic/jsiwek/ipv6-ext-headers:
  Update PacketFilter/Discarder code for IP version independence.
  Add a few comments to IP.h
  Fix some IPv6 header related bugs.
  Add IPv6 fragment reassembly.
  Add handling for IPv6 extension header chains (addresses #531)
2012-03-13 15:25:18 -07:00
Robin Sommer
e83714e178 Merge branch 'master' into topic/jsiwek/ipv6-ext-headers 2012-03-13 15:25:05 -07:00
Jon Siwek
bf3f184a01 Change IPv6 literal constant syntax to require encasing square brackets
This is to avoid ambiguity between compressed hex notation and
module namespacing, both which use "::". E.g.: "aaaa::bbbb" could
be an identifier or an IPv6 address, but "[aaaa::bbbb]" is now
clearly the address.

Also added IPv6 mixed notation to allow an IPv4 dotted-decimal
address to be specified in the lower 32-bits.
2012-03-13 13:47:07 -05:00
Bernhard Amann
92555badd4 cleanup, more sanity tests, a little bit more documentation 2012-03-11 20:43:26 -07:00
Robin Sommer
8eaf40ec18 Reverting accidental commit.
Thanks, Seth!
2012-03-08 20:24:12 -08:00
Robin Sommer
51009b73bc Finetuning communication CPU usage. 2012-03-08 18:13:17 -08:00
Jon Siwek
0b32c980bf Update PacketFilter/Discarder code for IP version independence.
The signatures of script-layer functions 'discarder_check_ip',
'discarder_check_tcp', 'discarder_check_udp', and 'discarder_check_icmp'
were changed to use the more general 'pkt_hdr' type as a parameter
instead of individual header types.
2012-03-08 13:12:04 -06:00
Bernhard Amann
cd78005d09 Merge remote-tracking branch 'origin/master' into topic/bernhard/input-threads 2012-03-07 13:43:48 -08:00
Bernhard Amann
7076c64a5e Merge remote-tracking branch 'origin/topic/robin/log-threads' into topic/bernhard/input-threads
(and move a little bit of functionality from ascii reader to backend)

Conflicts:
	src/threading/Manager.cc
2012-03-07 13:42:49 -08:00
Jon Siwek
a0e07018f4 Merge branch 'master' into topic/jsiwek/ipv6-ext-headers 2012-03-05 09:31:53 -06:00
Jon Siwek
eb9f686bb2 Add handling for IPv6 extension header chains (addresses #531)
- The script-layer 'pkt_hdr' type is extended with a new 'ip6' field
  representing the full IPv6 header chain.

- The 'new_packet' event is now raised for IPv6 packets (addresses #523)

- A new event called 'ipv6_ext_header' is raised for any IPv6 packet
  containing extension headers.

- A new event called 'esp_packet' is raised for any packets using ESP
  ('new_packet' and 'ipv6_ext_header' events provide connection info,
  but that info can't be provided here since the upper-layer payload
  is encrypted).

- The 'unknown_protocol' weird is now raised more reliably when Bro
  sees a transport protocol or IPv6 extension header it can't handle.
  (addresses #522)

Still need to do IPv6 fragment reassembly and needs more testing.
2012-03-02 20:20:57 -06:00
Daniel Thayer
e9728d82ab Merge remote-tracking branch 'origin/master' into topic/icmp6 2012-03-02 10:50:05 -06:00
Robin Sommer
fd1cc49d67 Merge remote-tracking branch 'origin/fastpath'
* origin/fastpath:
  Standardized on the &default function for SSL constants.
2012-03-01 08:31:52 -08:00
Seth Hall
87ad77cc10 Standardized on the &default function for SSL constants. 2012-02-29 22:34:31 -05:00
Seth Hall
b3764dcad5 Cleaned up dead code from the old SSL analyzers. (Reported by Julien Sentier) 2012-02-29 09:43:37 -05:00
Daniel Thayer
54d5d38fd3 Merge remote-tracking branch 'origin/master' into topic/icmp6
Conflicts:
	src/Sessions.cc
2012-02-28 18:39:31 -06:00
Bernhard Amann
417542f283 Merge branch 'topic/bernhard/log-threads' into topic/bernhard/input-threads
Seems to work -- all test pass.
But there are thread-safety issues at the moment, because the constructors of IPAddr and IPPrefix are not thread-safe, but needed by workers.

Conflicts:
	src/logging/Manager.cc
2012-02-27 22:59:08 -08:00
Daniel Thayer
0a659c0907 Merge remote-tracking branch 'origin/master' into topic/icmp6 2012-02-27 13:33:05 -06:00
Jon Siwek
e07470c7f1 Remove connection compressor (addresses #559). 2012-02-27 11:35:25 -06:00
Robin Sommer
4ef8607e60 Merge remote-tracking branch 'origin/topic/dnthayer/ftp-ipv6'
* origin/topic/dnthayer/ftp-ipv6:
  Add test case for FTP over IPv4
  Fix IPv6 URLs
  Add a test for FTP over IPv6
  Update FTP EPSV response processing for IPv6
  Fix parsing of FTP EPRT command and EPSV response

Conflicts:
	src/bro.bif

Closes #778.
2012-02-24 15:00:41 -08:00
Bernhard Amann
7e5f733826 raw input reader for seth, which can simply read a file into string-events given a line separator. 2012-02-22 09:44:45 -08:00
Daniel Thayer
8e32e89ce4 Merge remote-tracking branch 'origin/master' into topic/icmp6
Conflicts:
	src/ICMP.cc
	src/net_util.cc
	src/net_util.h
2012-02-20 17:18:55 -06:00
Daniel Thayer
d61fad4f9e Fix IPv6 URLs 2012-02-17 10:55:17 -06:00
Bernhard Amann
91943c2655 * rework script interface, add autostart stream flag that starts up a stream automatically when first filter has been added ( probably the most common use case )
* change internal reader interface again
* remove some quite embarassing bugs that must have been in the interface for rather long
* add different read methods to script & internal interface (like normal, streaming, etc). Not implemented in ascii reader yet.
2012-02-16 15:03:20 -08:00
Bernhard Amann
84883348ec interface documentation.
to a big part stolen from the logging framework
2012-02-16 11:27:10 -08:00
Seth Hall
430cd9b146 Initial rework of packet filter framework.
- Large rework on packet filter framework to make many things easier.
   - Removed the PacketFilter::all_packets variable because it was confusing.
   - New variable (PacketFilter::enable_auto_protocol_capture_filters) to re-enable the old filtering model of only sniffing ports for analyzed protocols.
   - In progress plugin model for adding filtering mechanisms.
   - New default single item for capture_filters = { ["default"] = PacketFilter::default_capture_filter };
   - Mechanism and helper functions to "shunt" traffic with filters.

- Created the Protocols framework to assist with reworking how base protocol scripts are registered with DPD and other things.
   - Protocols framework creates BPF filters for registered analyzers. (if using PacketFilter framework in that mode).
2012-02-16 11:14:57 -05:00
Daniel Thayer
2f2509fdce Merge remote-tracking branch 'origin/topic/v6-addr' into topic/dnthayer/ftp-ipv6 2012-02-15 16:29:31 -06:00
Daniel Thayer
ef31099fd7 Merge branch 'master' into topic/icmp6
Conflicts:
	policy/bro.init
	policy/icmp.bro
	src/Analyzer.cc
	src/AnalyzerTags.h
	src/DPM.cc
	src/ICMP.h
	src/Sessions.cc
	src/event.bif
	src/net_util.cc
2012-02-15 15:58:53 -06:00
Robin Sommer
7458ebf385 Checkpoint after pass. 2012-02-15 13:07:08 -08:00
Bernhard Amann
88233efb2c It works. Even including all unit tests.
But: there are still a few places where I am sure that there are race conditions & memory leaks & I do not really like the current interface & I have to add a few more messages between the front and backend.

But - it works :)
2012-02-13 22:29:55 -08:00