Commit graph

2993 commits

Author SHA1 Message Date
Bernhard Amann
81edec8b2e Fix bug, where in dns.log rcode always was set to 0/NOERROR when
no reply package was seen.

In the fixed version rcode is only set when a reply packet was seen.

Updates for the baseline have been commited separately in the
topic/bernhard/dns-fix branch.
2012-07-17 14:16:15 -07:00
Seth Hall
485e473561 Merge remote-tracking branch 'origin/master' into topic/seth/elasticsearch 2012-07-17 13:57:59 -04:00
Seth Hall
ce05600a71 Mozilla's current certificate bundle. 2012-07-13 22:24:34 -04:00
Robin Sommer
86f6f36f70 Merge remote-tracking branch 'vlad/info_record_fixes'
* vlad/info_record_fixes:
  Fix some Info:Record field documentation.

Closes #851. Many thanks!
2012-07-13 16:20:49 -07:00
Vlad Grigorescu
f43576cff3 Fix some Info:Record field documentation. 2012-07-13 14:04:24 -04:00
Daniel Thayer
56b4209597 Merge remote-tracking branch 'origin/master' into topic/dnthayer/alarms-mail 2012-07-12 12:20:16 -05:00
Seth Hall
5607e86ad3 Reporter warnings and error now print to stderr by default.
- Changed the geoip warnings to Info.
2012-07-12 12:55:34 -04:00
Seth Hall
a44612788e Some small fixes to further reduce SOCKS false positive logs. 2012-07-11 16:53:46 -04:00
Seth Hall
84e91b8b8d Bringing elasticsearch branch up to date with master. 2012-07-09 16:38:05 -04:00
Seth Hall
601d1cf37e Merge remote-tracking branch 'origin/master' into topic/seth/elasticsearch
Conflicts:
	aux/binpac
	aux/bro-aux
	aux/broccoli
	aux/broctl
	scripts/base/frameworks/logging/__load__.bro
	src/logging.bif
2012-07-06 12:01:16 -04:00
Robin Sommer
06d2fd52bd Merge remote-tracking branch 'origin/topic/bernhard/reader-info'
* origin/topic/bernhard/reader-info:
  fix small bug - now configuration actually is passed.
  add mode to readerinfo - no need to have it separately everywhere anymore.
  introduce reader-info struct analogous to writer-info.
  Introduce support for a table of key/value pairs with further configuration options, with the same userinterface as in the logging interface.
  make writer-info work when debugging is enabled

Conflicts:
	testing/btest/Baseline/scripts.base.frameworks.input.event/out
	testing/btest/Baseline/scripts.base.frameworks.input.executeraw/out
	testing/btest/Baseline/scripts.base.frameworks.input.raw/out
	testing/btest/Baseline/scripts.base.frameworks.input.rereadraw/out
	testing/btest/Baseline/scripts.base.frameworks.input.tableevent/out

Closes #841.
2012-07-02 15:33:08 -07:00
Robin Sommer
90763bb2f2 Merge branch 'robin/topic/writer-info'
* robin/topic/writer-info:
  Extending the log writer DoInit() API.
  Reworking log writer API to make it easier to pass additional information to a writer's initialization method.

Conflicts:
	src/logging/WriterBackend.cc
	src/logging/WriterBackend.h
	src/logging/WriterFrontend.cc
2012-07-02 15:20:37 -07:00
Bernhard Amann
b8ad4567fb Merge branch 'topic/bernhard/reader-info' into topic/bernhard/sqlite
Now uses optional dbname configuration option

Conflicts:
	scripts/base/frameworks/logging/__load__.bro
	src/logging.bif
2012-07-02 10:13:54 -07:00
Bernhard Amann
f820ee9f5c Introduce support for a table of key/value pairs with further configuration options,
with the same userinterface as in the logging interface.

Not really tested, but tests still work.
2012-06-28 16:16:48 -07:00
Jon Siwek
9ae9b2aa4d Extract ICMPv6 NDP options and include in ICMP events (addresses #833).
This adds a new parameter of type "icmp6_nd_options" to the ICMPv6
neighbor discovery events (icmp_redirect, icmp_router_solicitation,
icmp_router_advertisement, icmp_neighbor_solicitation,
icmp_neighbor_advertisement) which includes data extracted from
all neighbor discovery options (RFC 4861) that are present in the
ICMPv6 message.
2012-06-26 17:10:00 -05:00
Robin Sommer
19eea409c3 Extending the log writer DoInit() API.
We now pass in a Info struct that contains:

    - the path name (as before)
    - the rotation interval
    - the log_rotate_base_time in seconds
    - a table of key/value pairs with further configuration options.

To fill the table, log filters have a new field "config: table[string]
of strings". This gives a way to pass arbitrary values from
script-land to writers. Interpretation is left up to the writer.

Also splits calc_next_rotate() into two functions, one of which is
thread-safe and can be used with the log_rotate_base_time value from
DoInit().

Includes also updates to the None writer:

    - It gets its own script writers/none.bro.

    - New bool option LogNone::debug to enable debug output. It then
      prints out all the values passed to DoInit(). That's used by a
      btest test to ensure the new DoInit() values are right.

    - Fixed a bug that prevented Bro from terminating..

(scripts.base.frameworks.logging.rotate-custom currently fails.
Haven't yet investigated why.)
2012-06-21 17:42:33 -07:00
Robin Sommer
fb8e9fc5f0 Merge remote-tracking branch 'origin/topic/seth/tunnels-merge'
* origin/topic/seth/tunnels-merge:
  Add another SOCKS command.

Conflicts:
	scripts/base/protocols/socks/consts.bro
2012-06-21 10:12:43 -07:00
Seth Hall
3eb16e5738 Add another SOCKS command. 2012-06-21 02:07:30 -04:00
Robin Sommer
54670817fa Merge remote-tracking branch 'origin/topic/seth/tunnels-merge'
* origin/topic/seth/tunnels-merge:
  Fixed some problems with the SOCKS analyzer and tests.

Conflicts:
	scripts/base/protocols/socks/main.bro
	src/socks-analyzer.pac
2012-06-20 21:05:04 -07:00
Seth Hall
6b8b4dab71 Fixed some problems with the SOCKS analyzer and tests. 2012-06-20 22:57:46 -04:00
Robin Sommer
6cda00c75e Merge remote-tracking branch 'origin/topic/seth/tunnels-merge'
* origin/topic/seth/tunnels-merge:
  SOCKS DPD fixes.
  Fix a bug in the SOCKS analyzer.
  SOCKS and tunnel test updates.
  Updates for the SOCKS analyzer.
  Very small updates to the tunnels framework.
2012-06-20 14:33:18 -07:00
Seth Hall
f59736cb17 SOCKS DPD fixes.
- Restricted the SOCKS 5 DPD signatures further.

- Added protocol violations.
2012-06-20 15:12:52 -04:00
Seth Hall
896f252a31 Updates for the SOCKS analyzer.
- Now supports SOCKSv5 in the analyzer and the DPD sigs.

- Reworked the core events.

- Tests.

- A SOCKS log!
2012-06-20 13:58:25 -04:00
Seth Hall
c30c0d5ff2 Very small updates to the tunnels framework.
- Make the uid field optional since it's conceptually incorrect
  for proxies being treated as tunnels to have it.

- Reordered two fields in the log.

- Reduced the default tunnel expiration interface to something more
  reasonable (1 hour).
2012-06-20 13:56:42 -04:00
Robin Sommer
b096168318 Merge branch 'topic/robin/tunnels-merge'
* topic/robin/tunnels-merge: (51 commits)
  Updating baselines and NEWS.
  Remove &synchronized from Tunnel::active table.
  Refactor of interal tunnel analysis code.
  Add state management of NetSessions's IP tunnel map.
  Add "encap_hdr_size" option back in.
  Script-layer tunnel interface cleanup.
  Fix performance problem checking packet encapsulation. (addresses #830)
  Adding a SOCKS test case.
  Updating DataSeries baselines.
  Moving my todos over to the tracker ticket.
  Extend weird names that occur in core packet processing during decapsulation.
  Add Teredo analysis option to reduce false positive decapsulation.
  Just some cleanup/documentation of new tunnel-handling code.
  Memory leak fixes
  Add a config.h definition for IPPROTO_IPV4.
  Add AYIYA tunnel decapsulation unit test.
  Add Teredo-specific events.
  Refactor some of the NetSessions routines that recurse on IP packets.
  Add independent options to toggle the different decapsulation methods
  Add more sanity checks before recursing on encapsulated IP packets.
  ...

Conflicts:
	src/event.bif
2012-06-19 13:21:31 -07:00
Jon Siwek
d6286d953d Remove &synchronized from Tunnel::active table. 2012-06-18 17:04:46 -05:00
Jon Siwek
e04d629733 Add state management of NetSessions's IP tunnel map.
Entries are checked for inactivity at an interval controlled by
"Tunnel::ip_tunnel_timeout" and discarded if needed.
2012-06-18 15:44:34 -05:00
Jon Siwek
2ba3f5420b Add "encap_hdr_size" option back in.
The "tunnel_port" and "parse_udp_tunnels" options are still gone
as those did not work entirely (e.g. IPv6 support and misnaming
of tunnel_port/udp_tunnel_port).
2012-06-18 12:48:03 -05:00
Jon Siwek
f3b3e73eba Script-layer tunnel interface cleanup.
- Clarify "tunnel_changed" event documentation.

- Make expiration of "Tunnel::active" elements configuration via
  "Tunnel::expiration_interval".

- Remove redundant registration of a connection's tunnels in
  tunnel/main.bro's "tunnel_changed" handler.

- Rename "parents" field of "Conn::Info" to "tunnel_parents"
  to give more context.
2012-06-18 12:29:49 -05:00
Seth Hall
cd8169dda3 Bug fix and feature.
- Fixed bug with how data is sent to elasticsearch.

- Added a feature to only allow data of a certain
  size to be buffered before sending to the
  elasticsearch server.  Configured with the
  LogElasticSearch::max_byte_size variable.
2012-06-16 22:22:40 -04:00
Seth Hall
a4df914ab7 Reduce the batch size to 1000 and add a maximum time interval for batches. 2012-06-15 20:53:09 -04:00
Robin Sommer
c7c3ff7af9 Adding a SOCKS test case.
However, I'm not sure the output is right.
2012-06-15 16:01:59 -07:00
Robin Sommer
19cf93be69 Moving my todos over to the tracker ticket. 2012-06-15 14:45:07 -07:00
Vlad Grigorescu
93a7a46af0 Merge branch 'master' of git://git.bro-ids.org/bro into elasticsearch 2012-06-15 10:46:48 -04:00
Bernhard Amann
c664c40ac2 now the writer supports tables and vectors.
still not tested, but using Log::default_writer=Log::WRITER_SQLITE seems to generate all
the right log-databases, etc.
2012-06-14 15:54:22 -07:00
Daniel Thayer
b66b022be0 Fix a warning message
When building the docs, a WARNING message was visible.  Reworded the
doc comment to prevent this warning.
2012-06-13 12:21:16 -05:00
Robin Sommer
1acb9fd91d Checkpointing the merge. Not done yet.
Merge remote-tracking branch 'origin/topic/tunnels' into topic/robin/tunnels-merge

* origin/topic/tunnels: (41 commits)
  Extend weird names that occur in core packet processing during decapsulation.
  Add Teredo analysis option to reduce false positive decapsulation.
  Just some cleanup/documentation of new tunnel-handling code.
  Memory leak fixes
  Add a config.h definition for IPPROTO_IPV4.
  Add AYIYA tunnel decapsulation unit test.
  Add Teredo-specific events.
  Refactor some of the NetSessions routines that recurse on IP packets.
  Add independent options to toggle the different decapsulation methods
  Add more sanity checks before recursing on encapsulated IP packets.
  Suppress Teredo weirds unless decapsulation was successful once before.
  Tunnel support performance optimization.
  Add Teredo tunnel decapsulation.
  Fix for IP tunnel UID persistence.
  Fix AYIYA analyzer tag.
  Add summary documentation to tunnels/main.bro.
  Make tunnels always identifiable by UID, tunnel.log now gets populated.
  Some improvements to the AYIYA analyzer.
  Remove Tunnel::decapsulate_ip option.
  Remove invalid IP-in-IP encapsulated protocol value.
  ...
2012-06-11 17:24:18 -07:00
Jon Siwek
6f346c8406 Add Teredo analysis option to reduce false positive decapsulation.
The Tunnel::yielding_teredo_decapsulation (on by default) makes it so
the Teredo analyzer doesn't attempt to decapsulate payloads when
there's already a sibling analyzer that thinks it's parsing the right
protocol.  Sometimes, UDP payloads just happen to look like they are
validly Teredo-encapsulated and doing further analysis on the
decapsulated packet can quickly turn into a weird; this change helps
reduce such weirds.
2012-06-07 13:01:10 -05:00
Jon Siwek
9ddb70b109 Merge branch 'master' into topic/tunnels
Conflicts:
	scripts/base/init-bare.bro
2012-06-06 14:53:57 -05:00
Jon Siwek
beacf581d3 Just some cleanup/documentation of new tunnel-handling code. 2012-06-06 14:40:11 -05:00
Robin Sommer
45f5900547 Merge remote-tracking branch 'origin/topic/jsiwek/load-sigs'
* origin/topic/jsiwek/load-sigs:
  Add @load-sigs directive for loading signature files (addresses #551).

Closes #551.
2012-06-06 11:49:39 -07:00
Jon Siwek
7599ac8f31 Memory leak fixes for bad usages of VectorVal ctor.
Many usages of the VectorVal ctor didn't account for the fact that
it automatically Ref's the VectorType argument and end up leaking it.
2012-06-06 11:50:15 -05:00
Jon Siwek
0bdbeb89e2 Memory leak fixes 2012-06-06 10:41:36 -05:00
Vlad Grigorescu
a3b330dbc6 Make default index name 'bro'. 2012-06-05 17:24:13 -04:00
Jon Siwek
854c625275 Add Teredo-specific events.
These are called "teredo_packet", "teredo_authentication",
"teredo_origin_indication", and "teredo_bubble" and may be raised
on a per-packet basis depending on Teredo encapsulation method.
2012-06-05 15:07:56 -05:00
Jon Siwek
976e8db155 Add independent options to toggle the different decapsulation methods 2012-06-05 10:17:41 -05:00
Vlad Grigorescu
7bee0b0d8e Added sending messages to ElasticSearch over HTTP. 2012-06-03 22:07:20 -04:00
Jon Siwek
dd4dd0ca6e Add @load-sigs directive for loading signature files (addresses #551). 2012-06-01 14:10:23 -05:00
Daniel Thayer
5f8cbf8a51 Change alarm summary email interval to 24hrs 2012-05-31 17:30:48 -05:00
Jon Siwek
9fb0065696 Merge branch 'master' into topic/tunnels
Conflicts:
	scripts/base/init-bare.bro
	src/Conn.cc
	src/Conn.h
	src/Sessions.cc
	src/Sessions.h
	src/const.bif
	src/types.bif
2012-05-31 16:13:14 -05:00