Mirror/zeek
1
0
Fork 0
mirror of https://github.com/zeek/zeek.git synced 2025-10-08 01:28:20 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 3

First prototype of new analyzer framework.

Browse source 
This is a larger internal change that moves the analyzer
infrastructure to a more flexible model where the available analyzers
don't need to be hardcoded at compile time anymore. While currently
they actually still are, this will in the future enable external
analyzer plugins. For now, it does already add the capability to
dynamically enable/disable analyzers from script-land, replacing the
old Analyzer::Available() methods.

There are three major parts going into this:

    - A new plugin infrastructure in src/plugin. This is independent
      of analyzers and will eventually support plugins for other parts
      of Bro as well (think: readers and writers). The goal is that
      plugins can be alternatively compiled in statically or loadead
      dynamically at runtime from a shared library. While the latter
      isn't there yet, there'll be almost no code change for a plugin
      to make it dynamic later (hopefully :)

    - New analyzer infrastructure in src/analyzer. I've moved a number
      of analyzer-related classes here, including Analyzer and DPM;
      the latter now renamed to Analyzer::Manager. More will move here
      later. Currently, there's only one plugin here, which provides
      *all* existing analyzers. We can modularize this further in the
      future (or not).

    - A new script interface in base/framework/analyzer. I think that
      this will eventually replace the dpm framework, but for now
      that's still there as well, though some parts have moved over.

I've also remove the dpd_config table; ports are now configured via
the analyzer framework. For exmaple, for SSH:

    const ports = { 22/tcp } &redef;

    event bro_init() &priority=5
        {
        ...
        Analyzer::register_for_ports(Analyzer::ANALYZER_SSH, ports);
        }

As you can see, the old ANALYZER_SSH constants have more into an enum
in the Analyzer namespace.

This is all hardly tested right now, and not everything works yet.
There's also a lot more cleanup to do (moving more classes around;
removing no longer used functionality; documenting script and C++
interfaces; regression tests). But it seems to generally work with a
small trace at least.

The debug stream "dpm" shows more about the loaded/enabled analyzers.

A new option -N lists loaded plugins and what they provide (including
those compiled in statically; i.e., right now it outputs all the
analyzers).

This is all not cast-in-stone yet, for some things we need to see if
they make sense this way. Feedback welcome.
This commit is contained in:
Robin Sommer 2013-03-20 13:32:36 -07:00
parent 9caf6e4884
commit af1809aaa3
166 changed files with 2717 additions and 1642 deletions
Download patch file Download diff file Expand all files Collapse all files

6
scripts/base/protocols/conn/inactivity.bro
View file

@ -6,9 +6,9 @@ module Conn;
export {
## Define inactivity timeouts by the service detected being used over
## the connection.
const analyzer_inactivity_timeouts: table[AnalyzerTag] of interval = {
const analyzer_inactivity_timeouts: table[Analyzer::Tag] of interval = {
# For interactive services, allow longer periods of inactivity.
[[ANALYZER_SSH, ANALYZER_FTP]] = 1 hrs,
[[Analyzer::ANALYZER_SSH, Analyzer::ANALYZER_FTP]] = 1 hrs,
} &redef;
## Define inactivity timeouts based on common protocol ports.
@ -18,7 +18,7 @@ export {
}
event protocol_confirmation(c: connection, atype: count, aid: count)
event protocol_confirmation(c: connection, atype: Analyzer::Tag, aid: count)
{
if ( atype in analyzer_inactivity_timeouts )
set_inactivity_timeout(c$id, analyzer_inactivity_timeouts[atype]);

10
scripts/base/protocols/dns/main.bro
View file

@ -117,19 +117,17 @@ redef capture_filters += {
["netbios-ns"] = "udp port 137",
};
const dns_ports = { 53/udp, 53/tcp, 137/udp, 5353/udp, 5355/udp };
redef dpd_config += { [ANALYZER_DNS] = [$ports = dns_ports] };
const dns_udp_ports = { 53/udp, 137/udp, 5353/udp, 5355/udp };
const dns_tcp_ports = { 53/tcp };
redef dpd_config += { [ANALYZER_DNS_UDP_BINPAC] = [$ports = dns_udp_ports] };
redef dpd_config += { [ANALYZER_DNS_TCP_BINPAC] = [$ports = dns_tcp_ports] };
redef likely_server_ports += { 53/udp, 53/tcp, 137/udp, 5353/udp, 5355/udp };
redef likely_server_ports += { dns_udp_ports, dns_tcp_ports };
event bro_init() &priority=5
{
Log::create_stream(DNS::LOG, [$columns=Info, $ev=log_dns]);
Analyzer::register_for_ports(Analyzer::ANALYZER_DNS_TCP_BINPAC, dns_tcp_ports);
Analyzer::register_for_ports(Analyzer::ANALYZER_DNS_UDP_BINPAC, dns_udp_ports);
}
function new_session(c: connection, trans_id: count): Info

10
scripts/base/protocols/ftp/main.bro
View file

@ -96,11 +96,10 @@ redef record connection += {
};
# Configure DPD
const ports = { 21/tcp, 2811/tcp } &redef; # 2811/tcp is GridFTP.
redef capture_filters += { ["ftp"] = "port 21 and port 2811" };
redef dpd_config += { [ANALYZER_FTP] = [$ports = ports] };
redef likely_server_ports += { 21/tcp, 2811/tcp };
const ports = { 21/tcp, 2811/tcp };
redef likely_server_ports += { ports };
# Establish the variable for tracking expected connections.
global ftp_data_expected: table[addr, port] of Info &create_expire=5mins;
@ -108,6 +107,7 @@ global ftp_data_expected: table[addr, port] of Info &create_expire=5mins;
event bro_init() &priority=5
{
Log::create_stream(FTP::LOG, [$columns=Info, $ev=log_ftp]);
Analyzer::register_for_ports(Analyzer::ANALYZER_FTP, ports);
}
## A set of commands where the argument can be expected to refer
@ -228,7 +228,7 @@ event ftp_request(c: connection, command: string, arg: string) &priority=5
{
c$ftp$passive=F;
ftp_data_expected[data$h, data$p] = c$ftp;
expect_connection(id$resp_h, data$h, data$p, ANALYZER_FILE, 5mins);
Analyzer::expect_connection(id$resp_h, data$h, data$p, Analyzer::ANALYZER_FILE, 5mins);
}
else
{
@ -281,7 +281,7 @@ event ftp_reply(c: connection, code: count, msg: string, cont_resp: bool) &prior
data$h = id$resp_h;
ftp_data_expected[data$h, data$p] = c$ftp;
expect_connection(id$orig_h, data$h, data$p, ANALYZER_FILE, 5mins);
Analyzer::expect_connection(id$orig_h, data$h, data$p, Analyzer::ANALYZER_FILE, 5mins);
}
else
{

27
scripts/base/protocols/http/main.bro
View file

@ -119,29 +119,26 @@ redef record connection += {
http_state: State &optional;
};
# Initialize the HTTP logging stream.
event bro_init() &priority=5
{
Log::create_stream(HTTP::LOG, [$columns=Info, $ev=log_http]);
}
# DPD configuration.
const ports = {
80/tcp, 81/tcp, 631/tcp, 1080/tcp, 3128/tcp,
8000/tcp, 8080/tcp, 8888/tcp,
};
redef dpd_config += {
[[ANALYZER_HTTP, ANALYZER_HTTP_BINPAC]] = [$ports = ports],
};
redef capture_filters += {
["http"] = "tcp and port (80 or 81 or 631 or 1080 or 3138 or 8000 or 8080 or 8888)"
};
redef likely_server_ports += {
80/tcp, 81/tcp, 631/tcp, 1080/tcp, 3138/tcp,
const ports = {
80/tcp, 81/tcp, 631/tcp, 1080/tcp, 3128/tcp,
8000/tcp, 8080/tcp, 8888/tcp,
};
redef likely_server_ports += { ports };
# Initialize the HTTP logging stream and ports.
event bro_init() &priority=5
{
Log::create_stream(HTTP::LOG, [$columns=Info, $ev=log_http]);
Analyzer::register_for_ports(Analyzer::ANALYZER_HTTP, ports);
}
function code_in_range(c: count, min: count, max: count) : bool
{
return c >= min && c <= max;

2
scripts/base/protocols/irc/dcc-send.bro
View file

@ -104,7 +104,7 @@ event irc_dcc_message(c: connection, is_orig: bool,
c$irc$dcc_file_name = argument;
c$irc$dcc_file_size = size;
local p = count_to_port(dest_port, tcp);
expect_connection(to_addr("0.0.0.0"), address, p, ANALYZER_FILE, 5 min);
Analyzer::expect_connection(to_addr("0.0.0.0"), address, p, Analyzer::ANALYZER_FILE, 5 min);
dcc_expected_transfers[address, p] = c$irc;
}

7
scripts/base/protocols/irc/main.bro
View file

@ -45,14 +45,13 @@ redef capture_filters += { ["irc-6668"] = "port 6668" };
redef capture_filters += { ["irc-6669"] = "port 6669" };
# DPD configuration.
const irc_ports = { 6666/tcp, 6667/tcp, 6668/tcp, 6669/tcp };
redef dpd_config += { [ANALYZER_IRC] = [$ports = irc_ports] };
redef likely_server_ports += { 6666/tcp, 6667/tcp, 6668/tcp, 6669/tcp };
const ports = { 6666/tcp, 6667/tcp, 6668/tcp, 6669/tcp };
redef likely_server_ports += { ports };
event bro_init() &priority=5
{
Log::create_stream(IRC::LOG, [$columns=Info, $ev=irc_log]);
Analyzer::register_for_ports(Analyzer::ANALYZER_IRC, ports);
}
function new_session(c: connection): Info

6
scripts/base/protocols/modbus/main.bro
View file

@ -31,12 +31,14 @@ redef record connection += {
# Configure DPD and the packet filter.
redef capture_filters += { ["modbus"] = "tcp port 502" };
redef dpd_config += { [ANALYZER_MODBUS] = [$ports = set(502/tcp)] };
redef likely_server_ports += { 502/tcp };
const ports = { 502/tcp };
redef likely_server_ports += { ports };
event bro_init() &priority=5
{
Log::create_stream(Modbus::LOG, [$columns=Info, $ev=log_modbus]);
Analyzer::register_for_ports(Analyzer::ANALYZER_MODBUS, ports);
}
event modbus_message(c: connection, headers: ModbusHeaders, is_orig: bool) &priority=5

8
scripts/base/protocols/smtp/main.bro
View file

@ -74,9 +74,6 @@ export {
const mail_path_capture = ALL_HOSTS &redef;
global log_smtp: event(rec: Info);
## Configure the default ports for SMTP analysis.
const ports = { 25/tcp, 587/tcp } &redef;
}
redef record connection += {
@ -86,13 +83,14 @@ redef record connection += {
# Configure DPD
redef capture_filters += { ["smtp"] = "tcp port 25 or tcp port 587" };
redef dpd_config += { [ANALYZER_SMTP] = [$ports = ports] };
redef likely_server_ports += { 25/tcp, 587/tcp };
const ports = { 25/tcp, 587/tcp };
redef likely_server_ports += { ports };
event bro_init() &priority=5
{
Log::create_stream(SMTP::LOG, [$columns=SMTP::Info, $ev=log_smtp]);
Analyzer::register_for_ports(Analyzer::ANALYZER_SMTP, ports);
}
function find_address_in_smtp_header(header: string): string

5
scripts/base/protocols/socks/main.bro
View file

@ -34,9 +34,13 @@ export {
global log_socks: event(rec: Info);
}
const ports = { 1080/tcp };
redef likely_server_ports += { ports };
event bro_init() &priority=5
{
Log::create_stream(SOCKS::LOG, [$columns=Info, $ev=log_socks]);
Analyzer::register_for_ports(Analyzer::ANALYZER_SOCKS, ports);
}
redef record connection += {
@ -45,7 +49,6 @@ redef record connection += {
# Configure DPD
redef capture_filters += { ["socks"] = "tcp port 1080" };
redef dpd_config += { [ANALYZER_SOCKS] = [$ports = set(1080/tcp)] };
redef likely_server_ports += { 1080/tcp };
function set_session(c: connection, version: count)

8
scripts/base/protocols/ssh/main.bro
View file

@ -76,10 +76,11 @@ export {
}
# Configure DPD and the packet filter
redef capture_filters += { ["ssh"] = "tcp port 22" };
redef dpd_config += { [ANALYZER_SSH] = [$ports = set(22/tcp)] };
redef likely_server_ports += { 22/tcp };
const ports = { 22/tcp };
redef capture_filters += { ["ssh"] = "tcp port 22" };
redef likely_server_ports += { ports };
redef record connection += {
ssh: Info &optional;
@ -88,6 +89,7 @@ redef record connection += {
event bro_init() &priority=5
{
Log::create_stream(SSH::LOG, [$columns=Info, $ev=log_ssh]);
Analyzer::register_for_ports(Analyzer::ANALYZER_SSH, ports);
}
function set_session(c: connection)

28
scripts/base/protocols/ssl/main.bro
View file

@ -94,11 +94,6 @@ redef record Info += {
delay_tokens: set[string] &optional;
};
event bro_init() &priority=5
{
Log::create_stream(SSL::LOG, [$columns=Info, $ev=log_ssl]);
}
redef capture_filters += {
["ssl"] = "tcp port 443",
["nntps"] = "tcp port 563",
@ -117,16 +112,9 @@ redef capture_filters += {
const ports = {
443/tcp, 563/tcp, 585/tcp, 614/tcp, 636/tcp,
989/tcp, 990/tcp, 992/tcp, 993/tcp, 995/tcp, 5223/tcp
};
} &redef;
redef dpd_config += {
[[ANALYZER_SSL]] = [$ports = ports]
};
redef likely_server_ports += {
443/tcp, 563/tcp, 585/tcp, 614/tcp, 636/tcp,
989/tcp, 990/tcp, 992/tcp, 993/tcp, 995/tcp, 5223/tcp
};
redef likely_server_ports += { ports };
# A queue that buffers log records.
global log_delay_queue: table[count] of Info;
@ -135,6 +123,12 @@ global log_delay_queue_head = 0;
# The bottom queue index that points to the next record to be flushed.
global log_delay_queue_tail = 0;
event bro_init() &priority=5
{
Log::create_stream(SSL::LOG, [$columns=Info, $ev=log_ssl]);
Analyzer::register_for_ports(Analyzer::ANALYZER_SSL, ports);
}
function set_session(c: connection)
{
if ( ! c?$ssl )
@ -288,14 +282,14 @@ event ssl_established(c: connection) &priority=-5
finish(c);
}
event protocol_confirmation(c: connection, atype: count, aid: count) &priority=5
event protocol_confirmation(c: connection, atype: Analyzer::Tag, aid: count) &priority=5
{
# Check by checking for existence of c$ssl record.
if ( c?$ssl && analyzer_name(atype) == "SSL" )
if ( c?$ssl && atype == Analyzer::ANALYZER_SSL )
c$ssl$analyzer_id = aid;
}
event protocol_violation(c: connection, atype: count, aid: count,
event protocol_violation(c: connection, atype: Analyzer::Tag, aid: count,
reason: string) &priority=5
{
if ( c?$ssl )

6
scripts/base/protocols/syslog/main.bro
View file

@ -27,10 +27,9 @@ export {
}
redef capture_filters += { ["syslog"] = "port 514" };
const ports = { 514/udp } &redef;
redef dpd_config += { [ANALYZER_SYSLOG_BINPAC] = [$ports = ports] };
redef likely_server_ports += { 514/udp };
const ports = { 514/udp };
redef likely_server_ports += { ports };
redef record connection += {
syslog: Info &optional;
@ -39,6 +38,7 @@ redef record connection += {
event bro_init() &priority=5
{
Log::create_stream(Syslog::LOG, [$columns=Info]);
Analyzer::register_for_ports(Analyzer::ANALYZER_SYSLOG_BINPAC, ports);
}
event syslog_message(c: connection, facility: count, severity: count, msg: string) &priority=5