Commit graph

1835 commits

Author SHA1 Message Date
Jon Siwek
bd4bc25eda Remove unneeded documentation cross-referencing.
Noticed these gave warnings due to missing namespace, but rather than
fix I'm just removing because they reference names in the same
module/file that will appear inches away from each other in the final
output.
2015-04-20 13:04:33 -05:00
Vlad Grigorescu
d0e4d17f31 Tweak the PE OS versions based on real-world traffic. 2015-04-20 12:49:42 -04:00
Vlad Grigorescu
928f870f58 Update pe/main.bro to user register_for_mime_types, ensuring it will also work with the upcoming Files framework changes. 2015-04-20 11:54:34 -04:00
Seth Hall
ed375167c8 File API updates complete.
Addresses BIT-1368.
2015-04-20 10:46:48 -04:00
Seth Hall
038e4c24f6 Merge remote-tracking branch 'origin/topic/jsiwek/bit-1368' into topic/seth/more-file-type-ident-fixes
Conflicts:
	src/file_analysis/File.cc
	testing/btest/Baseline/plugins.hooks/output
2015-04-20 09:36:40 -04:00
Seth Hall
faabe8a5e3 Fixes for file type identification.
- Backed out eTag changes.  The real world is more complicated
   than just using eTags to identify the same file.
 - A bit of code simplication in the http base scripts.
 - Test updates (more existing small problems were identified!).
 -
2015-04-20 09:34:09 -04:00
Vlad Grigorescu
c5ef85497d Register SIP analyzer to well known port. 2015-04-19 23:08:47 -04:00
Vlad Grigorescu
fb35a86b45 Fix indenting issue in main.bro 2015-04-19 22:25:48 -04:00
Vlad Grigorescu
11eb8a6371 Merge branch 'master' of ssh://git.bro.org/bro into topic/vladg/sip
Conflicts:
	scripts/base/init-default.bro
2015-04-19 22:14:35 -04:00
Vlad Grigorescu
efef5c6013 Merge branch 'topic/vladg/sip' of ssh://git.bro.org/bro into topic/vladg/sip
Conflicts:
	scripts/base/protocols/sip/main.bro
2015-04-19 22:13:33 -04:00
Vlad Grigorescu
e3d63bfee8 A bit of final script cleanup. 2015-04-19 21:38:34 -04:00
Vlad Grigorescu
71230fec81 Update baselines. 2015-04-19 21:16:35 -04:00
Vlad Grigorescu
d4bd5672c0 Documentation and a bit of overall cleanup. 2015-04-19 20:21:49 -04:00
Vlad Grigorescu
a2eff14e05 Add data about which tables are present. 2015-04-19 18:41:32 -04:00
Vlad Grigorescu
ea36686524 Remove the .idata parsing, as it can be more complicated in some cases. 2015-04-19 18:15:21 -04:00
Vlad Grigorescu
da89837693 Merge remote-tracking branch 'origin/master' into topic/vladg/file-analysis-exe-analyzer 2015-04-18 11:22:25 -04:00
Seth Hall
94afcd89c0 Small update for the SIP logs and DPD sig.
The logs were incomplete for intermediate 1xx responses.
2015-04-17 21:46:56 -04:00
Vlad Grigorescu
891813696a Add known ports to krb/main.bro 2015-04-17 21:26:15 -04:00
Vlad Grigorescu
1ff45c9fe1 Merge remote-tracking branch 'origin/master' into topic/vladg/kerberos 2015-04-17 20:29:34 -04:00
Vlad Grigorescu
e653036699 Fix SSH direction field being unset (addresses BIT-1365) 2015-04-17 19:55:56 -04:00
Robin Sommer
1e010fbb76 Merge remote-tracking branch 'origin/topic/johanna/conn-threshold'
* origin/topic/johanna/conn-threshold:
  Wrap threshold stuff up - fix two small bugs and update baselines.
  update GridFTP analyzer to use connection thresholding instead of polling
  Add high level api for thresholding that holds lists of thresholds and raises an event for each threshold exactly once.
  Allow setting packet and byte thresholds for connections.

BIT-1377 #merged
2015-04-17 13:02:31 -07:00
Robin Sommer
43a873d9b3 Merge remote-tracking branch 'origin/topic/jsiwek/bit-1376'
* origin/topic/jsiwek/bit-1376:
  Fix attempt for "internal error: unknown msg type 115 in Poll()"

BIT-1376 #merged
2015-04-17 12:40:05 -07:00
Johanna Amann
b44b725d59 Wrap threshold stuff up - fix two small bugs and update baselines. 2015-04-17 09:59:34 -07:00
Johanna Amann
024bb7206e update GridFTP analyzer to use connection thresholding instead
of polling
2015-04-17 07:15:53 -07:00
Johanna Amann
86c54c3dbd Add high level api for thresholding that holds lists of thresholds
and raises an event for each threshold exactly once.
2015-04-17 06:57:58 -07:00
Vlad Grigorescu
81bafb6c36 PE: Rehash the log a bit. 2015-04-16 22:56:47 -04:00
Vlad Grigorescu
4753e4a3c2 Make base_of_data optional. 2015-04-16 19:44:39 -05:00
Johanna Amann
e21238d454 add a few more flow_mod options and the option to check via a predicate
if a module wants to be responsible for a certain rule...
2015-04-16 15:44:46 -07:00
Johanna Amann
d876c044df Add high level api for thresholding that holds lists of thresholds
and raises an event for each threshold exactly once.
2015-04-16 15:36:48 -07:00
Jon Siwek
effeaa5b13 Fix attempt for "internal error: unknown msg type 115 in Poll()"
Under remote communication overload conditions, the child->parent
chunked IO may start rejecting chunks if over the hard cap.  Some
messages are made of two chunks, accepting the first part, but rejecting
the second can put the parent in a bad state and the next two chunks it
reads are likely to cause the error.

This patch just removes the rejecting functionality completely and so
now relies solely on shutting down remote peer connections to help
alleviate temporary overload conditions. The
"chunked_io_buffer_soft_cap" script variable can now tune when this
shutting down starts happening and the default setting is now double
what it used to be.  For constant overload conditions, communication.log
should keep stating "queue to parent filling up; shutting down heaviest
connection".

An alternative to completely removing the hard cap rejection code could
be ensuring that messages that involve a pair of chunks can never have
the second chunk be rejected when attempting to write it.

Addresses BIT-1376
2015-04-16 17:15:42 -05:00
Johanna Amann
a3bfa92125 introduce more mac address mac types, support them in OpenFlow plugin,
add support for a few rule types in OpenFlow plugin and add predicates
for matches and flow_mod modifiers.
2015-04-15 11:11:40 -07:00
Johanna Amann
236a895418 Update mozilla CA list.
This also patches a few tests to contain certificates that were removed.
Furthermore, we include the old CA file with the external tests and load
it automatically. Those traces are kind of old now, more and more of the
CAs in them are no longer valid and it does not really make sense to
update them on each change...
2015-04-14 16:40:41 -07:00
Johanna Amann
7d7578146f Add basic OpenFlow plugin for Pacf.
This also changes a few types in pacf and adds a few needed bits and
pieces to the OpenFlow framework.

And - it even has a testcase...
2015-04-14 15:24:22 -07:00
Johanna Amann
aad988d2f2 get rid of pacf prototype that we did not use. 2015-04-14 15:15:25 -07:00
Johanna Amann
c42fbdab12 move openflow to use subnets instead of addr for matches. 2015-04-14 15:00:49 -07:00
Johanna Amann
00204ab8a6 introduce &weaken attribute, which basically only prevents
the describe function for types to descend into record fields that
are marked with it.

With this, we can actually load the pacf scripts without crashing Bro
when running tests :)
2015-04-13 16:05:55 -07:00
Vlad Grigorescu
fa7946ae7d Checkpoint - Import Address Table being parsed. 2015-04-13 16:34:18 -05:00
Johanna Amann
21b78b7d92 add really simple log output plugin for openflow. 2015-04-13 12:55:38 -07:00
Jon Siwek
a55ce01ef3 API changes to file analysis mime type detection.
Removed "file_mime_type" and "file_mime_types" event, replacing them
with a new event called "file_metadata_inferred".  It has a record
argument of type "inferred_file_metadata", which contains the mime type
information that the earlier events used to supply.  The idea here is
that future extensions to the record with new metadata will be less
likely to break user code than the alternatives (adding new events or
new event parameters).

Addresses BIT-1368.
2015-04-10 16:31:29 -05:00
Johanna Amann
46058d0b02 a few small fixes to openflow
*rename module from Openflow to OpenFlow
*add match_conn function to convert conn_id to openflow match
*add a few things back into the openflow records like... table_id
*and - a test
2015-04-10 11:21:55 -07:00
Johanna Amann
70f6635cb1 do not load pacf by default. There is some recursive record
definition in there Bro does not like too much...
2015-04-10 11:19:09 -07:00
Seth Hall
49926ad7bf Merge remote-tracking branch 'origin/master' into topic/seth/more-file-type-ident-fixes 2015-04-09 23:58:52 -04:00
Seth Hall
0ee7d82e19 Make HTTP 206 reassembly require ETags by default. 2015-04-09 23:58:46 -04:00
Seth Hall
e8c87e19bd More file type identification improvements
- Split fonts into their own file.
 - Improved JSON matching.
 - Added XML-RPC content matching using application/xml-rpc
 - Added OCSP requests
2015-04-09 01:23:55 -04:00
Johanna Amann
94c67dc030 Merge remote-tracking branch 'origin/topic/robin/pacf' into topic/johanna/openflow 2015-04-07 17:28:38 -07:00
Johanna Amann
883da516ee move pacf skeleton away to be able to replace it with old proposal of
Robin.
2015-04-07 17:27:50 -07:00
Johanna Amann
dbc51371cb Rewrite big parts of the Openflow framework.
The API now does not follow the openflow specification quite as closely,
however I think it is much more usable. Furthermore, the Ryu plugin was
basically completely rewritten and is now more usable for general flow
manipulation.

This also adds a debug mode that just outputs the json fragments that
would be sent to ryu. At the moment, Ryu still assumes that every
request that it receives succeeds - it is not possible to get an error
message from the controller. Instead, one has to check if a flow was
added by doing a second REST request. Which seems unnecessary, and also
requires complete json parsing functionality. Hence we are not doing
that at the moment.

The alternative would be to use an external script for the actual
add-and-check-operation.
2015-04-07 15:37:46 -07:00
Vlad Grigorescu
b4498a4142 Some changes to fix PE analyzer on master. 2015-04-07 14:45:15 -07:00
Vlad Grigorescu
010cd80f2f Merge remote-tracking branch 'origin/master' into topic/vladg/file-analysis-exe-analyzer 2015-04-07 14:19:15 -07:00
Johanna Amann
4195a0066a move the json stuff into utils - I guess we will need functionality like
this not just for openflow at some point of time.
2015-04-07 10:43:36 -07:00