* origin/fastpath:
Fix missing action in notice policy for looking up GeoIP data.
Better persistent state config warning messages (fixes#433).
A few updates for SQL injection detection.
Fixed some DPD signatures for IRC. Fixes ticket #311.
Removing Off_Port_Protocol_Found notice.
SSH::Interesting_Hostname_Login cleanup. Fixes#664.
Teach Broxygen to more generally reference attribute values by name.
Fixed a really dumb bug that was causing the malware hash registry script to break.
Fix Broxygen confusing scoped id at start of line as function parameter.
Remove remnant of libmagic optionality
- The biggest change is the change in notice names from
HTTP::SQL_Injection_Attack_Against to
HTTP::SQL_Injection_Victim
- A few new SQL injection attacks in the tests that we need to
support at some point.
- Added a field named $last_alert to the SSL log. This doesn't even
indicate the direction the alert was sent, but we need to start somewhere.
- The x509_certificate function has an is_orig field now instead of
is_server and it's position in the argument list has moved.
- A bit of reorganization and cleanup in the core analyzer.
- New script extracted from weird.bro to implement the
connection related "weird" data into an optionally
loaded script.
- Adjusted the default notice tuning to stop ignoring
the connection related weirds since they aren't loaded
by default anymore.
- Certificate validation volume has been greatly cut down by
caching results.
- Cert hashing is now done in one place instead of being repeated
everywhere a cert hash was needed.
- Some small cleanups for notice suppression that should greatly reduce
duplicate notice volume about invalid certificates.
- With the software-browser-plugins script you can watch for Omniture
advertising servers to grab the list of installed plugins.
- I reorganized the plugin detection a bit too to abstract it better.
- Removed the WEB_ prefix from all of the Software::Type HTTP enums.
They were essentially redundant due to the full name already being
HTTP::SERVER (for example).
- I was wildly misunderstanding the semantics of the
connection_state_remove event. That's fixed now in
my brain and in the script.
- If a service isn't detected, logging is delayed by
2 minutes to try and allow for another connection
to happen that actually does the protocol correctly
and detectably.
- scan.bro and hot.conn.bro will be returning soon.
- The rest are going to return as updated protocol analysis
scripts and new/updated frameworks later.
Updated README and collected coverage-related tests in a common dir.
There are still coverage failures resulting from either the following
scripts not being @load'd in the default bro mode:
base/frameworks/time-machine/notice.bro
base/protocols/http/partial-content.bro
base/protocols/rpc/main.bro
Or the following result in errors when @load'd:
policy/protocols/conn/scan.bro
policy/hot.conn.bro
If these are all scripts-in-progress, can we move them all to live
outside the main scripts/ directory until they're ready?
- protocols/ssl/expiring-certs uses time based information from
certificates to determine if they will expire soon, have already
expired, or haven't yet become valid.
- protocols/ssl/extract-certs-pem is a script for taking certs off
the line and converting them to PEM certificates with the openssl
command line tool then dumping them to a file.
- Removed an notice definition from the base SSL scripts.
- Moved a logging stream ID into the export section for known-services
and bumped priority for creating the stream.
- Adding configuration knobs for the SQL injection attack detection
script and renaming the HTTP::SQL_Injection_Attack notice to
HTTP::SQL_Injection_Attack_Against
- Bumped priority when creating Known::CERTS_LOG.
- Log path's are generated in the scripting land
now. The default Log stream ID to path string
mapping works like this:
- Notice::LOG -> "notice"
- Notice::POLICY_LOG -> "notice_policy"
- TestModule::LOG -> "test_module"
- Logging streams updated across all of the shipped
scripts to be more user friendly. Instead of
the logging stream ID HTTP::HTTP, we now have
HTTP::LOG, etc.
- The priorities on some bro_init handlers have
been adjusted to make the process of applying
filters or disabling streams easier for users.
- Missing GeoIP databases now generate warnings/errors that go through
the reporter framework instead of hitting GeoIP's internal use of
stderr
- lookup_location now just queries for country code if the city database
was not loaded, which gets rid of invalid database type errors.
- lookup_location now leaves missing fields uninitialized in the
returned geo_location record value. Updated existing scripts to
check for initialized fields in geo_location records before use.
- Fixed support for GeoIP's IPv6 API and databases
* origin/topic/gregor/script-polishing:
Tune when c$conn is set.
Set c$conn (for logging) in new_connection() event.
(Semiautomatically) convert the comments in bare-init.bro into autodoc ones.
Add ConnSize_Analyzer's fields to conn.log
SSH base scripts: make sure ConnSizeAnalyzer variables are available before using them.
Make reference to the other script a link
- Since each host in a cluster has it's own view of the metrics
the only time the manager would get a chance for a global view
is the break_interval. This update improves that time. If a
worker crosses 10% of the full threshold, it will send it's
value to the manager which can then ask the rest of the cluster
for a global view. The manager then adds all of the values for
each workers metric indexes together and will do the notice
if it crosses the threshold so that it isn't dependent on
waiting for the break interval to hit. This functionality
works completely independently of the break_interval too. Logging
will happen as normal.
- Small update for SSH bruteforcer detection to match additions in
the metrics framework API.
- The hope is that this update is mostly invisible from anyone's
perspective. The only affect it should have on users is to better
the detection of metric values crossing thresholds on cluster
deployments.
- fixing some Metrics::add_data() call signatures
- slight refactors to cluster framework @if and adding a NONE NodeType for
so local_node_type() will return that instead of just emitting an error
when cluster mode isn't enabled
- `make restdoc` target now uses bro's bare-mode
- bro scripts generated from bifs now really only live in the build/src/base/
directory and changed the DocSourcesList.cmake to dynamically figure out
what bifs exist by looking in src/ instead of build/src/
- add some missing @load dependencies
