traces and more complex Bro configurations.
That's "Type 2" in the terminology of
http://www-new.bro-ids.org/devel/projects/testing.html
This leverages btest as well and will replace the set of scripts that
the old private test-suite is using. It can accomodate tests with both
public and private traces.
This is a checkpoint commmit, not much there yet in terms of actual
testing. Also still need to figure out a mechanmism to allow for
different baselines based on which branch we're testing.
* remotes/origin/topic/policy-scripts-new:
Add RPC/SSL scripts to doc generation target.
Update the generated script doc organization.
Changing back the last commit. It made things worse.
Trying a different method of record parsing for SSL analyzer.
Moved the RPC script into the right place.
More SSL script cleanup.
Fixed a segfault from empty strings in the SSL analyzer.
Removing what I believe is the last stray print statement from the SSL analyzer.
Fixed more eternal looping bugs in the SSL analyzer.
Updates and fixes for the SSL analyzer.
Changes to make generated script docs understand new policy/ hierarchy.
Added an additional master TOC index for Bro script packages that
automatically gets filled in at ``make doc`` time. The master TOC
links to per-package indexes which contains links to all the scripts
contained within it along with their summary. The per-package index
is also now automatically generated/derived from the path component
of the script passed into the rest_target() macro.
- Fixed a pretty major eternal looping bug in the SSL analyzer and
a few other parsing problems.
- New core scripts and log for SSL connection analysis.
- New known-certs script for logging certificate assets.
Added an arg to the search_for_files() util function that can return
the subpath of BROPATH's policy/ dir in which the loaded file is found.
This subpath is then used in both the the reST file's document title
(so that script's named e.g. "base.bro" actually have some context) and
in figuring out how to interlink with other generated docs of other
scripts that are found in @load directives.
I still need to overhaul things so the loading of "packages" is
documented in a meaningful way and that the CMake targets are able
to generate indexes for packages.
* remotes/origin/topic/policy-scripts-new: (264 commits)
Update core.vlan-mpls baseline.
Update core.conn-uid test/baseline.
Fixing tests / updating baselines.
Renaming a poorly named notice.
Resolving a conflict in scan.l.
Notice framework updates.
Fixed bug I just introduced to known-services.
Known services now only logs for "normal" connections.
Fix the name of the DPD log.
Updating the netstats script to match the new internal code.
Removed expected_connection_timeout variable.
New default notice actions for emailing network admins.
New default notice action for emailing network admins.
weird.bro moved into notice framework.
Removed a redundant and out of place mozilla CA list.
Renamed the stream for the notice log.
Small consistency tweaks for the communications framework.
Finishing the inactivity script.
Intel framework test passes now.
Reorganized some of the tests for consistency.
...
- core.load-unload: scripts that get loaded by default changed, so to
make the test insensitive to that in the future, I changed the test
to just check that the stdout is empty (the @unload'd script would have
had output there)
- policy.frameworks.logging.rotate-custom: I saw that the ordering of
the log postprocessor output caused a failure for me even though the
overall content was the same, so it now sorts that part before diff'ing
- core.print-bpf-filters-ipv[4|6]: packet-filter log file name changed
- policy.protocols.conn.known-services: logging file names changes and
local_nets is now in the Site module
Caused by changes made while working on the way loaded scripts are tracked
(commit 9de6e9170c to master branch) that
didn't take into account an unmerged commit on the policy-scripts-new branch
(56a946568a).
(re-added warning comments that were removed in the former commit since they
still seem relevant now)
- Fixed problem where notices were logged even if they
didn't have the ACTION_FILE action applied.
- New PolicyItem element, $halt. It's used for halting
the policy processing if a predicate returns T.
This replaces the ACTION_STOP action.
- Initial hacky email extension mechanism.
- Removed the IDMEF line. When that added back later
it will likely be done more modularly.
* origin/topic/seth/net-stats-bif:
Removing a stray print statement.
Changed netstats (packet loss) handling to script-land.
Nice idea to pass the old data into a regular scheduled event!
Conflicts:
src/event.bif
* origin/fastpath:
Raise internal error when failing to read contents of state file with -x option
I've changed this to use error() instead of internal_error(). The
latter should only be used for logic errors that indicate a bug in
Bro. In this case, the message flags a problem that's more likely to
be external.
The Logger class is now in charge of reporting all errors, warnings,
informational messages, weirds, and syslogs. All other components
route their messages through the global bro_logger singleton.
The Logger class comes with these reporting methods:
void Message(const char* fmt, ...);
void Warning(const char* fmt, ...);
void Error(const char* fmt, ...);
void FatalError(const char* fmt, ...); // Terminate Bro.
void Weird(const char* name);
[ .. some more Weird() variants ... ]
void Syslog(const char* fmt, ...);
void InternalWarning(const char* fmt, ...);
void InternalError(const char* fmt, ...); // Terminates Bro.
See Logger.h for more information on these.
Generally, the reporting now works as follows:
- All non-fatal message are reported in one of two ways:
(1) At startup (i.e., before we start processing packets),
they are logged to stderr.
(2) During processing, they turn into events:
event log_message%(msg: string, location: string%);
event log_warning%(msg: string, location: string%);
event log_error%(msg: string, location: string%);
The script level can then handle them as desired.
If we don't have an event handler, we fall back to
reporting on stderr.
- All fatal errors are logged to stderr and Bro terminates
immediately.
- Syslog(msg) directly syslogs, but doesn't do anything else.
The three main types of messages can also be generated on the
scripting layer via new Log::* bifs:
Log::error(msg: string);
Log::warning(msg: string);
Log::message(msg: string);
These pass through the bro_logger as well and thus are handled in the
same way. Their output includes location information.
More changes:
- Removed the alarm statement and the alarm_hook event.
- Adapted lots of locations to use the bro_logger, including some
of the messages that were previously either just written to
stdout, or even funneled through the alarm mechanism.
- No distinction anymore between Error() and RunTime(). There's
now only one class of errors; the line was quite blurred already
anyway.
- util.h: all the error()/warn()/message()/run_time()/pinpoint()
functions are gone. Use the bro_logger instead now.
- Script errors are formatted a bit differently due to the
changes. What I've seen so far looks ok to me, but let me know
if there's something odd.
Notes:
- The default handlers for the new log_* events are just dummy
implementations for now since we need to integrate all this into
the new scripts anyway.
- I'm not too happy with the names of the Logger class and its
instance bro_logger. We now have a LogMgr as well, which makes
this all a bit confusing. But I didn't have a good idea for
better names so I stuck with them for now.
Perhaps we should merge Logger and LogMgr?
- When ACTION_EMAIL_ADMIN_ORIG or ACTION_EMAIL_ADMIN_RESP
is applied to a notice,
the email addresses associated with the address
are collected from the new local_admins table
and the email is sent to all discovered email addresses.
- The site.bro script is now in the Site module.
- Some other small cleanup.
- When ACTION_EMAIL_ADMIN is applied to a notice,
the email addresses associated with the address
are collected from the new local_admins table
and the email is sent to all discovered email addresses.
- The site.bro script is now in the Site module.
- Some other small cleanup.
