This change doubles the performance of for loops over empty tables.
A bro binary that prints out this size shows for
testing/external/bro-testing/2009-M57-day11-18.trace, for loops are run
over tables of size:
11477 for size 0
8371 for size 1
1227 for size 3
239 for size 2
141 for size 6
57 for size 5
10 for size 4
5 for size 7
2 for size 13
2 for size 8
2 for size 11
1 for size 9
~53% of the for loops were across an empty table. These loops come from
things like the for loop in the http script over c$http_state$pending
This change prevents the creation of an iteration cookie entirely if the
table is empty.
Using this test script:
const scan_ports: table[port] of count = { };
local x = 0;
while ( x < 20000000 ) {
for(p in scan_ports) {
}
++x;
}
$ time bro.orig -b ___bench.bro
real 0m10.732s
user 0m10.415s
sys 0m0.113s
$ time bro.nocookie -b ___bench.bro
real 0m4.694s
user 0m4.464s
sys 0m0.086s
Replaced some with InternalWarning or InternalAnalyzerError, the later
being a new method which signals the analyzer to not process further
input. Some usages I just removed if they didn't make sense or clearly
couldn't happen. Also did some minor refactors of related code while
reviewing/exploring ways to get rid of InternalError usages.
Also, for TCP content file write failures there's a new event:
"contents_file_write_failure".
* origin/topic/jsiwek/no-switch-fallthrough:
Add "fallthrough" keyword, require a flow statement to end case blocks.
Disable automatic case fallthrough in switch stmts. Addresses #754.
I've added a test for the error case where no break/fallthrough/return
is given.
Closes#754.
Since values for local variables are referenced by offset within a Frame
(not by identifier name), and event/hook handler bodies share a common
Frame, the value offsets for local variables in different handlers may
overlap. This meant locals in a handler without an initialization may
actually end up referring to the value of a previous handler's local
that has the same Frame offset. When executing the body, that can
possibly result in a type-conflict error or give give unexpected
results instead of a "use of uninitialized value" error.
This patch makes it so uninitialized locals do always refer to a null
value before executing the body of a event/hook handler, so that using
them without assigning a value within the body will connsistently give
a "use of uninitialized value" error.
Addresses #932.
Case blocks in switch statements now must end in a break, return, or
fallthrough statement to give best mix of safety, readability, and
flexibility.
The new fallthrough keyword explicitly allows control to be passed to the
next case block in a switch statement.
Addresses #754.
Case bodies now don't require a "break" statement to prevent fallthrough
to case bodies below. Empty case bodies generate an error message at
parse-time to help indicate the absence of automatic fallthrough; to
associate multiple values with a case, use "case 1, 2:" instead of
"case 1: case 2:".
They behave like C-style switches except case labels can be comprised
of multiple literal constants delimited by commas. Only atomic types
are allowed for now. Case label bodies that don't execute a "return"
or "break" statement will fall through to subsequent cases. A default
case label is allowed.
The return value of the call is an implicit boolean value of T if all
hook handlers ran, or F if one hook handler exited as a result of a
break statement and potentially prevented other handlers from running.
Scripts don't need to declare hooks with an explicit return type of bool
(internally, that's assumed), and any values given to (optional) return
statements in handler definitions are just ignored.
Addresses #918.
When iterating over a set with a "for" loop, bro would segfault
when the number of index variables was less than required.
Example: for ( [c1,c2] in s1 ) ...
where s1 is defined as set[addr,port,count].
* origin/topic/jsiwek/brofiler:
Fix superfluous/duplicate data getting in to testing coverage log.
Add "# @no-test" tag to blacklist statements from test coverage analysis.
Test coverage integration for external tests and complete suite.
Integrate Bro script coverage profiling with the btest suite.
Add simple profiling class to accumulate Stmt usage stats across runs.
Renaming environment variable BROFILER_FILE to BRO_PROFILER_FILE for
consistency. Yeah, I know, such a nice name! :)
Use the BROFILER_FILE environment variable to point to a file in
which Stmt usage statistics from Bro script-layer can be output.
This should be able to be used to check Bro script coverage that
that e.g. the entire test suite covers.
When using a `print` statement to write to a file that has raw output
enabled, NUL characters in string are no longer interpreted into "\0",
no newline is appended afterwards, and each argument to `print` is
written to the file without any additional separation.
(Re)Assigning to identifiers with the &raw_output attribute should also
now correctly apply the attribute to the file value being assigned.
Note that the write_file BiF should already be capable of raw string
data to a file, expect it bypasses the print_hook event.
Addresses #474
The Logger class is now in charge of reporting all errors, warnings,
informational messages, weirds, and syslogs. All other components
route their messages through the global bro_logger singleton.
The Logger class comes with these reporting methods:
void Message(const char* fmt, ...);
void Warning(const char* fmt, ...);
void Error(const char* fmt, ...);
void FatalError(const char* fmt, ...); // Terminate Bro.
void Weird(const char* name);
[ .. some more Weird() variants ... ]
void Syslog(const char* fmt, ...);
void InternalWarning(const char* fmt, ...);
void InternalError(const char* fmt, ...); // Terminates Bro.
See Logger.h for more information on these.
Generally, the reporting now works as follows:
- All non-fatal message are reported in one of two ways:
(1) At startup (i.e., before we start processing packets),
they are logged to stderr.
(2) During processing, they turn into events:
event log_message%(msg: string, location: string%);
event log_warning%(msg: string, location: string%);
event log_error%(msg: string, location: string%);
The script level can then handle them as desired.
If we don't have an event handler, we fall back to
reporting on stderr.
- All fatal errors are logged to stderr and Bro terminates
immediately.
- Syslog(msg) directly syslogs, but doesn't do anything else.
The three main types of messages can also be generated on the
scripting layer via new Log::* bifs:
Log::error(msg: string);
Log::warning(msg: string);
Log::message(msg: string);
These pass through the bro_logger as well and thus are handled in the
same way. Their output includes location information.
More changes:
- Removed the alarm statement and the alarm_hook event.
- Adapted lots of locations to use the bro_logger, including some
of the messages that were previously either just written to
stdout, or even funneled through the alarm mechanism.
- No distinction anymore between Error() and RunTime(). There's
now only one class of errors; the line was quite blurred already
anyway.
- util.h: all the error()/warn()/message()/run_time()/pinpoint()
functions are gone. Use the bro_logger instead now.
- Script errors are formatted a bit differently due to the
changes. What I've seen so far looks ok to me, but let me know
if there's something odd.
Notes:
- The default handlers for the new log_* events are just dummy
implementations for now since we need to integrate all this into
the new scripts anyway.
- I'm not too happy with the names of the Logger class and its
instance bro_logger. We now have a LogMgr as well, which makes
this all a bit confusing. But I didn't have a good idea for
better names so I stuck with them for now.
Perhaps we should merge Logger and LogMgr?
