If the input framework was used to read event streams and
those streams contained records with more than one field, not all
elements of the threading Values were cleaned up.
The reason for this is, that the SendEventStreamEvent function
returned the number of record elements in the outmost record
instead of the number of unrolled elements in the whole vector.
This number is later used to determine how many objects to delete.
Also - add a whole bunch of leak checks for the input framework
that would have caught that and should cover quite a number of
use-cases.
The dump-events.bro generates output that would show in the Broxygen
all-script tests. Suppressing that manually in the test for lack of a
better idea.
Omission of the low index defaults to 0:
s = "12345"; s[:3] == "123"
Omission of the high index defaults to length of the string:
s = "12345"; s[3:] == "45"
Changes:
- Changing semantics of the new_event() meta event: it's raised
only for events that have a handler defined. There are too many
checks in Bro that prevent events wo/ handler from being even
prepared to raise to do that differently.
- Adding test case.
* topic/robin/event-dumper:
New script misc/dump-events.bro, along with core support, that dumps events Bro is raising in an easily readable form.
Prettyfing Describe() for record types.
BIT-1098
* origin/topic/jsiwek/broxygen:
Fix Broxygen-related compile errors.
Add a Broxygen coverage test.
Internal Broxygen organization/documentation/polish.
Add unit tests for Broxygen config file targets.
Change Broxygen config file format.
Broxygen doc-related test updates. Fix two regressions.
A couple documentation fixes.
Integrate new Broxygen functionality into Sphinx.
Implement majority of Broxygen features delegated to Bro.
Broxygen can now read a config file specifying particular targets.
Remove unneeded Broxygen comments in scan.bro.
Replace safe_basename/safe_dirname w/ SafeBasename/SafeDirname.
Add BIF interface for retrieving comments/docs.
Quick optimization to Broxygen doc gathering.
Flesh out Broxygen doc-gathering skeleton.
Refactor search_for_file() util function.
Initial skeleton of new Broxygen infrastructure.
May fix a sporadic failure, in which case it was just not getting enough
time to run or lookup_hostname() taking longer than ~3 seconds. Else,
the new output should give more hints on what's going wrong. In any
case, termination conditions for the test are now explicit.
Slice ranges were not correctly determined for negative indices and also
off by one in general (included one more element at the end of the
substring than what actually matched the index range).
It's now equivalent to Python slice notation. Accessing a string at
a single index is also the same as Python except that an out-of-range
index returns an empty string instead of throwing an expection.
Broccoli clients can only recv events w/ vectors for now.
Also changed ordering of Bro type tag enum -- the addition of opaque
types changed the value of the vector type, making broccoli.h's
definition out of sync. Probably could have just changed broccoli's
definition, but seems more correct to go back to using the same value
for vectors as they were before opaques. It's also better in case
there's some other location I'm not aware of where the values are
replicated.
in an easily readable form.
This is for debugging purposes, obviously.
Example, including only SMTP events:
> bro -r smtp.trace misc/dump-events.bro DumpEvents::include=/smtp/
[...]
1254722768.219663 smtp_reply
[0] c: connection = [id=[orig_h=10.10.1.4, orig_p=1470/tcp, resp_h=74.53.140.153, [...]
[1] is_orig: bool = F
[2] code: count = 220
[3] cmd: string = >
[4] msg: string = xc90.websitewelcome.com ESMTP Exim 4.69 #1 Mon, 05 Oct 2009 01:05:54 -0500
[5] cont_resp: bool = T
1254722768.219663 smtp_reply
[0] c: connection = [id=[orig_h=10.10.1.4, orig_p=1470/tcp, resp_h=74.53.140.153, [...]
[1] is_orig: bool = F
[2] code: count = 220
[3] cmd: string = >
[4] msg: string = We do not authorize the use of this system to transport unsolicited,
[5] cont_resp: bool = T
[...]
Fixed typos in documentation of hexstr_to_bytestring.
Also added documentation that was missing for function parameters
and return values of other BIFs.
- Move notice index wrapper doc to doc/script-reference -- doc/scripts
no longer contains any static documentation because that location
will be managed by Bro to generate per-script docs.
- :doc: references for generated per-script docs now need the ".bro"
suffix. (IMO this is better since it directly mirrors the actual
script's file name and can't be confused w/ a package).
Add a "broxygen" domain Sphinx extension w/ directives to allow
on-the-fly documentation to be generated w/ Bro and included in files.
This means all autogenerated reST docs are now done by Bro. The odd
CMake/Python glue scipts which used to generate some portions are now
gone. Bro and the Sphinx extension handle checking for outdated docs
themselves.
Parallel builds of `make doc` target should now work (mostly because
I don't think there's any tasks that can be done in parallel anymore).
Overall, this seems to simplify things and make the Broxygen-generated
portions of the documentation visible/traceable from the main Sphinx
source tree. The one odd thing still is that per-script documentation
is rsync'd in to a shadow copy of the Sphinx source tree within the
build dir. This is less elegant than using the new broxygen extension
to make per-script docs, but rsync is faster and simpler. Simpler as in
less code because it seems like, in the best case, I'd need to write a
custom Sphinx Builder to be able to get that to even work.
Segmentation fault caused by accessing fields with pos which is
one-based for setting SQLite field values. Fix is to simply subtract one
from pos. Discovered when trying to store HTTP traffic to a SQLite
database with the following Bro script:
event bro_init() {
local filter: Log::Filter = [
$name = "sqlite",
$path = "http",
$config = table(["tablename"] = "http_logs"),
$writer = Log::WRITER_SQLITE
];
Log::add_filter(HTTP::LOG, filter);
}
