* origin/topic/seth/ie11-software-parsing:
Updated software framework to support parsing IE11 user-agent strings.
Fix the irc_reply event for several server message types.
BIT-1104 #merged
Changes:
- Changing semantics of the new_event() meta event: it's raised
only for events that have a handler defined. There are too many
checks in Bro that prevent events wo/ handler from being even
prepared to raise to do that differently.
- Adding test case.
* topic/robin/event-dumper:
New script misc/dump-events.bro, along with core support, that dumps events Bro is raising in an easily readable form.
Prettyfing Describe() for record types.
- Enable manager to associate analyzers with a MIME type. With that,
one can now say enable all analyzers for, e.g., "image/gif". This is
exposed to script-land as
Files::add_analyzers_for_mime_type(f: fa_file, mtype: string)
For MIME types identified via libmagic, this happens automatically
(via the file_new() handler in files/main.bro).
- Extend the analyzer API to better match that of protocol analyzers:
- Adding unique analyzer IDs so that we can refer to instances
from script-land.
- Adding subtypes to Components so that a single analyzer
implementation can support different types of analyzers
internally.
- Add an analyzer method SetTag() that allows to set the tag after
construction.
- Adding Init() and Done() methods for consistency with what other
classes offer.
- Add debug logging to the file_analysis stream.
TODO: test cases missing for the new script-land functionality.
in an easily readable form.
This is for debugging purposes, obviously.
Example, including only SMTP events:
> bro -r smtp.trace misc/dump-events.bro DumpEvents::include=/smtp/
[...]
1254722768.219663 smtp_reply
[0] c: connection = [id=[orig_h=10.10.1.4, orig_p=1470/tcp, resp_h=74.53.140.153, [...]
[1] is_orig: bool = F
[2] code: count = 220
[3] cmd: string = >
[4] msg: string = xc90.websitewelcome.com ESMTP Exim 4.69 #1 Mon, 05 Oct 2009 01:05:54 -0500
[5] cont_resp: bool = T
1254722768.219663 smtp_reply
[0] c: connection = [id=[orig_h=10.10.1.4, orig_p=1470/tcp, resp_h=74.53.140.153, [...]
[1] is_orig: bool = F
[2] code: count = 220
[3] cmd: string = >
[4] msg: string = We do not authorize the use of this system to transport unsolicited,
[5] cont_resp: bool = T
[...]
Add a "broxygen" domain Sphinx extension w/ directives to allow
on-the-fly documentation to be generated w/ Bro and included in files.
This means all autogenerated reST docs are now done by Bro. The odd
CMake/Python glue scipts which used to generate some portions are now
gone. Bro and the Sphinx extension handle checking for outdated docs
themselves.
Parallel builds of `make doc` target should now work (mostly because
I don't think there's any tasks that can be done in parallel anymore).
Overall, this seems to simplify things and make the Broxygen-generated
portions of the documentation visible/traceable from the main Sphinx
source tree. The one odd thing still is that per-script documentation
is rsync'd in to a shadow copy of the Sphinx source tree within the
build dir. This is less elegant than using the new broxygen extension
to make per-script docs, but rsync is faster and simpler. Simpler as in
less code because it seems like, in the best case, I'd need to write a
custom Sphinx Builder to be able to get that to even work.
* origin/topic/dnthayer/doc-changes-for-2.2:
Add gawk to list of optional packages
Add more script package README files
Add NEWS about new features of broctl and upgrade info
Some documentation of commented-out items was being output to HTML,
which was causing that documentation to be merged with the
documentation of the next item. Fixed by changing "##" to "#" so
that the doc comments are not included in the HTML.
Snapshotting the work in this branch. I'll merge it again later as we
get closer to the release.
* origin/topic/dnthayer/doc-changes-for-2.2: (29 commits)
Add README files for base/protocols
Fix incorrect uses of reST directives
Fix typos and formatting in the BiFs docs
Fix typos and formatting in the base/utils docs
Fix typos and formatting in the other protocol docs
Fix typos and formatting in the ssl protocol docs
Fix typos and formatting in the http protocol docs
Fix typos and formatting in the ftp protocol docs
Fix typos and formatting in the dns protocol docs
Fix typos and formatting in the dhcp protocol docs
Adjust line numbers to match changes in conn/main.bro
Fix typos and formatting in the conn protocol docs
Update FreeBSD install instructions
Improvements to file analysis docs
Add README files for most Bro frameworks
Fix typos and formatting in various other framework docs
Fix typos and formatting in the software framework docs
Fix typos and formatting in the sumstats docs
Fix typos and formatting in the packet filter docs
Fix typos and formatting in the logging framework docs
...
Fixed reference to wrong field name.
Added documentation of a function arg.
Added a couple references to other parts of the documentation.
Explained how not specifying extraction filename results in automatic
filename generation.
Several other minor clarifications.
The text from these README files appears on the "Bro Script Packages"
page after building the documentation. The text for these was mostly just
copied from the existing docs.
