This updates the "lookup_location" and "lookup_asn" BIFs to use
libmaxminddb. The motivation for this is that MaxMind is discontinuing
GeoLite Legacy databases: no updates after April 1, 2018, no downloads
after January 2, 2019. It's also noted that all GeoIP Legacy databases
may be discontinued as they are superseded by GeoIP2.
* 'topic/dopheide/broker-docs' of https://github.com/dopheide-esnet/bro:
A suggestion for reminding folks about calling events in Module namespaces.
I've changed this to include more accurate examples
Includes slight editing of the new text.
Closes BIT-1905.
* origin/topic/dnthayer/ticket1905:
Add documentation of the configuration framework
Add documentation of "option" declarations
Improve config framework documentation comments
Fix minor typos and doc build warnings in NEWS
Great work, and great documentation!
I'm getting one test failure with
scripts.base.frameworks.netcontrol.catch-and-release-cluster Going
ahead and commiting, Jenkins will show the details I assume.
BIT-1584 #merged
* origin/topic/johanna/netcontrol-improvements:
SMTP does not need to pull in the notice framework.
Write NetControl framework documentation.
Use NetControl for ACTION_DROP of notice framework.
NetControl: slightly update catch and release logging
NetControl: fix several small logging issues
NetControl: more catch and release logging and cluster fix
NetControl: rewrite catch and release and small fixes.
NetControl: find_rules_subnet works in cluster mode
NetControl: fix acld whitelist command
NetControl: add rule exists as state besides added and failure.
NetControl: Suppress duplicate "plugin activated" messages.
NetControl: make new broker plugin options accessible
NetControl: add predicates to broker plugin
Fixed more typos, reformatted the code examples to remove the
horizontal scroll bars, and removed some redundant sections that were
just outdated copies of information in the auto-generated reference
docs.
Updated the install section for FreeBSD and OS X.
Added a section to explain how to quickly test that everything is
setup correctly.
Improved the usage section by removing the misleading record definition
(a link to the reference doc is provided), and explaining that some
fields will be uninitialized.
Corrected the example so that it doesn't try to access uninitialized
fields.
Lots of good stuff! Thanks for catchign the plugin doc inconsistencies!
* origin/topic/dnthayer/doc-improvements-2.4:
Add missing documentation on the "Bro Package Index" page
More improvements to the Logging Framework doc
Fix documentation typo
Update the "Log Files" documentation
Add links in the logging framework doc
Add a link to the bro-plugins documentation
Update bro man page
Update script language reference documentation
Fix typos in the "writing bro plugins" doc
Fix a "make doc" warning
Improve logging framework doc
Add link to broctl doc from the quickstart doc
Update install documentation and fix some typos
Minor improvements to logging framework documentation
Correct a minor typo in the docs
Reorganized the content to be easier to follow, added a few more examples,
fixed some ugly formatting (removed scrollbars that make the examples
difficult to read).
Removed "file_mime_type" and "file_mime_types" event, replacing them
with a new event called "file_metadata_inferred". It has a record
argument of type "inferred_file_metadata", which contains the mime type
information that the earlier events used to supply. The idea here is
that future extensions to the record with new metadata will be less
likely to break user code than the alternatives (adding new events or
new event parameters).
Addresses BIT-1368.
- Some scripts used wrong SSH module/namespace scoping on events.
- Fix outdated notice documentation related to SSH password guessing.
- Add a unit test for SSH pasword guessing notice.
I added the $path to the create_stream() calls inside doc/ as well.
* origin/topic/jsiwek/bit-1324:
Allow logging filters to inherit default path from stream.
BIT-1324: #merged
Notable changes:
- libmagic is no longer used at all. All MIME type detection is
done through new Bro signatures, and there's no longer a means to get
verbose file type descriptions (e.g. "PNG image data, 1435 x 170").
The majority of the default file magic signatures are derived
from the default magic database of libmagic ~5.17.
- File magic signatures consist of two new constructs in the
signature rule parsing grammar: "file-magic" gives a regular
expression to match against, and "file-mime" gives the MIME type
string of content that matches the magic and an optional strength
value for the match.
- Modified signature/rule syntax for identifiers: they can no longer
start with a '-', which made for ambiguous syntax when doing negative
strength values in "file-mime". Also brought syntax for Bro script
identifiers in line with reality (they can't start with numbers or
include '-' at all).
- A new Built-In Function, "file_magic", can be used to get all
file magic matches and their corresponding strength against a given
chunk of data
- The second parameter of the "identify_data" Built-In Function
can no longer be used to get verbose file type descriptions, though it
can still be used to get the strongest matching file magic signature.
- The "file_transferred" event's "descr" parameter no longer
contains verbose file type descriptions.
- The BROMAGIC environment variable no longer changes any behavior
in Bro as magic databases are no longer used/installed.
- Reverted back to minimum requirement of CMake 2.6.3 from 2.8.0
(it's back to being the same requirement as the Bro v2.2 release).
The bump was to accomodate building libmagic as an external project,
which is no longer needed.
Addresses BIT-1143.
- Move notice index wrapper doc to doc/script-reference -- doc/scripts
no longer contains any static documentation because that location
will be managed by Bro to generate per-script docs.
- :doc: references for generated per-script docs now need the ".bro"
suffix. (IMO this is better since it directly mirrors the actual
script's file name and can't be confused w/ a package).
Add a "broxygen" domain Sphinx extension w/ directives to allow
on-the-fly documentation to be generated w/ Bro and included in files.
This means all autogenerated reST docs are now done by Bro. The odd
CMake/Python glue scipts which used to generate some portions are now
gone. Bro and the Sphinx extension handle checking for outdated docs
themselves.
Parallel builds of `make doc` target should now work (mostly because
I don't think there's any tasks that can be done in parallel anymore).
Overall, this seems to simplify things and make the Broxygen-generated
portions of the documentation visible/traceable from the main Sphinx
source tree. The one odd thing still is that per-script documentation
is rsync'd in to a shadow copy of the Sphinx source tree within the
build dir. This is less elegant than using the new broxygen extension
to make per-script docs, but rsync is faster and simpler. Simpler as in
less code because it seems like, in the best case, I'd need to write a
custom Sphinx Builder to be able to get that to even work.
