* origin/topic/jsiwek/openssl-1.1:
Update install instructions for OpenSSL 1.1 compat
Remove requestorName parameter of ocsp_request event
Adjust x509 unit tests to work around OpenSSL 1.0 vs. 1.1 differences
Fixes for OpenSSL 1.1 support
This updates the "lookup_location" and "lookup_asn" BIFs to use
libmaxminddb. The motivation for this is that MaxMind is discontinuing
GeoLite Legacy databases: no updates after April 1, 2018, no downloads
after January 2, 2019. It's also noted that all GeoIP Legacy databases
may be discontinued as they are superseded by GeoIP2.
As opposed to plugins depending on a Bro source/build tree. This
required installing various Bro headers, BinPAC and it's headers,
bifcl, and Bro's custom CMake modules.
It turns out that --with-openssl does not currently work because even
when the correct library is chosen, the include path ordering will
(typically) cause the system OpenSSL to be included.
Push OpenSSL header directory as the highest priority one. This probably
also makes sense since OpenSSL is/will be one of the most common
libraries that needs specific include directories.
This just provides a convient way of indicating that ccache should
be used as compiler-wrapper during builds. e.g. when I want dev/debug
builds that (re)compile quickly, I do:
./configure --build-type=debug --generator=Ninja --ccache
time.
People keep running into the problem that they upgrade Bro but forget
to recompile their plugins--which can lead to crashes. While the
plugins' API version was supposed to catch this, it's not reliable as
that check may come too late. This change takes a different tack: We
compile a C function into the Bro binary that has Bro's version number
encoded into its name. A plugin can then reference that function. If
the Bro version changes, the function goes away and the plugin won't
load anymore.
I've integrated that function reference into the plugin skeleton code
so that new plugins get it automatically (unless explicitly removed).
I couldn't see a way to do it transparently for already existing
plugins unfortunately.
The version number used for the function name is slightly normalized
to skip any git revision postfixes (i.e., "2.5-xxx" is always treated
as "2.5-git") so that one doesn't need to recompile all plugins after
every master commit. That seems good enough, usually people run into
this when upgrading to a new release.
If one loads an old plugin into a new Bro, the error message looks
like this:
$ bro -NN Demo::Foo
fatal error in /home/robin/bro/master/scripts/base/init-bare.bro, line 1:
cannot load plugin library /home/robin/tmp/p/build//lib/Demo-Foo.linux-x86_64.so:
/home/robin/tmp/p/build//lib/Demo-Foo.linux-x86_64.so: undefined symbol: bro_version_2_5_git_debug
Not the prettiest, but better than a crash!
TODO: I'm still unsure if we should remove the plugin API version
altogetger now. This link-time check should catch everything the API
version does, except for master commits.
I reworked this a bit:
- Moved the globals into a new Pcap::* namespace, and renamed them
slightly.
- Moved the definitions of the globals into pcap/const.bif.
- Also moved the existing 'snaplen' into Pcap::* and removed
SnapLen() from the PktSrc API (it's really a pcap thing).
- Likewise moved the existing functions precompile_pcap_filter,
install_pcap_filter, and pcap_error, into Pcap::*.
- Did some more refactoring for the pcap code.
* 'master' of https://github.com/knielander/bro:
Refactored patch (removed options, less ambiguous name)
Allow Bro to run in fanout mode.
Allow libpcap buffer size to be set manually.
Allow Bro to run in fanout mode.
Allowed libpcap buffer size to be set via configuration.
The new --enable-broker flag can be used to toggle the use of Broker,
which also implies building with -std=c++11, though nothing makes
use of these features at the moment.
* origin/topic/robin/reader-writer-plugins:
Adding plugin testing to Makefile's test-all.
Fixing tests.
Removing DataSeries and ElasticSearch from configure script.
Updating submodule.
Move DataSeries and ElasticSearch into plugins.
More polishing of some of the branche's changes.
More polishing.
Test case for a dynamic input reader.
Test case for a dynamic log writer.
Final fixes, preparing for merge.
Converting log writers and input readers to plugins.
BIT-1222 #merged
Notable changes:
- libmagic is no longer used at all. All MIME type detection is
done through new Bro signatures, and there's no longer a means to get
verbose file type descriptions (e.g. "PNG image data, 1435 x 170").
The majority of the default file magic signatures are derived
from the default magic database of libmagic ~5.17.
- File magic signatures consist of two new constructs in the
signature rule parsing grammar: "file-magic" gives a regular
expression to match against, and "file-mime" gives the MIME type
string of content that matches the magic and an optional strength
value for the match.
- Modified signature/rule syntax for identifiers: they can no longer
start with a '-', which made for ambiguous syntax when doing negative
strength values in "file-mime". Also brought syntax for Bro script
identifiers in line with reality (they can't start with numbers or
include '-' at all).
- A new Built-In Function, "file_magic", can be used to get all
file magic matches and their corresponding strength against a given
chunk of data
- The second parameter of the "identify_data" Built-In Function
can no longer be used to get verbose file type descriptions, though it
can still be used to get the strongest matching file magic signature.
- The "file_transferred" event's "descr" parameter no longer
contains verbose file type descriptions.
- The BROMAGIC environment variable no longer changes any behavior
in Bro as magic databases are no longer used/installed.
- Reverted back to minimum requirement of CMake 2.6.3 from 2.8.0
(it's back to being the same requirement as the Bro v2.2 release).
The bump was to accomodate building libmagic as an external project,
which is no longer needed.
Addresses BIT-1143.
The Plugin.cc file is now just a standard class, with the interface
changed a bit to make it more easy to write. However, there're still
some conventions that one must follow to make everything work (like
using the right namespace structure).
This commit also includes the option to compile built-in plugins
dynamically instead of statically by adding
SET(BRO_PLUGIN_BUILD_DYNAMIC TRUE) to their CMake config. This hasn't
been tested much yet, and I'm still undecided if it's somethign we
would want to do by default--but we could now if wanted. :)
Also some minor other cleanup of plugin APIs and built infrastructure.
All tested on MacOS only currently.
