The magic files couldn't be in the root of that repo or else
libmagic would abort when it ran in to the .git* files and tried
to treat them like magic files, too.
- It's derived from the magic database of libmagic 5.14, but with most
everything not related to mime types removed.
- The custom database is always used by default for mime detection, but
the more verbose file type detection will fall back on the default
libmagic installation's database. The result is: mime type strings
are now guaranteed to be consistent across platforms, but the verbose
file type descriptions are not.
- The custom database gets installed in $prefix/share/bro/magic, and
should even be extensible if files with new patterns are added inside
the directory.
- The search path for the mime magic database can be controlled via
BROMAGIC environment variable.
- Remove mime_desc field from ftp.log.
- Stop using the mime/file type canonifier with unit tests.
- libmagic >= 5.04 is now a requirement.
--enable-perftools must now explicity be supplied to ./configure on
non-Linux systems to link against the tcmalloc library that a
gperftools installation provides. Linux systems still automatically
link it if it's found. The rationale is that gperftools was developed
and most throroughly tested on Linux so it's safer there.
There especially seems to be potential problems with gperftools on OS X
(e.g. see http://code.google.com/p/gperftools/issues/detail?id=413),
and Bro currently doesn't work with gpertools there using clang or gcc.
I've only tested that it compiles, not whether it still works. The
fact that we don't have any tests for this makes me uneasy ...
* remotes/origin/topic/seth/elasticsearch: (35 commits)
Some documentation updates for elasticsearch plugin.
Temporarily removing the ES timeout because it works with signals and is incompatible with Bro threads.
Changed ES index names to localtime and added a meta index.
New script for easily duplicating logs to ElasticSearch.
Some better elasticsearch reliability.
Fixed small elasticsearch problem in configure output.
Re-adding the needed call to FinishedRotation in the ES writer plugin.
Tiny updates.
Bringing elasticsearch branch up to date with master.
Adding a define to make the stdint C macros available.
Adding an extra header.
Fixed a bug with messed up time value passing to elasticsearch.
Small updates and a little standardization for config.h.in naming.
Bug fixes.
Bug fix and feature.
Forgot to call the parent method for DoHeartBeat.
Changed the escaping method.
Flush logs to ES daemon as Bro is shutting down.
Reduce the batch size to 1000 and add a maximum time interval for batches.
Reworked bulk operation string construction to use ODesc and added json escaping.
...
The variable was never "used" in the case google perftools was not
found, so CMake warned about it being unused since it was manually
specified as a cache variable in the configure script. There might
be a better way to organize that, but this is a quick/easy hack.
I copied the code over manually, no merging, because (1) it needed to
be adapted to the new threading API, and (2) there's more stuff in the
branch that I haven't ported yet.
The DS output generally seems to work, but it has seen no further
testing yet.
Not unit tests yet either.
Also renaming --enable-perftools to --enable-perftool-debug to
indicate that the switch is only relevant for debugging the heap. It's
not needed to pick up tcmalloc for better performance.
--with-perftools can still (and always) be used to give a hint where
to find the libraries.
With the threading, using tcmalloc improves memory usage on FreeBSD
significantly when running on a trace. If it fixes the live problems,
remains to be seen ...
Note, I didn't see the topic/jsiwek/cmake-rpath branch in the Bro
repository so I copied the change from the submodules, assuming it
applies in the same way.
* origin/topic/jsiwek/autodoc-fixes:
Update doc sources and touch up a few script comments.
Fixes for script auto-documentation.
Conflicts:
scripts/base/frameworks/logging/main.bro
- Fixing the parts of the `make restdoc` and `make doc` process that were
broken by the last Bro script re-organization
- Generated documentation for Bro scripts derived from BiFs now use the
original BiF source file as the "original source file" link
- Renaming of the internal POLICYDEST definition and other misc places that
refer to "policy" scripts; that terminology doesn't make total sense now
- Added a documentation blacklist reminder test that will fail if there's
scripts that are blacklisted from being documentated because they're still
in progress
- Some minor Bro script changes to fix small @load dependency errors
Addresses #543
- policy/ renamed to scripts/
- By default BROPATH now contains:
- scripts/
- scripts/policy
- scripts/site
- *Nearly* all tests pass.
- All of scripts/base/ is loaded by main.cc
- Can be disabled by setting $BRO_NO_BASE_SCRIPTS
- Scripts in scripts/base/ don't use relative path loading to ease use of BRO_NO_BASE_SCRIPTS (to copy and paste that script).
- The scripts in scripts/base/protocols/ only (or soon will only) do logging and state building.
- The scripts in scripts/base/frameworks/ add functionality without causing any additional overhead.
- All "detection" activity happens through scripts in scripts/policy/.
- Communications framework modified temporarily to need an environment variable to actually enable (ENABLE_COMMUNICATION=1)
- This is so the communications framework can be loaded as part
of the base without causing trouble when it's not needed.
- This will be removed once a resolution to ticket #540 is reached.
Adds two extra shell scripts, one to handle either the c-shell
or bourne-shell syntax for setting environment variables, that can
be sourced in order to set the BROPATH directly in the current
shell/environment in a way that's compatible with running bro
from the build/ directory.
- pre/post install scripts now track configuration files that may
be clobbered on package install/upgrade through the
INSTALLED_CONFIG_FILES CMake variable and attempts to make backups
when the distribution's file differs from the existing file.
- Refactored all packaging related stuff into a single CMake module
- Build should no longer fail when optional sources (e.g. broctl)
do not exist in the source directory, instead a warning is issued
- Additional configure options to change packaging behavior
Made order of --enable-broctl and --enable-cluster not matter.
--enable-cluster always enables Broctl in cluster configuration even if
the other option is used.
--enable-broctl will enable Broctl in standalone configuration only if
--enable-cluster isn't present.
To POLICYDIR since that's more clear. Also decided that restricting
it to a subdirectory of prefix isn't much use as an option, so went
back to having it be an absolute path.
