Commit graph

4477 commits

Author SHA1 Message Date
Jon Siwek
9967aea52c Integrate new Broxygen functionality into Sphinx.
Add a "broxygen" domain Sphinx extension w/ directives to allow
on-the-fly documentation to be generated w/ Bro and included in files.

This means all autogenerated reST docs are now done by Bro.  The odd
CMake/Python glue scipts which used to generate some portions are now
gone.  Bro and the Sphinx extension handle checking for outdated docs
themselves.

Parallel builds of `make doc` target should now work (mostly because
I don't think there's any tasks that can be done in parallel anymore).

Overall, this seems to simplify things and make the Broxygen-generated
portions of the documentation visible/traceable from the main Sphinx
source tree.  The one odd thing still is that per-script documentation
is rsync'd in to a shadow copy of the Sphinx source tree within the
build dir.  This is less elegant than using the new broxygen extension
to make per-script docs, but rsync is faster and simpler.  Simpler as in
less code because it seems like, in the best case, I'd need to write a
custom Sphinx Builder to be able to get that to even work.
2013-11-21 14:34:32 -06:00
Jon Siwek
4f6d01000a Implement majority of Broxygen features delegated to Bro.
Still have to update the Sphinx integration.
2013-11-14 14:00:51 -06:00
Jon Siwek
bdd359d58c Broxygen can now read a config file specifying particular targets.
Though nothing currently gets built as most dependency/outdated
checks and doc-generation methods are still skeleton code.
2013-11-05 16:40:24 -06:00
Jon Siwek
1fabbd441c Remove unneeded Broxygen comments in scan.bro.
They just duplicated the text from where the events are originally
declared and also it's not generally useful to Broxygen-style comment
event *handlers* (they're more of an implementation detail of a script,
not a user-facing element).
2013-11-05 10:54:05 -06:00
Jon Siwek
3046013d69 Replace safe_basename/safe_dirname w/ SafeBasename/SafeDirname.
So errors can be better handled.
2013-11-04 11:42:39 -06:00
Jon Siwek
96ed7aed1a Merge branch 'master' into topic/jsiwek/broxygen 2013-11-04 10:17:24 -06:00
Robin Sommer
886266e8af Merge remote-tracking branch 'origin/fastpath'
* origin/fastpath:
  Remove swig-ruby from required packages section of install doc
2013-11-03 22:53:54 -08:00
Daniel Thayer
13ff4374dd Remove swig-ruby from required packages section of install doc 2013-11-04 00:00:23 -06:00
Robin Sommer
a9d8748614 Updating submodule(s).
[nomail]
2013-11-01 05:26:32 -07:00
Robin Sommer
101fbdeff7 Adding source and original copyright statement to Mozilla cert list. 2013-11-01 05:24:43 -07:00
Robin Sommer
5f863df91f Canonfying an intel test to not depend on output order. 2013-11-01 05:03:54 -07:00
Robin Sommer
114c9a6c9c Updating submodule(s).
[nomail]
2013-11-01 04:52:43 -07:00
Robin Sommer
ae8e12c629 Updating submodule(s).
[nomail]
2013-10-31 00:20:58 -07:00
Jon Siwek
b38efa58d0 Merge branch 'master' into topic/jsiwek/broxygen
Conflicts:
	testing/btest/Baseline/coverage.bare-load-baseline/canonified_loaded_scripts.log
	testing/btest/Baseline/coverage.default-load-baseline/canonified_loaded_scripts.log
2013-10-30 16:20:48 -05:00
Jon Siwek
3a99aaaf0a Add BIF interface for retrieving comments/docs.
The new BIFs:
    - get_identifier_comments
    - get_script_comments
    - get_package_readme
    - get_record_field_comments
2013-10-30 16:07:57 -05:00
Robin Sommer
49f31acebe Merge remote-tracking branch 'origin/topic/jsiwek/thread-termination'
* origin/topic/jsiwek/thread-termination:
  Fix thread processing/termination conditions.

BIT-1093 #merged
2013-10-30 05:05:06 -07:00
Jon Siwek
22d35d2c8c Fix thread processing/termination conditions.
A thread that is done/killed should signify that the thread manager has
some processing to do -- it needs to process any messages in its out
queue, join the thread, and delete it.  Otherwise the thread manager
may reach a state where it makes no progress in processing the last
remaining done/killed thread.
2013-10-29 14:40:07 -05:00
Robin Sommer
601b1b9fd0 Merge branch 'master' of git.bro.org:bro 2013-10-29 09:31:59 -07:00
Robin Sommer
943f44314a Merge remote-tracking branch 'origin/fastpath'
* origin/fastpath:
  Don't build broccoli ruby bindings by default, use --enable-ruby to do so.
  Revert "Fix race condition in unit test."
  Fix race condition in unit test.
  Change percent_lost in capture-loss from a string to a double.
2013-10-29 09:28:13 -07:00
Jon Siwek
7c7967c1ab Don't build broccoli ruby bindings by default, use --enable-ruby to do so. 2013-10-29 11:04:34 -05:00
Jon Siwek
b2d6ccfb19 Revert "Fix race condition in unit test."
This reverts commit b255aedc26.

The test should work as it was -- actually seems to be a race in the
thread termination code.
2013-10-29 10:45:11 -05:00
Seth Hall
26be082951 Return the Dir module to file name tracking instead of inode tracking. 2013-10-29 11:09:55 -04:00
Vlad Grigorescu
e1d2f6d82f Change percent_lost in capture-loss from a string to a double. 2013-10-29 06:03:34 -07:00
Jon Siwek
b255aedc26 Fix race condition in unit test.
Removing an input stream immediately after it's created causes a race
to read the entire file before the reader gets the signal to stop.
2013-10-28 13:24:24 -05:00
Jon Siwek
8d0a74d606 Merge branch 'master' into fastpath 2013-10-28 13:18:37 -05:00
Vlad Grigorescu
31c7c1a673 Change percent_lost in capture-loss from a string to a double. 2013-10-28 08:09:16 -04:00
Robin Sommer
eff96bef37 Updating submodule(s).
[nomail]
2013-10-26 19:20:09 -07:00
Robin Sommer
7ca3f5f9ed Merge remote-tracking branch 'origin/fastpath'
* origin/fastpath:
  Revert "Fix the dir module."
  Fix the dir module.
2013-10-26 19:16:49 -07:00
Robin Sommer
64812daa50 Next version of the threading queue deadlock fix.
We now just use the read/write counters, as suggested by Gilbert.
2013-10-26 19:15:43 -07:00
Bernhard Amann
f90fcdf152 Revert "Fix the dir module."
This reverts commit 4b0ee2e7ca.

Sorry, bad idea.
2013-10-25 18:27:57 -07:00
Bernhard Amann
4b0ee2e7ca Fix the dir module.
Internally, Dir kept track of the files in directory
by storing all inode numbers in a set. However, when a file
is deleted and a new file is created in a directory, the old
file may get the same inode number as the old one. In this
case, bro did not notice the new file.

The patch simply changes the indexing of files - now files
are indexed by inode and creation time.

This should fix the scripts.base.utils.dir test failures.
2013-10-25 18:01:46 -07:00
Robin Sommer
fbdbe6d35b Merge remote-tracking branch 'origin/fastpath'
* origin/fastpath:
  Add curl to list of optional dependencies
  Update test and baseline for a recent doc test fix
2013-10-25 15:48:45 -07:00
Robin Sommer
be5b0105b0 Updating README with download/git information.
The git instructions were hard to find and this way they will show up
on github.
2013-10-25 15:06:13 -07:00
Daniel Thayer
c299a71b83 Add curl to list of optional dependencies
The curl utility is used by the active-http.bro script, but this
isn't mentioned anywhere.
2013-10-25 15:14:52 -05:00
Daniel Thayer
32d7c96cd4 Update test and baseline for a recent doc test fix 2013-10-25 15:11:20 -05:00
Robin Sommer
2818bb3dbc Merge remote-tracking branch 'origin/fastpath'
* origin/fastpath:
  make the documentation of the SQLite reader/writer a bit nicer.
  Wrong example file was included - reported by Michael Auger @LM4K
2013-10-25 11:12:51 -07:00
Robin Sommer
15e72d1422 Merge remote-tracking branch 'origin/topic/bernhard/alternative-deadlock-fix'
* origin/topic/bernhard/alternative-deadlock-fix:
  alternative deadlock issue fix.
2013-10-25 11:10:08 -07:00
Bernhard Amann
a09d8e94e0 make the documentation of the SQLite reader/writer a bit nicer. 2013-10-25 10:21:33 -07:00
Bernhard Amann
0fdbdff3c4 Wrong example file was included - reported by Michael Auger @LM4K 2013-10-25 09:57:12 -07:00
Bernhard Amann
2ac0d77f06 alternative deadlock issue fix.
This fix also fixes the deadlock issue without putting any
new strain into the main packet processing path.

Instead of occasionally returning true in MaybeReady sometime,
we occasionally process threads if time_mgr time is not running.

If time_mgr time is running, we have heartbeat messages that will
trigger processing in any case -- processing always checks the
exact state of the Queues.

This fix probably also means that we can remove the communication
loads from all input framework tests and run them all simultaneously.
2013-10-24 23:39:16 -07:00
Robin Sommer
c980d1055e Fix for input readers occasionally dead-locking.
Bernhard and I tracked it down we believe: the thread queue could
deadlock in certain cases. As a fix we tuned the heuristic for telling
if a queue might have input to occasionaly err on the safe side by
flagging "yes", so that processing will proceed.

It's a bit unfortunate to apply this fix last minute before the
release as it could potentially impact performance if the heuristic
fails to often. We believe the chosen parmaterization should be fine ...
2013-10-24 18:21:24 -07:00
Robin Sommer
42c4a51da3 Updating submodule(s).
[nomail]
2013-10-24 16:59:31 -07:00
Robin Sommer
5c80fd169f Merge remote-tracking branch 'origin/fastpath'
* origin/fastpath:
  change temporary workaround to permanent fix.

Conflicts:
	src/input/readers/Ascii.cc
2013-10-24 16:33:11 -07:00
Bernhard Amann
9a58a28da0 change temporary workaround to permanent fix.
According to Howard Hinnant, in use cases like ours you
need to call file->sync() to synchronize the internal ifstream
buffer with the file.

See http://stackoverflow.com/questions/19558376/clang-3-3-xcode-libc-stdgetline-does-not-read-data-after-calling-ifstream
2013-10-24 15:46:16 -07:00
Robin Sommer
a7cebff99d Merge remote-tracking branch 'origin/topic/dnthayer/doc-changes-for-2.2'
* origin/topic/dnthayer/doc-changes-for-2.2:
  Add gawk to list of optional packages
  Add more script package README files
  Add NEWS about new features of broctl and upgrade info
2013-10-24 14:35:21 -07:00
Robin Sommer
8ce3865bd2 Updating baselines for packet filter changes. 2013-10-24 14:31:23 -07:00
Robin Sommer
1f91b2936e Merge remote-tracking branch 'origin/fastpath'
* origin/fastpath:
  rework libc++ patch to use AvailabilityMacros.h instead of Availability.h

Conflicts:
	src/input/readers/Ascii.cc
2013-10-24 14:01:44 -07:00
Robin Sommer
34ba97c1e1 Merge branch 'master' of git.bro.org:bro 2013-10-24 14:00:34 -07:00
Bernhard Amann
c640e29193 rework libc++ patch to use AvailabilityMacros.h instead of Availability.h
AvailabilityMacros.h was introduced in 10.2, Availability is only available in
10.5+ - this increases backwards compatibility.
2013-10-24 13:34:17 -07:00
Seth Hall
ba93d0df08 Intel framework notes added to NEWS 2013-10-24 16:28:10 -04:00