This would usually manifest in raising truncated_IP weirds, which is
misleading because it wasn't actually an IP packet in the first place.
Now unknown_packet_type weird is raised instead.
Send protocol type to log writers - the ascii writer simply ignores this, but the input reader needs support for this.
Conflicts:
src/LogMgr.h
src/logging/Manager.cc
most stuff is inplace, logging framework needs a few changes merged before continuing here...
Conflicts:
src/CMakeLists.txt
src/LogMgr.h
src/logging/Manager.cc
src/main.cc
Internally, all BROv6 preprocessor switches were removed and
addr/subnet representations wrapped in the new IPAddr/IPPrefix classes.
Some script-layer changes of note:
- dns_AAAA_reply event signature changed: the string representation
of an IPv6 addr is easily derived from the addr value, it doesn't
need to be another parameter. This event also now generated directly
by the DNS analyzer instead of being "faked" into a dns_A_reply event.
- removed addr_to_count BIF. It used to return the host-order
count representation of IPv4 addresses only. To make it more
generic, we might later add a BIF to return a vector of counts
in order to support IPv6.
- changed the result of enclosing addr variables in vertical pipes
(e.g. |my_addr|) to return the bit-width of the address type which
is 128 for IPv6 and 32 for IPv4. It used to function the same
way as addr_to_count mentioned above.
- remove bro_has_ipv6 BIF
Sending SIGTERM triggers a normal shutdown of all threads that waits
until they have processed their remaining data. However, sending a 2nd
SIGTERM while waiting for them to finish will immediately kill them
all.
- This branch removes the attempt at bittorrent resynchronization.
I don't think that the bittorrent resynchronization would really
work very well anyway.
- This need to be merged after the topic/seth/64bit-type branch in binpac.
This is based on Gilbert's code but I ended up refactoring it quite a
bit. That's why I didn't do a direct merge but started with a new
branch and copied things over to adapt. It looks quite a bit different
now as I tried to generalize things a bit more to also support the
Input Framework.
The larger changes code are:
- Moved all logging code into subdirectory src/logging/. Code
here is in namespace "logging".
- Moved all threading code into subdirectory src/threading/. Code
here is in namespace "threading".
- Introduced a central thread manager that tracks threads and is
in charge of termination and (eventually) statistics.
- Refactored logging independent threading code into base classes
BasicThread and MsgThread. The former encapsulates all the
pthread code with simple start/stop methods and provides a
single Run() method to override.
The latter is derived from BasicThread and adds bi-directional
message passing between main and child threads. The hope is that
the Input Framework can reuse this part quite directly.
- A log writer is now split into a general WriterFrontend
(LogEmissary in Gilbert's code) and a type-specific
WriterBackend. Specific writers are implemented by deriving from
the latter. (The plugin interface is almost unchanged compared
to the 2.0 version.).
Frontend and backend communicate via MsgThread's message
passing.
- MsgThread (and thus WriterBackend) has a Heartbeat() method that
a thread can override to execute code on a regular basis. It's
triggered roughly once a second by the main thread.
- Integration into "the rest of Bro". Threads can send messages to
the reporter and do debugging output; they are hooked into the
I/O loop for sending messages back; and there's a new debugging
stream "threading" that logs, well, threading activity.
This all seems to work for the most part, but it's not done yet.
TODO list:
- Not all tests pass yet. In particular, diffs for the external
tests seem to indicate some memory problem (no crashes, just an
occasional weird character).
- Only tested in --enable-debug mode.
- Only tested on Linux.
- Needs leak check.
- Each log write is currently a single inter-thread message. Bring
Gilbert's bulk writes back.
- Code needs further cleanup.
- Document the class API.
- Document the internal structure of the logging framework.
- Check for robustness: live traffic, aborting, signals, etc.
- Add thread statistics to profile.log (most of the code is there).
- Customize the OS-visible thread names on platforms that support it.
* origin/topic/jsiwek/brofiler:
Fix superfluous/duplicate data getting in to testing coverage log.
Add "# @no-test" tag to blacklist statements from test coverage analysis.
Test coverage integration for external tests and complete suite.
Integrate Bro script coverage profiling with the btest suite.
Add simple profiling class to accumulate Stmt usage stats across runs.
Renaming environment variable BROFILER_FILE to BRO_PROFILER_FILE for
consistency. Yeah, I know, such a nice name! :)
* origin/topic/bif_cleanup:
Give mode2string a more generic name.
Change some BiF return values from bool to any.
Perform type checking on count-to-port conversion.
Remove redundant connection_record() BiF.
Remove redundant active_connection() BiF.
Make exit() parameterizable.
to_port() now parses a string instead of a count.
Closes#684.
* origin/topic/script-reference: (50 commits)
A few updates for the FAQ.
Fixing some doc warnings.
Forgot to add protocol identifier support for TLS 1.2
Finished SSL & syslog autodocs.
Adding the draft SSL extension type next_protocol_negotiation.
Fix some documentation errors.
Tweaks.
A set of script-reference polishing.
fixed a couple typos in comments
Add summary documentation to bif files.
Add ssl and syslog script documentation
Add Conn and DNS protocol script documentation. (fixes#731)
Small updates to the default local.bro.
Documentation updates for HTTP & IRC scripts.
SSH&FTP Documentation updates.
Fixing a warning from the documentation generation.
This completes framework documentation package 4.
Minor notice documentation tweaks.
Fix some malformed Broxygen xref roles.
Minor doc tweaks to init-bare.bro.
...
Conflicts:
aux/broccoli
aux/broctl
src/bro.bif
src/strings.bif
Includes:
- Updated baselines for autodoc tests.
- Now excluding stats.bro from external texts, it's not stable.
Includes:
- Quite a bit of CSS tweaking.
- Some small changes to BroDoc.cc to change the order of sections.
- A bit of tweaking some reST here and there.
