- Clarify "tunnel_changed" event documentation.
- Make expiration of "Tunnel::active" elements configuration via
"Tunnel::expiration_interval".
- Remove redundant registration of a connection's tunnels in
tunnel/main.bro's "tunnel_changed" handler.
- Rename "parents" field of "Conn::Info" to "tunnel_parents"
to give more context.
- Fixed bug with how data is sent to elasticsearch.
- Added a feature to only allow data of a certain
size to be buffered before sending to the
elasticsearch server. Configured with the
LogElasticSearch::max_byte_size variable.
Merge remote-tracking branch 'origin/topic/tunnels' into topic/robin/tunnels-merge
* origin/topic/tunnels: (41 commits)
Extend weird names that occur in core packet processing during decapsulation.
Add Teredo analysis option to reduce false positive decapsulation.
Just some cleanup/documentation of new tunnel-handling code.
Memory leak fixes
Add a config.h definition for IPPROTO_IPV4.
Add AYIYA tunnel decapsulation unit test.
Add Teredo-specific events.
Refactor some of the NetSessions routines that recurse on IP packets.
Add independent options to toggle the different decapsulation methods
Add more sanity checks before recursing on encapsulated IP packets.
Suppress Teredo weirds unless decapsulation was successful once before.
Tunnel support performance optimization.
Add Teredo tunnel decapsulation.
Fix for IP tunnel UID persistence.
Fix AYIYA analyzer tag.
Add summary documentation to tunnels/main.bro.
Make tunnels always identifiable by UID, tunnel.log now gets populated.
Some improvements to the AYIYA analyzer.
Remove Tunnel::decapsulate_ip option.
Remove invalid IP-in-IP encapsulated protocol value.
...
The Tunnel::yielding_teredo_decapsulation (on by default) makes it so
the Teredo analyzer doesn't attempt to decapsulate payloads when
there's already a sibling analyzer that thinks it's parsing the right
protocol. Sometimes, UDP payloads just happen to look like they are
validly Teredo-encapsulated and doing further analysis on the
decapsulated packet can quickly turn into a weird; this change helps
reduce such weirds.
These are called "teredo_packet", "teredo_authentication",
"teredo_origin_indication", and "teredo_bubble" and may be raised
on a per-packet basis depending on Teredo encapsulation method.
* topic/robin/input-threads-merge: (130 commits)
And now it even compiles after my earlier changes.
A set of input framework refactoring, cleanup, and polishing.
another small memory leak in ascii reader:
and another small memory leak when using streaming reads.
fix another memory lead (when updating tables).
Input framework merge in progress.
filters have been called streams for eternity. And I always was too lazy to change it everywhere...
reactivate network_time check in threading manager. previously this line made all input framework tests fail - it works now. Some of the other recent changes of the threading manager must have fixed that problem.
fix up the executeraw test - now it works for the first time and does not always fail
baselines for the autostart removal.
remove last remnants of autostart, which has been removed for quite a while.
make input framework source (hopefully) adhere to the usual indentation style. No functional changes.
fix two memory leaks which occured when one used filters.
update description to current interface.
rename a couple of structures and make the names in manager fit the api more.
fix memory leak in tables and vectors that are read into tables
fix missing get call for heart beat in benchmark reader.
fix heart_beat_interval -- initialization in constructor does not work anymore (probably due to change in init ordering?)
fix memory leak for tables... nearly completely.
fix a couple more leaks. But - still leaking quite a lot with tables.
...
Also fix header truncation check for IPv6 No Next header and add an
"ipv6_no_next" weird for such packets that aren't tunneled over Teredo
(which it calls "bubbles" and are used to create mappings in NATs).
* origin/topic/jsiwek/ipv6-comm:
Enable Bro to communicate with peers over non-global IPv6 addresses.
Add unit tests for Broccoli SSL and Broccoli IPv6 connectivity.
Remove AI_ADDRCONFIG getaddrinfo hints flag for listening sockets.
Undo communication protocol version bump.
Add support to Bro for connecting with peers over IPv6.
Closes#820.
Conflicts:
src/bro.bif
Without this change, flow labeling of connections over IPv6 are
only available in the per-packet types of events (e.g. new_packet)
in which header fields can be inspected, but now minimal tracking
of the most recent flow label is done internally and that's available
per-connection for all events that use connection record arguments.
Specifically, this adds a "flow_label" field to the "endpoint" record
type, which is used for both the "orig" and "resp" fields of
"connection" records. The new "connection_flow_label_changed" event
also allows tracking of changes in flow labels: it's raised each time
one direction of the connection starts using a different label.
* origin/topic/robin/dataseries:
Moving trace for rotation test into traces directory.
Fixing a rotation race condition at termination.
Portability fixes.
Extending DS docs with some examples.
Updating doc.
Fixing pack_scale and time-as-int.
Adding format specifier to DS spec to print out double as %.6f.
DataSeries updates and fixes.
DataSeries tuning.
Tweaking DataSeries support.
Extending log post-processor call to include the name of the writer.
Removing an unnecessary const cast.
DataSeries TODO list with open issues/questions.
Starting DataSeries HowTo.
Additional test output canonification for ds2txt's timestamps.
In threads, an internal error now immediately aborts.
DataSeries cleanup.
Working on DataSeries support.
Merging in DataSeries support from topic/gilbert/logging.
Fixing threads' DoFinish() method.
This usually requires specifying an additional zone identifier
(see RFC 4007). The connect() and listen() BIFs have been
changed to accept this zone identifier as an argument.
- Communication::listen_ipv6 needs to be redef'd to true in order
for IPv6 listening sockets to be opened.
- Added Communication::listen_retry option as an interval at which
to retry binding to socket addresses that were already in use.
- Added some explicit baselines to check in the istate.events
and istate.events-ssl tests -- the SSL test was incorrectly
passing because it compared two empty files. (The files being
empty because "http/base" was given as an argument to Bro which
it couldn't handle because that script doesn't exist anymore).
Instead, the `addr_to_uri` script-level function can be used to
explicitly add brackets to an address if it's IPv6 and will be
included in a URI or when a ":<port>" needs to be appended to it.
* origin/fastpath:
Add two more TLS extension values that we see in live traffic.
Fixed IPv6 link local unicast CIDR and added IPv6 loopback to private address space.
Fixed a problem where cluster workers were still processing notices in some cases.
Added Carrier Grade NAT CIDR and link local IPv6 to "private_address_space"
- Unserializing files that were previously kicked out of the open-file
cache would cause them to be fopen'd with the original access
permissions which is usually 'w' and causes truncation. They
are now opened in 'a' mode. (addresses #780)
- Add 'max_files_in_cache' script option to manually set the maximum
amount of opened files to keep cached. Mainly this just helped
to create a simple test case for the above change.
- Remove unused NO_HAVE_SETRLIMIT preprocessor switch.
- On systems that don't enforce a limit on number of files opened for
the process, raise default max size of open-file cache from
32 to 512.
- SSL related files and classes renamed to remove the "binpac" term.
- A small fix for DPD scripts to make the DPD log more helpful if
there are multiple continued failures. Also, fixed the SSL
analyzer to make it stop doing repeated violation messages for
some handshake failures.
- Added a $issuer_subject to the SSL log.
- Created a basic test for SSL.
