diff --git a/tools/binpac/README b/tools/binpac/README new file mode 100644 index 0000000000..f87c08addb --- /dev/null +++ b/tools/binpac/README @@ -0,0 +1,1115 @@ +.. -*- mode: rst-mode -*- + +====== +BinPAC +====== + +BinPAC is a high level language for describing protocol parsers and +generates C++ code. It is currently maintained and distributed with the +Zeek Network Security Monitor distribution, however, the generated parsers +may be used with other programs besides Zeek. + +BinPAC originally existed as a separate project to the main Zeek repository. +You can see the archived repository at https://github.com/zeek/binpac. The +repository only exists for historical reasons, as all new work done to +BinPAC is done in the main Zeek repo. + +.. contents:: + +Prerequisites +============= + +BinPAC relies on the following libraries and tools, which need to be +installed before you begin: + + * Flex (Fast Lexical Analyzer) + Flex is already installed on most systems, so with luck you can + skip having to install it yourself. + + * Bison (GNU Parser Generator) + Bison is also already installed on many system. + + * CMake 3.15.0 or greater + CMake is a cross-platform, open-source build system, typically + not installed by default. See http://www.cmake.org for more + information regarding CMake and the installation steps below for + how to use it to build this distribution. CMake generates native + Makefiles that depend on GNU Make by default + +Glossary and Convention +======================= + +To make this document easier to read, the following are the glossary +and convention used. + + - PAC grammar - .pac file written by user. + - PAC source - _pac.cc file generated by binpac + - PAC header - _pac.h file generated by binpac + - Analyzer - Protocol decoder generated by compiling PAC grammar + - Field - a member of a record + - Primary field - member of a record as direct result of parsing + - Derivative field - member of a record evaluated through post processing + +BinPAC Language Reference +========================= + +BinPAC language consists of: + + - analyzer + - type - data structure like definition describing parsing unit. Types can built on each other to form more complex type similar to yacc productions. + - flow - "flow" defines how data will be fed into the analyzer and the top level parsing unit. + - Keywords + - Built-in macros + +Defining an analyzer +-------------------- + +There are two components to an analyzer definition: the top level context +and the connection definition. + + +Context Definition +~~~~~~~~~~~~~~~~~~ + +Each analyzer requires a top level context defined by the following syntax: + +.. code:: + + analyzer withcontext { + ... context members ... + } + +Typically top level context contains pointer to top level analyzer +and connection definition like below: + +.. code:: + + analyzer HTTP withcontext { + connection : HTTP_analyzer; + flow : HTTP_flow; + }; + + +Connection Definition +~~~~~~~~~~~~~~~~~~~~~ + +A "connection" defines the entry point into the analyzer. It consists of +two "flow" definitions, an "upflow" and a "downflow". + +.. code:: + + connection (optional parameter) { + upflow = ; + downflow = ; + } + +Example: + +.. code:: + + connection HTTP_analyzer { + upflow = HTTP_flow (true); + downflow = HTTP_flow (false); + }; + +type +---- + +A "type" is the basic building block of binpac-generated parser, and describes +the structure of a byte segment. Each non-primitive "type" generates a C++ +class that can independently parse the structure which it describes. + +Syntax: + +.. code:: + + type {()} = { + cases or members declaration. + } ; + +Example: + +PAC grammar:: + + type myType = record { + data:uint8; + }; + +PAC header:: + + class myType{ + public: + myType(); + ~myType(); + int Parse(const_byteptr const t_begin_of_data, const_byteptr const t_end_of_data); + uint8 data() const { return data_; } + protected: + uint8 data_; + }; + + +Primitives +~~~~~~~~~~ + +Primitive type can be treated as #define in C language. They are embedded +into other type which reference them but do not generate any parsing +code of their own. Available primitive types are: + + - int8 + - int16 + - int32 + - uint8 + - uint16 + - uint32 + - Regular expression ( ``type HTTP_URI = RE/[[:alnum:][:punct:]]+/;`` ) + - bytestring + +Examples: + +.. code:: + + type foo = record { x: number; }; + +is equivalent to: + +.. code:: + + type foo = record { x: uint8[3]; }; + +(Note: this behavior may change in future versions of binpac.) + +record +~~~~~~ + +A "record" composes primitive type(s) and other record(s) to create +new "type". This new "type" in turn can be used as part of parent type +or directly for parsing. + +Example: + +.. code:: + + type SMB_body = record { + word_count : uint8; + parameter_words : uint16[word_count]; + byte_count : uint16; + } + +case +~~~~ + +The "case" compositor allows switching between different parsing methods. + +.. code:: + + type SMB_string(unicode: bool, offset: int) = case unicode of { + true -> u: SMB_unicode_string(offset); + false -> a: SMB_ascii_string; + }; + +A "case" supports an optional "default" label to denote none of the +above labels are matched. If no fields follow a given label, a user +can specify an arbitrary field name with the "empty" type. See +the following example. + +.. code:: + + type HTTP_Message(expect_body: ExpectBody) = record { + headers: HTTP_Headers; + body_or_not: case expect_body of { + BODY_NOT_EXPECTED -> none: empty; + default -> body: HTTP_Body(expect_body); + }; + }; + +Note that only one field is allowed after a given label. If multiple fields +are to be specified, they should be packed in another "record" type first. +The other usages of `case`_ are described later. + +array +~~~~~ + +A type can be defined as a sequence of "single-type elements". By default, +array type continue parsing for the array element in an infinite loop. +Or an array size can be specified to control the number of +match. &until can be also conditionally end parsing: + +.. code:: + + # This will match for 10 element only + type HTTP_Headers = HTTP_Header [10]; + + # This will match until the condition is met + type HTTP_Headers = HTTP_Header [] &until(/*Some condition*/); + +Array can also be used directly inside of "record". For example: + +.. code:: + + type DNS_message = record { + header: DNS_header; + question: DNS_question(this)[header.qdcount]; + answer: DNS_rr(this, DNS_ANSWER)[header.ancount]; + authority: DNS_rr(this, DNS_AUTHORITY)[header.nscount]; + additional: DNS_rr(this, DNS_ADDITIONAL)[header.arcount]; + }&byteorder = bigendian, &exportsourcedata + +flow +---- + +A "flow" defines how data is fed into the analyzer. It also maintains +custom state information declared by `%member`_. flow is configured by +specifying type of data unit. + +Syntax: + +.. code:: + + flow () { + = withcontext (); + }; + +When "flow" is added to top level context analyzer, it enables use of &oneline +and &length in "record" type. flow buffers data when there is not enough +to evaluate the record and dispatches data for evaluation when the +threshold is reached. + +flowunit +~~~~~~~~ + +When flowunit is used, the analyzer uses flow buffer to handle incremental +input and provide support for &oneline/&length. For further detail on +this, see `Buffering`_. + +.. code:: + + flowunit = HTTP_PDU(is_orig) withcontext (analyzer, this); + +datagram +~~~~~~~~ + +Opposite to flowunit, by declaring data unit as datagram, flow buffer is +opted out. This results in faster parsing but no incremental input +or buffering support. + +.. code:: + + datagram = HTTP_PDU(is_orig) withcontext (analyzer, this); + +Byte Ordering and Alignment +--------------------------- + +Byte Ordering +~~~~~~~~~~~~~ + +Byte Alignment +~~~~~~~~~~~~~~ + +.. code:: + + type RPC_Opaque = record { + length: uint32; + data: uint8[length]; + pad: padding align 4; # pad to 4-byte boundary + }; + +Functions +--------- + +User can define functions in binpac. +Function can be declared using one of the three ways: + +PAC with embedded body +~~~~~~~~~~~~~~~~~~~~~~ + +PAC style function prototype and embed the body using %{ %}:: + + function print_stuff(value :const_bytestring):bool + %{ + printf("Value [%s]\n", std_str(value).c_str()); + %} + +PAC with PAC-case body +~~~~~~~~~~~~~~~~~~~~~~ + +Pac style function with a case body, this type of declaration is useful for +extending later by casefunc:: + + function RPC_Service(prog: uint32, vers: uint32): EnumRPCService = + case prog of { + default -> RPC_SERVICE_UNKNOWN; + }; + + +Inlined by %code +~~~~~~~~~~~~~~~~ + +Function can be completely inlined by using %code:: + + %code{ + EnumRPCService RPC_Service(const RPC_Call* call) + { + return call ? call->service() : RPC_SERVICE_UNKNOWN; + } + %} + + +Extending +--------- + +PAC code can be extended by using "refine". This is useful for code +reusing and splitting functionality for parallel development. + +Extending record +~~~~~~~~~~~~~~~~ + +Record can be extended to add additional attribute(s) by +using "refine typeattr". One of the typical use is to add &let for split +protocol parsing from protocol analysis. + +.. code:: + + refine typeattr HTTP_RequestLine += &let { + process_request: bool = + process_func(method, uri, version); + }; + +Extending type case +~~~~~~~~~~~~~~~~~~~ + +.. code:: + + refine casetype RPC_Params += { + RPC_SERVICE_PORTMAP -> portmap: PortmapParams(call); + }; + +Extending function case +~~~~~~~~~~~~~~~~~~~~~~~ + +Function which is declared as a PAC case can be extended by adding +additional case into the switch. + +.. code:: + + refine casefunc RPC_BuildCallVal += { + RPC_SERVICE_PORTMAP -> + PortmapBuildCallVal(call, call.params.portmap); + }; + +Extending connection +~~~~~~~~~~~~~~~~~~~~ + +Connection can be extended to add functions and members. Example:: + + refine connection RPC_Conn += { + function ProcessPortmapReply(results: PortmapResults): bool + %{ + %} + }; + +State Management +---------------- + +State is maintained by extending parsing class by declaring derivative. +State lasts until the top level parsing unit (flowunit/datagram is destroyed). + +Keywords +-------- + +Source code embedding +~~~~~~~~~~~~~~~~~~~~~ + +C++ code can be embedded within the .pac file using the following +directives. These code will be copied into the final generated code. + +- %header{...%} + + Code to be inserted in binpac generated header file. + +- %code{...%} + + Code to be inserted at the beginning of binpac generated C++ file. + +.. _%member: + +- %member{...%} + + Add additional member(s) to connection (?) and flow class. + +- %init{...%} + + Code to be inserted in flow constructor. + +- %cleanup{...%} + + Code to be inserted in flow destructor. + +Embedded pac primitive +~~~~~~~~~~~~~~~~~~~~~~ + +- ${ + +- $set{ + +- $type{ + +- $typeof{ + +- $const_def{ + +Condition checking +~~~~~~~~~~~~~~~~~~ + +&until +...... + +"&until" is used in conjunction with array declaration. It specifies exit +condition for array parsing. + +.. code:: + + type HTTP_Headers = HTTP_Header[] &until($input.length() == 0); + +&requires +......... + +Process data dependencies before evaluating field. + +Example: typically, derivative field is evaluated after primary field. +However "&requires" is used to force evaluate of length before msg_body. + +.. code:: + + type RPC_Message = record { + xid: uint32; + msg_type: uint32; + msg_body: case msg_type of { + RPC_CALL -> call: RPC_Call(this); + RPC_REPLY -> reply: RPC_Reply(this); + } &requires(length); + } &let { + length = sourcedata.length(); # length of the RPC_Message + } &byteorder = bigendian, &exportsourcedata, &refcount; + +&if +... + +Evaluate field only if condition is met. + +.. code:: + + type DNS_label(msg: DNS_message) = record { + length: uint8; + data: case label_type of { + 0 -> label: bytestring &length = length; + 3 -> ptr_lo: uint8; + }; + } &let { + label_type: uint8 = length >> 6; + last: bool = (length == 0) || (label_type == 3); + ptr: DNS_name(msg) + withinput $context.flow.get_pointer(msg.sourcedata, + ((length & 0x3f) << 8) | ptr_lo) + &if(label_type == 3); + clear_pointer_set: bool = $context.flow.reset_pointer_set() + &if(last); + }; + +.. _case: + +case +.... + +There are two uses to the "case" keyword. + +* As part of record field. In this scenario, it allow alternative + methods to parse a field. Example:: + + type RPC_Reply(msg: RPC_Message) = record { + stat: uint32; + reply: case stat of { + MSG_ACCEPTED -> areply: RPC_AcceptedReply(call); + MSG_DENIED -> rreply: RPC_RejectedReply(call); + }; + } &let { + call: RPC_Call = context.connection.FindCall(msg.xid); + success: bool = (stat == MSG_ACCEPTED && areply.stat == SUCCESS); + }; + + +* As function definition. Example:: + + function RPC_Service(prog: uint32, vers: uint32): EnumRPCService = + case prog of { + default -> RPC_SERVICE_UNKNOWN; + }; + + +Note that one can "refine" both types of cases: + +.. code:: + + refine casefunc RPC_Service += { + 100000 -> RPC_SERVICE_PORTMAP; + }; + +Built-in macros +~~~~~~~~~~~~~~~ + +$input +...... + +This macro refers to the data that was passed into the ParseBuffer +function. When $input is used, binpac generate a const_bytestring +which contains the start and end pointer of the input. + +PAC grammar:: + + &until($input.length()==0); + +PAC source:: + + const_bytestring t_val__elem_input(t_begin_of_data, t_end_of_data); + if ( ( t_val__elem_input.length() == 0 ) ) + +$element +........ + +$element provides access to entry of the array type. Following are +the ways which $element can be used. + +* Current element. Check on the value of the most recently parsed entry. + This would get executed after each time an entry is parsed. Example:: + + type SMB_ascii_string = uint8[] &until($element == 0); + +* Current element's field. Example:: + + type DNS_label(msg: DNS_message) = record { + length: uint8; + data: case label_type of { + 0 -> label: bytestring &length = length; + 3 -> ptr_lo: uint8; + }; + } &let { + label_type: uint8 = length >> 6; + last: bool = (length == 0) || (label_type == 3); + }; + type DNS_name(msg: DNS_message) = record { + labels: DNS_label(msg)[] &until($element.last); + }; + +$context +........ + +This macro refers to the Analyzer context class (Context class gets +generated from analyzer withcontext {}). Using this macro, users +can gain access to the "flow" object and "analyzer" object. + +Other keywords +~~~~~~~~~~~~~~ + +&transient +.......... + +Do not create copy of the bytestring + +.. code:: + + type MIME_Line = record { + line: bytestring &restofdata &transient; + } &oneline; + +&let +.... + +Adds derivative field to a record + +.. code:: + + type ncp_request(length: uint32) = record { + data : uint8[length]; + } &let { + function = length > 0 ? data[0] : 0; + subfunction = length > 1 ? data[1] : 0; + }; + +let +... + +Declares global value. If the user does not specify a type, +the compiler will assume the "int" type. + +PAC grammar:: + + let myValue:uint8=10; + +PAC source:: + + uint8 const myValue = 10; + +PAC header:: + + extern uint8 const myValue; + +&restofdata +........... + +Grab the rest of the data available in the FlowBuffer. + +PAC grammar:: + + onebyte: uint8; + value: bytestring &restofdata &transient; + +PAC source:: + + // Parse "onebyte" + onebyte_ = *((uint8 const *) (t_begin_of_data)); + // Parse "value" + int t_value_string_length; + t_value_string_length = (t_end_of_data) - ((t_begin_of_data + 1)); + int t_value__size; + t_value__size = t_value_string_length; + value_.init((t_begin_of_data + 1), t_value_string_length); + +&length +....... + +Length can appear in two different contexts: as property of a field +or as property of a record. +Examples: +&length as field property:: + + protocol : bytestring &length = 4; + +translates into:: + + const_byteptr t_end_of_data = t_begin_of_data + 4; + int t_protocol_string_length; + t_protocol_string_length = 4; + int t_protocol__size; + t_protocol__size = t_protocol_string_length; + protocol_.init(t_begin_of_data, t_protocol_string_length); + + +&check +...... + +This was originally intended to implement the behavior of the +superseding "&enforce" attribute. It always has and always will just be +a no-op to ensure anything that uses this doesn't suddenly and +unintentionally break. + +&enforce +........ + +Check a condition and raise exception if not met. + +&chunked and $chunk +................... + +When parsing a long field with variable length, "chunked" can be used to +improve performance. However, chunked field are not buffered across +packet. Data for the chunk in the current packet can be access by +using "$chunk". + +&exportsourcedata +................. + +Data matched for a particular type, the data matched can be retained by +using "&exportsourcedata". + +.pac file + +.. code:: + + type myType = record { + data:uint8; + } &exportsourcedata; + +_pac.h + +.. code:: + + class myType + { + public: + myType(); + ~myType(); + int Parse(const_byteptr const t_begin_of_data, const_byteptr const _end_of_data); + uint8 myData() const { return myData_; } + const_bytestring const & sourcedata() const { return sourcedata_; } + protected: + uint8 myData_; + const_bytestring sourcedata_; + }; + +_pac.cc + +.. code:: + + sourcedata_ = const_bytestring(t_begin_of_data, t_end_of_data); + sourcedata_.set_end(t_begin_of_data + 1); + +Source data can be used within the type that match it or at the parent type. + +.. code:: + + type myParentType (child:myType) = record { + somedata:uint8; + } &let{ + do_something:bool = print_stuff(child.sourcedata); + }; + +translates into + +.. code:: + + do_something_ = print_stuff(child()->sourcedata()); + +&refcount +......... + + +withinput +......... + + +Parsing Methodology +=================== + +.. _Buffering: + +Buffering +--------- + +binpac supports incremental input to deal with packet fragmentation. This +is done via use of FlowBuffer class and maintaining buffering/parsing states. + +FlowBuffer Class +~~~~~~~~~~~~~~~~ + +FlowBuffer provides two mode of buffering: line and frame. Line mode is +useful for parsing line based language like HTTP. Frame mode is best for +fixed length message. Buffering mode can be switched during parsing and +is done transparently to the grammar writer. + +At compile time binpac calculates number of bytes required to evaluate +each field. During run time, data is buffered up in FlowBuffer until +there is enough to evaluate the "record". To optimize the buffering +process, if FlowBuffer has enough data to evaluate on the first NewData, +it would only mark the start and end pointer instead of copying. + +- void **NewMessage**\(); + + - Advances the orig_data_begin\_ pointer depend on current mode\_. Moves + by 1/2 characters in LINE_MODE, by frame_length\_ in FRAME_MODE + and nothing in UNKNOWN_MODE (default mode). + + - Set buffer_n\_ to 0 + + - Reset message_complete\_ + +- void **NewLine**\(); + + - Reset frame_length\_ and chunked\_, set mode\_ to LINE_MODE + +- void **NewFrame**\(int frame_length, bool chunked\_); + +- void **GrowFrame**\(int new_frame_length); + +- void **AppendToBuffer**\(const_byteptr data, int len); + + - Reallocate buffer\_ to add new data then copy data + +- void **ExpandBuffer**\(int length); + + - Reallocate buffer\_ to new size if new size is bigger than current size. + + - Set minimum size to 512 (optimization?) + +- void **MarkOrCopyLine**\(); + + - Seek current input for end of line (CR/LF/CRLF depend on line break mode). + If found append found data to buffer if one is already created or mark (set + frame_length\_) if one is not created (to minimize copying). If end of line + is not found, append partial data till end of input to buffer. Buffer + is created if one is not there. + +- const_byteptr **begin**\()/**end**\() + + - Returns buffer\_ and buffer_n\_ if a buffer exist, otherwise + orig_data_begin\_ and orig_data_begin\_ + frame_length\_. + +Parsing States +~~~~~~~~~~~~~~ + +* buffering_state\_ - each parsing class contains a flag indicating whether + there are enough data buffered to evaluate the next block. + +* parsing_state\_ - each parsing class which consists of multiple parsing + data unit (line/frames) has this flag indicating the parsing stage. Each + time new data comes in, it invokes parsing function and switch on + parsing_state to determine which sub parser to use next. + +Regular Expression +------------------ + +Evaluation Order +---------------- + +Running Binpac-generated Analyzer Standalone +============================================ + +To run binpac-generated code independent of Zeek. Regex library must be +substituted. Below is one way of doing it. Use the following three header +files. + +RE.h +---- + +.. code:: + + /*Dummy file to replace Zeek's file*/ + #include "binpac_pcre.h" + #include "bro_dummy.h" + +bro_dummy.h +----------- + +.. code:: + + #ifndef BRO_DUMMY + #define BRO_DUMMY + #define DEBUG_MSG(x...) fprintf(stderr, x) + /*Dummy to link, this function suppose to be in Zeek*/ + double network_time(); + #endif + +binpac_pcre.h +------------- + +.. code:: + + #ifndef bro_pcre_h + #define bro_pcre_h + #include + #include + #include + using namespace std; + // TODO: use configure to figure out the location of pcre.h + #include "pcre.h" + class RE_Matcher { + public: + RE_Matcher(const char* pat){ + pattern_ = "^"; + pattern_ += "("; + pattern_ += pat; + pattern_ += ")"; + pcre_ = NULL; + pextra_ = NULL; + } + ~RE_Matcher() { + if (pcre_) { + pcre_free(pcre_); + } + } + int Compile() { + const char *err = NULL; + int erroffset = 0; + pcre_ = pcre_compile(pattern_.c_str(), + 0, // options, + &err, + &erroffset, + NULL); + if (pcre_ == NULL) { + fprintf(stderr, + "Error in RE_Matcher::Compile(): %d:%s\n", + erroffset, err); + return 0; + } + return 1; + } + + int MatchPrefix (const char* s, int n){ + const char *err=NULL; + assert(pcre_); + const int MAX_NUM_OFFSETS = 30; + int offsets[MAX_NUM_OFFSETS]; + int ret = pcre_exec(pcre_, + pextra_, // pcre_extra + //NULL, // pcre_extra + s, n, + 0, // offset + 0, // options + offsets, + MAX_NUM_OFFSETS); + if (ret < 0) { + return -1; + } + assert(offsets[0] == 0); + return offsets[1]; + } + protected: + pcre *pcre_; + string pattern_; + }; + #endif + +main.cc +------- + +In your main source, add this dummy stub. + +.. code:: + + /*Dummy to link, this function suppose to be in Zeek*/ + double network_time(){ + return 0; + } + + +Q & A +===== + +* Does &oneline only work when "flow" is used? + + Yes. binpac uses the flowunit definition in "flow" to figure out which + types require buffering. For those that do, the parse function is: + + .. code:: + + bool ParseBuffer(flow_buffer_t t_flow_buffer, ContextHTTP * t_context); + + And the code of flow_buffer_t provides the functionality of buffering up to + one line. That's why &oneline is only active when "flow" is used and the + type requires buffering. + + In certain cases we would want to use &oneline even if the type does + not require buffering, binpac currently does not provide such functionality. + +* How would incremental input work in the case of regex? + + A regex should not take incremental input. (The binpac compiler will + complain when that happens.) It should always appear below some type + that has either &length=... or &oneline. + +* What is the role of Context_ class (generated by analyzer + withcontext)? + +* What is the difference between ''withcontext'' and w/o ''withcontext''? + + withcontext should always be there. It's fine to have an empty context. + +* Elaborate on $context and how it is related to "withcontext". + + A "context" parameter is passed to every type. It provides a vehicle to + pass something to every type without adding a parameter to every type. + In that sense, it's optional. It exists for convenience. + +* Example usage of composite type array. + + Please see HTTP_Headers in http-protocol.pac in the Zeek source code. + +* Clarification on "connection" keyword (binpac paper). + +* Need a new way to attach hook additional code to each class beside &let. + +* &transient, how is this different from declaring anonymous field? and + currently it doesn't seem to do much + + .. code:: + + type HTTP_Header = record { + name: HTTP_HEADER_NAME &transient; + : HTTP_WS; + value: bytestring &restofdata &transient; + } &oneline; + + .. code:: + + // Parse "name" + int t_name_string_length; + t_name_string_length = + HTTP_HEADER_NAME_re_011.MatchPrefix( + t_begin_of_data, + t_end_of_data - t_begin_of_data); + if ( t_name_string_length < 0 ) + { + throw ExceptionStringMismatch( "./http-protocol.pac:96", + "|([^: \\t]+:)", + string((const char *) (t_begin_of_data), (const char *) t_end_of_data).c_str() + ); + } + int t_name__size; + t_name__size = t_name_string_length; + name_.init(t_begin_of_data, t_name_string_length); + +* Detail on the globals ($context, $element, $input...etc) + +* How does BinPAC work with dynamic protocol detection? + + Well, you can use the code in DNS-binpac.cc as a reference. First, + create a pointer to the connection. (See the example in DNS-binpac.cc) + + .. code:: + + interp = new binpac::DNS::DNS_Conn(this); + + Pass the data received from "DeliverPacket" or "DeliverStream" to + "interp->NewData()". (Again, see the example in DNS-binpac.cc) + + .. code:: + + void DNS_UDP_Analyzer_binpac::DeliverPacket(int len, const u_char* data, bool orig, int seq, const IP_Hdr* ip, int caplen) + { + Analyzer::DeliverPacket(len, data, orig, seq, ip, caplen); + interp->NewData(orig, data, data + len); + } + +* Explanation of &withinput + +* Difference between using flow and not using flow (binpac generates Parse + method instead of ParseBuffer) + +* &check currently working? + +* Difference between flowunit and datagram, datagram and &oneline, &length? + +* Go over TODO list in binpac release + +* How would input get handle/buffered when length is not known (chunked) + +* More feature multi byte character? utf16 utf32 etc. + +TODO List +========= + +New Features +------------ + +* Provides a method to match simple ascii text. + +* Allows use fixed length array in addition to vector. + +Bugs +---- + +Small clean-ups +~~~~~~~~~~~~~~~ + +* Remove anonymous field bytestring assignment. + +* Redundant overflow checking/more efficient fixed length text copying. + +Warning/Errors +~~~~~~~~~~~~~~ + +Things that compiler should flag out at code generation time + +* Give warning when &transient is used on none bytestring + +* Give warning when &oneline, &length is used and flowunit is not. + +* Warning when more than one "connection" is defined