diff --git a/doc/scripts/builtins.rst b/doc/scripts/builtins.rst index 6abbd9110b..942e053b5f 100644 --- a/doc/scripts/builtins.rst +++ b/doc/scripts/builtins.rst @@ -6,159 +6,617 @@ Types The Bro scripting language supports the following built-in types. -.. TODO: add documentation - -.. bro:type:: void - .. bro:type:: bool + Reflects a value with one of two meanings: true or false. The two + ``bool`` constants are ``T`` and ``F``. + .. bro:type:: int + A numeric type representing a signed integer. An ``int`` constant + is a string of digits preceded by a ``+`` or ``-`` sign, e.g. + ``-42`` or ``+5``. When using type inferencing use care so that the + intended type is inferred, e.g. ``local size_difference = 0`` will + infer the :bro:type:`count` while ``local size_difference = +0`` + will infer :bro:type:`int`. + .. bro:type:: count + A numeric type representing an unsigned integer. A ``count`` + constant is a string of digits, e.g. ``1234`` or ``0``. + .. bro:type:: counter + An alias to :bro:type:`count` + +.. TODO: is there anything special about this type? + .. bro:type:: double + A numeric type representing a double-precision floating-point + number. Floating-point constants are written as a string of digits + with an optional decimal point, optional scale-factor in scientific + notation, and optional ``+`` or ``-`` sign. Examples are ``-1234``, + ``-1234e0``, ``3.14159``, and ``.003e-23``. + .. bro:type:: time + A temporal type representing an absolute time. There is currently + no way to specify a ``time`` constant, but one can use the + :bro:id:`current_time` or :bro:id:`network_time` built-in functions + to assign a value to a ``time``-typed variable. + .. bro:type:: interval + A temporal type representing a relative time. An ``interval`` + constant can be written as a numeric constant followed by a time + unit where the time unit is one of ``usec``, ``sec``, ``min``, + ``hr``, or ``day`` which respectively represent microseconds, + seconds, minutes, hours, and days. Whitespace between the numeric + constant and time unit is optional. Appending the letter "s" to the + time unit in order to pluralize it is also optional (to no semantic + effect). Examples of ``interval`` constants are ``3.5 min`` and + ``3.5mins``. An ``interval`` can also be negated, for example ``- + 12 hr`` represents "twelve hours in the past". Intervals also + support addition, subtraction, multiplication, division, and + comparison operations. + .. bro:type:: string + A type used to hold character-string values which represent text. + String constants are created by enclosing text in double quotes (") + and the backslash character (\) introduces escape sequences. + + Note that Bro represents strings internally as a count and vector of + bytes rather than a NUL-terminated byte string (although string + constants are also automatically NUL-terminated). This is because + network traffic can easily introduce NULs into strings either by + nature of an application, inadvertently, or maliciously. And while + NULs are allowed in Bro strings, when present in strings passed as + arguments to many functions, a run-time error can occur as their + presence likely indicates a sort of problem. In that case, the + string will also only be represented to the user as the literal + "" string. + .. bro:type:: pattern + A type representing regular-expression patterns which can be used + for fast text-searching operations. Pattern constants are created + by enclosing text within forward slashes (/) and is the same syntax + as the patterns supported by the `flex lexical analyzer + `_. The speed of + regular expression matching does not depend on the complexity or + size of the patterns. Patterns support two types of matching, exact + and embedded. + + In exact matching the ``==`` equality relational operator is used + with one :bro:type:`string` operand and one :bro:type:`pattern` + operand to check whether the full string exactly matches the + pattern. In this case, the ``^`` beginning-of-line and ``$`` + end-of-line anchors are redundant since pattern is implicitly + anchored to the beginning and end of the line to facilitate an exact + match. For example:: + + "foo" == /foo|bar/ + + yields true, while:: + + /foo|bar/ == "foobar" + + yields false. The ``!=`` operator would yield the negation of ``==``. + + In embedded matching the ``in`` operator is again used with one + :bro:type:`string` operand and one :bro:type:`pattern` operand + (which must be on the left-hand side), but tests whether the pattern + appears anywhere within the given string. For example:: + + /foo|bar/ in "foobar" + + yields true, while:: + + /^oob/ in "foobar" + + is false since "oob" does not appear at the start of "foobar". The + ``!in`` operator would yield the negation of ``in``. + .. bro:type:: enum + A type allowing the specification of a set of related values that + have no further structure. The only operations allowed on + enumerations are equality comparisons and they do not have + associated values or ordering. An example declaration: + + .. code:: bro + + type color: enum { Red, White, Blue, }; + + The last comma is after ``Blue`` is optional. + .. bro:type:: timer +.. TODO: is this a type that's exposed to users? + .. bro:type:: port + A type representing transport-level port numbers. Besides TCP and + UDP ports, there is a concept of an ICMP "port" where the source + port is the ICMP message type and the destination port the ICMP + message code. A ``port`` constant is written as an unsigned integer + followed by one of ``/tcp``, ``/udp``, ``/icmp``, or ``/unknown``. + + Ports can be compared for equality and also for ordering. When + comparing order across transport-level protocols, ``/unknown`` < + ``/tcp`` < ``/udp`` < ``icmp``, for example ``65535/tcp`` is smaller + than ``0/udp``. + .. bro:type:: addr -.. bro:type:: net + A type representing an IP address. Currently, Bro defaults to only + supporting IPv4 addresses unless configured/built with + ``--enable-brov6``, in which case, IPv6 addresses are supported. + + IPv4 address constants are written in "dotted quad" format, + ``A1.A2.A3.A4``, where Ai all lie between 0 and 255. + + IPv6 address constants are written as colon-separated hexadecimal form + as described by :rfc:`2373`. + + Hostname constants can also be used, but since a hostname can + correspond to multiple IP addresses, the type of such variable is a + :bro:type:`set` of :bro:type:`addr` elements. For example: + + .. code:: bro + + local a = www.google.com; + + Addresses can be compared for (in)equality using ``==`` and ``!=``. + They can also be masked with ``/`` to produce a :bro:type:`subnet`: + + .. code:: bro + + local a: addr = 192.168.1.100; + local s: subnet = 192.168.0.0/16; + if ( a/16 == s ) + print "true"; + + And checked for inclusion within a :bro:type:`subnet` using ``in`` : + + .. code:: bro + + local a: addr = 192.168.1.100; + local s: subnet = 192.168.0.0/16; + if ( a in s ) + print "true"; .. bro:type:: subnet + A type representing a block of IP addresses in CIDR notation. A + ``subnet`` constant is written as an :bro:type:`addr` followed by a + slash (/) and then the network prefix size specified as a decimal + number. For example, ``192.168.0.0/16``. + .. bro:type:: any + Used to bypass strong typing. For example, a function can take an + argument of type ``any`` when it may be of different types. + .. bro:type:: table -.. bro:type:: union + An associate array that maps from one set of values to another. The + values being mapped are termed the *index* or *indices* and the + result of the mapping is called the *yield*. Indexing into tables + is very efficient, and internally it is just a single hash table + lookup. -.. bro:type:: record + The table declaration syntax is:: -.. bro:type:: types + table [ type^+ ] of type -.. bro:type:: func + where *type^+* is one or more types, separated by commas. For example: -.. bro:type:: file + .. code:: bro -.. bro:type:: vector + global a: table[count] of string; -.. TODO: below are kind of "special cases" that bro knows about? + declares a table indexed by :bro:type:`count` values and yielding + :bro:type:`string` values. The yield type can also be more complex: + + .. code:: bro + + global a: table[count] of table[addr, port] of string; + + which declared a table indexed by :bro:type:`count` and yielding + another :bro:type:`table` which is indexed by an :bro:type:`addr` + and :bro:type:`port` to yield a :bro:type:`string`. + + Initialization of tables occurs by enclosing a set of initializers within + braces, for example: + + .. code:: bro + + global t: table[count] of string = { + [11] = "eleven", + [5] = "five", + }; + + Accessing table elements if provided by enclosing values within square + brackets (``[]``), for example: + + .. code:: bro + + t[13] = "thirteen"; + + And membership can be tested with ``in``: + + .. code:: bro + + if ( 13 in t ) + ... + + Iterate over tables with a ``for`` loop: + + .. code:: bro + + local t: table[count] of string; + for ( n in t ) + ... + + local services: table[addr, port] of string; + for ( [a, p] in services ) + ... + + Remove individual table elements with ``delete``: + + .. code:: bro + + delete t[13]; + + Nothing happens if the element with value ``13`` isn't present in + the table. + + Table size can be obtained by placing the table identifier between + vertical pipe (|) characters: + + .. code:: bro + + |t| .. bro:type:: set + A set is like a :bro:type:`table`, but it is a collection of indices + that do not map to any yield value. They are declared with the + syntax:: + + set [ type^+ ] + + where *type^+* is one or more types separated by commas. + + Sets are initialized by listing elements enclosed by curly braces: + + .. code:: bro + + global s: set[port] = { 21/tcp, 23/tcp, 80/tcp, 443/tcp }; + global s2: set[port, string] = { [21/tcp, "ftp"], [23/tcp, "telnet"] }; + + The types are explicitly shown in the example above, but they could + have been left to type inference. + + Set membership is tested with ``in``: + + .. code:: bro + + if ( 21/tcp in s ) + ... + + Elements are added with ``add``: + + .. code:: bro + + add s[22/tcp]; + + And removed with ``delete``: + + .. code:: bro + + delete s[21/tcp]; + + Set size can be obtained by placing the set identifier between + vertical pipe (|) characters: + + .. code:: bro + + |s| + +.. bro:type:: vector + + A vector is like a :bro:type:`table`, except it's always indexed by a + :bro:type:`count`. A vector is declared like: + + .. code:: bro + + global v: vector of string; + + And can be initialized with the vector constructor: + + .. code:: bro + + global v: vector of string = vector("one", "two", "three"); + + Adding an element to a vector involves accessing/assigning it: + + .. code:: bro + + v[3] = "four" + + Note how the vector indexing is 0-based. + + Vector size can be obtained by placing the vector identifier between + vertical pipe (|) characters: + + .. code:: bro + + |v| + +.. bro:type:: record + + A ``record`` is a collection of values. Each value has a field name + and a type. Values do not need to have the same type and the types + have no restrictions. An example record type definition: + + .. code:: bro + + type MyRecordType: record { + c: count; + s: string &optional; + }; + + Access to a record field uses the dollar sign (``$``) operator: + + .. code:: bro + + global r: MyRecordType; + r$c = 13; + + Record assignment can be done field by field or as a whole like: + + .. code:: bro + + r = [$c = 13, $s = "thirteen"]; + + When assigning a whole record value, all fields that are not + :bro:attr:`&optional` or have a :bro:attr:`&default` attribute must + be specified. + + To test for existence of field that is :bro:attr:`&optional`, use the + ``?$`` operator: + + .. code:: bro + + if ( r?$s ) + ... + +.. bro:type:: file + + Bro supports writing to files, but not reading from them. For + example, declare, open, and write to a file and finally close it + like: + + .. code:: bro + + global f: file = open("myfile"); + print f, "hello, world"; + close(f); + + Writing to files like this for logging usually isn't recommend, for better + logging support see :doc:`/logging`. + .. bro:type:: function + Function types in Bro are declared using:: + + function( argument* ): type + + where *argument* is a (possibly empty) comma-separated list of + arguments, and *type* is an optional return type. For example: + + .. code:: bro + + global greeting: function(name: string): string; + + Here ``greeting`` is an identifier with a certain function type. + The function body is not defined yet and ``greeting`` could even + have different function body values at different times. To define + a function including a body value, the syntax is like: + + .. code:: bro + + function greeting(name: string): string + { + return "Hello, " + name; + } + + Note that in the definition above, it's not necessary for us to have + done the first (forward) declaration of ``greeting`` as a function + type, but when it is, the argument list and return type much match + exactly. + + Function types don't need to have a name and can be assigned anonymously: + + .. code:: bro + + greeting = function(name: string): string { return "Hi, " + name; }; + + And finally, the function can be called like: + + .. code:: bro + + print greeting("Dave"); + .. bro:type:: event + Event handlers are nearly identical in both syntax and semantics to + a :bro:type:`function`, with the two differences being that event + handlers have no return type since they never return a value, and + you cannot call an event handler. Instead of directly calling an + event handler from a script, event handler bodies are executed when + they are invoked by one of three different methods: + + - From the event engine + + When the event engine detects an event for which you have + defined a corresponding event handler, it queues an event for + that handler. The handler is invoked as soon as the event + engine finishes processing the current packet and flushing the + invocation of other event handlers that were queued first. + + - With the ``event`` statement from a script + + Immediately queuing invocation of an event handler occurs like: + + .. code:: bro + + event password_exposed(user, password); + + This assumes that ``password_exposed`` was previously declared + as an event handler type with compatible arguments. + + - Via the ``schedule`` expression in a script + + This delays the invocation of event handlers until some time in + the future. For example: + + .. code:: bro + + schedule 5 secs { password_exposed(user, password) }; + + Multiple event handler bodies can be defined for the same event handler + identifier and the body of each will be executed in turn. Ordering + of execution can be influenced with :bro:attr:`&priority`. + Attributes ---------- Attributes occur at the end of type/event declarations and change their -behavior. The syntax is ``&key`` or ``&key=val``, e.g., -``type T: set[count] &read_expire=5min`` or ``event foo() &priority=-3``. -The Bro scripting language supports the following built-in attributes. +behavior. The syntax is ``&key`` or ``&key=val``, e.g., ``type T: +set[count] &read_expire=5min`` or ``event foo() &priority=-3``. The Bro +scripting language supports the following built-in attributes. -## Allows record field to be missing. For example the type -## ``record { a: int, b: port &optional }`` could be instantiated both as -## singleton ``[$a=127.0.0.1]`` or pair ``[$a=127.0.0.1, $b=80/tcp]``. .. bro:attr:: &optional -## Uses a default value for a record field or container elements. For example, -## ``table[int] of string &default="foo" }`` would create table that returns -## The :bro:type:`string` ``"foo"`` for any non-existing index. + Allows record field to be missing. For example the type ``record { + a: int, b: port &optional }`` could be instantiated both as + singleton ``[$a=127.0.0.1]`` or pair ``[$a=127.0.0.1, $b=80/tcp]``. + .. bro:attr:: &default -## Allows for redefinition of initial object values. This is typically used -## with constants, for example, ``const clever = T &redef;`` would allow the -## constant to be redifined at some later point during script execution. + Uses a default value for a record field or container elements. For + example, ``table[int] of string &default="foo" }`` would create + table that returns The :bro:type:`string` ``"foo"`` for any + non-existing index. + .. bro:attr:: &redef -## Rotates a file after a specified interval. + Allows for redefinition of initial object values. This is typically + used with constants, for example, ``const clever = T &redef;`` would + allow the constant to be redifined at some later point during script + execution. + .. bro:attr:: &rotate_interval -## Rotates af file after it has reached a given size in bytes. + Rotates a file after a specified interval. + .. bro:attr:: &rotate_size -## ..TODO: needs to be documented. + Rotates af file after it has reached a given size in bytes. + .. bro:attr:: &add_func -## ..TODO: needs to be documented. +.. TODO: needs to be documented. + .. bro:attr:: &delete_func -## Called right before a container element expires. +.. TODO: needs to be documented. + .. bro:attr:: &expire_func -## Specifies a read expiration timeout for container elements. That is, the -## element expires after the given amount of time since the last time it has -## been read. Note that a write also counts as a read. + Called right before a container element expires. + .. bro:attr:: &read_expire -## Specifies a write expiration timeout for container elements. That is, the -## element expires after the given amount of time since the last time it has -## been written. + Specifies a read expiration timeout for container elements. That is, + the element expires after the given amount of time since the last + time it has been read. Note that a write also counts as a read. + .. bro:attr:: &write_expire -## Specifies a creation expiration timeout for container elements. That is, the -## element expires after the given amount of time since it has been inserted -## into the container, regardless of any reads or writes. + Specifies a write expiration timeout for container elements. That + is, the element expires after the given amount of time since the + last time it has been written. + .. bro:attr:: &create_expire -## Makes a variable persistent, i.e., its value is writen to disk (per default -## at shutdown time). + Specifies a creation expiration timeout for container elements. That + is, the element expires after the given amount of time since it has + been inserted into the container, regardless of any reads or writes. + .. bro:attr:: &persistent -## Synchronizes variable accesses across nodes. The value of a -## ``&synchronized`` variable is automatically propagated to all peers when it -## changes. + Makes a variable persistent, i.e., its value is writen to disk (per + default at shutdown time). + .. bro:attr:: &synchronized -## ..TODO: needs to be documented. + Synchronizes variable accesses across nodes. The value of a + ``&synchronized`` variable is automatically propagated to all peers + when it changes. + .. bro:attr:: &postprocessor -## Encryptes files right before writing them to disk. -## ..TODO: needs to be documented in more detail. +.. TODO: needs to be documented. + .. bro:attr:: &encrypt -## ..TODO: needs to be documented. + Encrypts files right before writing them to disk. + +.. TODO: needs to be documented in more detail. + .. bro:attr:: &match -## Deprecated. Will be removed. +.. TODO: needs to be documented. + .. bro:attr:: &disable_print_hook -## Opens a file in raw mode, i.e., non-ASCII characters are not escaped. + Deprecated. Will be removed. + .. bro:attr:: &raw_output -## Prefers set union to assignment for synchronized state. This attribute is -## used in conjunction with :bro:attr:`synchronized` container types: when the -## same container is updated at two peers with different value, the propagation -## of the state causes a race condition, where the last update succeeds. This -## can cause inconsistencies and can be avoided by unifying the two sets, -## rather than merely overwriting the old value. + Opens a file in raw mode, i.e., non-ASCII characters are not + escaped. + .. bro:attr:: &mergeable -## Specifies the execution priority of an event handler. Higher values are -## executed before lower ones. The default value is 0. + Prefers set union to assignment for synchronized state. This + attribute is used in conjunction with :bro:attr:`&synchronized` + container types: when the same container is updated at two peers + with different value, the propagation of the state causes a race + condition, where the last update succeeds. This can cause + inconsistencies and can be avoided by unifying the two sets, rather + than merely overwriting the old value. + .. bro:attr:: &priority -## Groups event handlers such that those in the same group can be jointly -## activated or deactivated. + Specifies the execution priority of an event handler. Higher values + are executed before lower ones. The default value is 0. + .. bro:attr:: &group -## Writes a record field to the associated log stream. + Groups event handlers such that those in the same group can be + jointly activated or deactivated. + .. bro:attr:: &log + Writes a record field to the associated log stream. + .. bro:attr:: (&tracked) + +.. TODO: needs documented or removed if it's not used anywhere.