From 2ff746fea7224013b720d44cc57e1a9d134e04ae Mon Sep 17 00:00:00 2001 From: Jon Siwek Date: Thu, 17 Jan 2019 14:09:29 -0600 Subject: [PATCH] Change doc/ subdir into a git submodule The docs now live at https://github.com/zeek/zeek-docs --- .gitmodules | 3 + .readthedocs.yml | 4 - CHANGES | 6 + CMakeLists.txt | 1 - Makefile | 11 +- NEWS | 2372 +-- VERSION | 2 +- doc | 1 + doc/.gitignore | 2 - doc/CMakeLists.txt | 20 - doc/LICENSE | 5 - doc/README | 28 - doc/_templates/breadcrumbs.html | 15 - doc/_templates/layout.html | 7 - doc/cluster/index.rst | 188 - doc/components/binpac/README.rst | 1 - doc/components/bro-aux/README.rst | 1 - doc/components/broctl/README.rst | 1 - doc/components/broker/README.rst | 1 - doc/components/btest/README.rst | 1 - doc/components/capstats/README.rst | 1 - doc/components/index.rst | 22 - doc/components/pysubnettree/README.rst | 1 - doc/components/trace-summary/README.rst | 1 - doc/conf.py | 235 - doc/configuration/index.rst | 253 - doc/devel/plugins.rst | 488 - doc/examples/httpmonitor/file_extraction.bro | 24 - doc/examples/httpmonitor/http_proxy_01.bro | 5 - doc/examples/httpmonitor/http_proxy_02.bro | 26 - doc/examples/httpmonitor/http_proxy_03.bro | 31 - doc/examples/httpmonitor/http_proxy_04.bro | 40 - doc/examples/httpmonitor/index.rst | 196 - doc/examples/ids/index.rst | 203 - doc/examples/index.rst | 13 - doc/examples/logs/index.rst | 308 - doc/examples/mimestats/index.rst | 108 - doc/examples/mimestats/mimestats.bro | 64 - .../scripting/connection_record_01.bro | 6 - .../scripting/connection_record_02.bro | 7 - .../scripting/data_struct_record_01.bro | 22 - .../scripting/data_struct_record_02.bro | 41 - .../scripting/data_struct_set_declaration.bro | 22 - .../scripting/data_struct_table_complex.bro | 13 - .../data_struct_table_declaration.bro | 19 - doc/examples/scripting/data_struct_vector.bro | 7 - .../data_struct_vector_declaration.bro | 15 - .../scripting/data_struct_vector_iter.bro | 7 - doc/examples/scripting/data_type_const.bro | 9 - .../scripting/data_type_const_simple.bro | 4 - .../scripting/data_type_declaration.bro | 9 - doc/examples/scripting/data_type_interval.bro | 18 - doc/examples/scripting/data_type_local.bro | 11 - .../scripting/data_type_pattern_01.bro | 13 - .../scripting/data_type_pattern_02.bro | 10 - doc/examples/scripting/data_type_record.bro | 25 - doc/examples/scripting/data_type_subnets.bro | 15 - doc/examples/scripting/data_type_time.bro | 4 - .../framework_logging_factorial_01.bro | 19 - .../framework_logging_factorial_02.bro | 35 - .../framework_logging_factorial_03.bro | 45 - .../framework_logging_factorial_04.bro | 50 - .../scripting/framework_notice_hook_01.bro | 7 - .../framework_notice_hook_suppression_01.bro | 7 - .../framework_notice_shortcuts_01.bro | 7 - .../framework_notice_shortcuts_02.bro | 6 - doc/examples/scripting/http_main.bro | 7 - doc/examples/scripting/index.rst | 1706 -- doc/examples/scripting/using_bro_sandbox_01 | 4 - doc/examples/scripting/using_bro_sandbox_02 | 4 - doc/ext/bro.py | 298 - doc/frameworks/broker.rst | 618 - doc/frameworks/broker/cluster-layout.png | Bin 56499 -> 0 bytes doc/frameworks/broker/cluster-layout.xml | 2 - .../broker/connecting-connector.bro | 12 - doc/frameworks/broker/connecting-listener.bro | 17 - doc/frameworks/broker/events-connector.bro | 35 - doc/frameworks/broker/events-listener.bro | 33 - doc/frameworks/broker/logs-connector.bro | 36 - doc/frameworks/broker/logs-listener.bro | 22 - doc/frameworks/broker/stores-connector.bro | 29 - doc/frameworks/broker/stores-listener.bro | 79 - doc/frameworks/broker/testlog.bro | 17 - doc/frameworks/configuration.rst | 198 - doc/frameworks/file-analysis.rst | 148 - doc/frameworks/file_analysis_01.bro | 20 - doc/frameworks/file_analysis_02.bro | 12 - doc/frameworks/file_analysis_03.bro | 25 - doc/frameworks/geoip.rst | 146 - doc/frameworks/index.rst | 19 - doc/frameworks/input.rst | 334 - doc/frameworks/intel.rst | 143 - doc/frameworks/logging-input-sqlite.rst | 167 - doc/frameworks/logging.rst | 534 - .../netcontrol-1-drop-with-debug.bro | 10 - doc/frameworks/netcontrol-10-use-skeleton.bro | 10 - doc/frameworks/netcontrol-2-ssh-guesser.bro | 16 - doc/frameworks/netcontrol-3-ssh-guesser.bro | 16 - doc/frameworks/netcontrol-4-drop.bro | 26 - doc/frameworks/netcontrol-5-hook.bro | 22 - doc/frameworks/netcontrol-6-find.bro | 17 - doc/frameworks/netcontrol-7-catch-release.bro | 10 - doc/frameworks/netcontrol-8-multiple.bro | 29 - doc/frameworks/netcontrol-9-skeleton.bro | 39 - doc/frameworks/netcontrol-architecture.png | Bin 103171 -> 0 bytes doc/frameworks/netcontrol-openflow.png | Bin 85573 -> 0 bytes doc/frameworks/netcontrol-rules.png | Bin 91598 -> 0 bytes doc/frameworks/netcontrol.rst | 806 - doc/frameworks/notice.rst | 368 - doc/frameworks/notice_ssh_guesser.bro | 10 - doc/frameworks/signatures.rst | 421 - doc/frameworks/sqlite-conn-filter.bro | 12 - doc/frameworks/sqlite-read-events.bro | 40 - doc/frameworks/sqlite-read-table.bro | 35 - doc/frameworks/sumstats-countconns.bro | 36 - doc/frameworks/sumstats-toy-scan.bro | 45 - doc/frameworks/sumstats.rst | 110 - doc/images/deployment.png | Bin 36081 -> 0 bytes doc/index.rst | 50 - doc/install/CHANGES-binpac.txt | 1 - doc/install/CHANGES-bro-aux.txt | 1 - doc/install/CHANGES-bro.txt | 1 - doc/install/CHANGES-broctl.txt | 1 - doc/install/CHANGES-broker.txt | 1 - doc/install/CHANGES-btest.txt | 1 - doc/install/CHANGES-capstats.txt | 1 - doc/install/CHANGES-pysubnettree.txt | 1 - doc/install/CHANGES-trace-summary.txt | 1 - doc/install/NEWS.rst | 1 - doc/install/changes.rst | 61 - doc/install/cross-compiling.rst | 83 - doc/install/guidelines.rst | 62 - doc/install/index.rst | 13 - doc/install/install.rst | 226 - doc/install/release-notes.rst | 11 - doc/install/upgrade.rst | 10 - doc/intro/architecture.png | Bin 139697 -> 0 bytes doc/intro/bro-eyes.png | Bin 46415 -> 0 bytes doc/intro/history.png | Bin 163112 -> 0 bytes doc/intro/index.rst | 245 - doc/quickstart/conditional-notice.bro | 24 - doc/quickstart/index.rst | 452 - doc/script-reference/attributes.rst | 241 - .../autogenerated-file-analyzer-index.rst | 946 - .../autogenerated-package-index.rst | 309 - .../autogenerated-protocol-analyzer-index.rst | 14385 ---------------- .../autogenerated-script-index.rst | 470 - doc/script-reference/directives.rst | 200 - doc/script-reference/file-analyzers.rst | 1 - doc/script-reference/index.rst | 21 - doc/script-reference/log-files.rst | 190 - doc/script-reference/notices.rst | 8 - doc/script-reference/operators.rst | 304 - doc/script-reference/packages.rst | 14 - doc/script-reference/proto-analyzers.rst | 1 - doc/script-reference/scripts.rst | 5 - doc/script-reference/statements.rst | 723 - doc/script-reference/types.rst | 974 -- doc/scripts/base/bif/__load__.bro.rst | 14 - doc/scripts/base/bif/analyzer.bif.bro.rst | 66 - doc/scripts/base/bif/bloom-filter.bif.bro.rst | 205 - doc/scripts/base/bif/bro.bif.bro.rst | 3567 ---- doc/scripts/base/bif/broxygen.bif.bro.rst | 88 - .../base/bif/cardinality-counter.bif.bro.rst | 117 - doc/scripts/base/bif/comm.bif.bro.rst | 96 - doc/scripts/base/bif/const.bif.bro.rst | 18 - doc/scripts/base/bif/data.bif.bro.rst | 322 - doc/scripts/base/bif/event.bif.bro.rst | 1507 -- .../base/bif/file_analysis.bif.bro.rst | 111 - doc/scripts/base/bif/index.rst | 466 - doc/scripts/base/bif/input.bif.bro.rst | 54 - doc/scripts/base/bif/logging.bif.bro.rst | 78 - doc/scripts/base/bif/messaging.bif.bro.rst | 151 - doc/scripts/base/bif/option.bif.bro.rst | 86 - doc/scripts/base/bif/pcap.bif.bro.rst | 98 - .../bif/plugins/Bro_ARP.events.bif.bro.rst | 113 - .../plugins/Bro_AsciiReader.ascii.bif.bro.rst | 16 - .../plugins/Bro_AsciiWriter.ascii.bif.bro.rst | 16 - .../plugins/Bro_BackDoor.events.bif.bro.rst | 99 - .../Bro_BenchmarkReader.benchmark.bif.bro.rst | 16 - .../Bro_BinaryReader.binary.bif.bro.rst | 16 - .../plugins/Bro_BitTorrent.events.bif.bro.rst | 310 - .../Bro_ConfigReader.config.bif.bro.rst | 16 - .../plugins/Bro_ConnSize.events.bif.bro.rst | 64 - .../Bro_ConnSize.functions.bif.bro.rst | 103 - .../plugins/Bro_DCE_RPC.consts.bif.bro.rst | 15 - .../plugins/Bro_DCE_RPC.events.bif.bro.rst | 198 - .../bif/plugins/Bro_DCE_RPC.types.bif.bro.rst | 105 - .../bif/plugins/Bro_DHCP.events.bif.bro.rst | 43 - .../bif/plugins/Bro_DHCP.types.bif.bro.rst | 16 - .../bif/plugins/Bro_DNP3.events.bif.bro.rst | 567 - .../bif/plugins/Bro_DNS.events.bif.bro.rst | 838 - .../bif/plugins/Bro_FTP.events.bif.bro.rst | 72 - .../bif/plugins/Bro_FTP.functions.bif.bro.rst | 110 - .../bif/plugins/Bro_File.events.bif.bro.rst | 44 - .../Bro_FileEntropy.events.bif.bro.rst | 38 - .../Bro_FileExtract.events.bif.bro.rst | 50 - .../Bro_FileExtract.functions.bif.bro.rst | 31 - .../plugins/Bro_FileHash.events.bif.bro.rst | 43 - .../bif/plugins/Bro_Finger.events.bif.bro.rst | 74 - .../bif/plugins/Bro_GSSAPI.events.bif.bro.rst | 36 - .../bif/plugins/Bro_GTPv1.events.bif.bro.rst | 151 - .../plugins/Bro_Gnutella.events.bif.bro.rst | 131 - .../bif/plugins/Bro_HTTP.events.bif.bro.rst | 358 - .../plugins/Bro_HTTP.functions.bif.bro.rst | 58 - .../bif/plugins/Bro_ICMP.events.bif.bro.rst | 459 - .../bif/plugins/Bro_IMAP.events.bif.bro.rst | 49 - .../bif/plugins/Bro_IRC.events.bif.bro.rst | 1098 -- .../bif/plugins/Bro_Ident.events.bif.bro.rst | 109 - .../plugins/Bro_InterConn.events.bif.bro.rst | 36 - .../bif/plugins/Bro_KRB.events.bif.bro.rst | 250 - .../bif/plugins/Bro_KRB.types.bif.bro.rst | 16 - .../bif/plugins/Bro_Login.events.bif.bro.rst | 547 - .../plugins/Bro_Login.functions.bif.bro.rst | 69 - .../bif/plugins/Bro_MIME.events.bif.bro.rst | 280 - .../bif/plugins/Bro_Modbus.events.bif.bro.rst | 530 - .../bif/plugins/Bro_MySQL.events.bif.bro.rst | 138 - .../bif/plugins/Bro_NCP.consts.bif.bro.rst | 15 - .../bif/plugins/Bro_NCP.events.bif.bro.rst | 86 - .../bif/plugins/Bro_NTLM.events.bif.bro.rst | 67 - .../bif/plugins/Bro_NTLM.types.bif.bro.rst | 16 - .../bif/plugins/Bro_NTP.events.bif.bro.rst | 50 - .../plugins/Bro_NetBIOS.events.bif.bro.rst | 277 - .../plugins/Bro_NetBIOS.functions.bif.bro.rst | 53 - .../plugins/Bro_NoneWriter.none.bif.bro.rst | 16 - .../bif/plugins/Bro_PE.events.bif.bro.rst | 110 - .../bif/plugins/Bro_POP3.events.bif.bro.rst | 234 - .../bif/plugins/Bro_RADIUS.events.bif.bro.rst | 59 - .../bif/plugins/Bro_RDP.events.bif.bro.rst | 134 - .../bif/plugins/Bro_RDP.types.bif.bro.rst | 16 - .../bif/plugins/Bro_RFB.events.bif.bro.rst | 116 - .../bif/plugins/Bro_RPC.events.bif.bro.rst | 1281 -- .../bif/plugins/Bro_RawReader.raw.bif.bro.rst | 16 - .../bif/plugins/Bro_SIP.events.bif.bro.rst | 157 - .../bif/plugins/Bro_SMB.consts.bif.bro.rst | 15 - .../bif/plugins/Bro_SMB.events.bif.bro.rst | 39 - ...o_SMB.smb1_com_check_directory.bif.bro.rst | 65 - .../Bro_SMB.smb1_com_close.bif.bro.rst | 45 - ..._SMB.smb1_com_create_directory.bif.bro.rst | 67 - .../plugins/Bro_SMB.smb1_com_echo.bif.bro.rst | 73 - .../Bro_SMB.smb1_com_logoff_andx.bif.bro.rst | 44 - .../Bro_SMB.smb1_com_negotiate.bif.bro.rst | 69 - .../Bro_SMB.smb1_com_nt_cancel.bif.bro.rst | 42 - ...ro_SMB.smb1_com_nt_create_andx.bif.bro.rst | 75 - ...SMB.smb1_com_query_information.bif.bro.rst | 46 - .../Bro_SMB.smb1_com_read_andx.bif.bro.rst | 74 - ...MB.smb1_com_session_setup_andx.bif.bro.rst | 66 - .../Bro_SMB.smb1_com_transaction.bif.bro.rst | 81 - .../Bro_SMB.smb1_com_transaction2.bif.bro.rst | 122 - ...mb1_com_transaction2_secondary.bif.bro.rst | 49 - ...smb1_com_transaction_secondary.bif.bro.rst | 48 - ...SMB.smb1_com_tree_connect_andx.bif.bro.rst | 74 - ...o_SMB.smb1_com_tree_disconnect.bif.bro.rst | 45 - .../Bro_SMB.smb1_com_write_andx.bif.bro.rst | 72 - .../plugins/Bro_SMB.smb1_events.bif.bro.rst | 86 - .../Bro_SMB.smb2_com_close.bif.bro.rst | 68 - .../Bro_SMB.smb2_com_create.bif.bro.rst | 68 - .../Bro_SMB.smb2_com_negotiate.bif.bro.rst | 68 - .../plugins/Bro_SMB.smb2_com_read.bif.bro.rst | 51 - ...Bro_SMB.smb2_com_session_setup.bif.bro.rst | 69 - .../Bro_SMB.smb2_com_set_info.bif.bro.rst | 101 - .../Bro_SMB.smb2_com_tree_connect.bif.bro.rst | 68 - ...o_SMB.smb2_com_tree_disconnect.bif.bro.rst | 58 - .../Bro_SMB.smb2_com_write.bif.bro.rst | 51 - .../plugins/Bro_SMB.smb2_events.bif.bro.rst | 48 - .../bif/plugins/Bro_SMB.types.bif.bro.rst | 15 - .../bif/plugins/Bro_SMTP.events.bif.bro.rst | 169 - .../plugins/Bro_SMTP.functions.bif.bro.rst | 34 - .../bif/plugins/Bro_SNMP.events.bif.bro.rst | 281 - .../bif/plugins/Bro_SNMP.types.bif.bro.rst | 16 - .../bif/plugins/Bro_SOCKS.events.bif.bro.rst | 99 - .../Bro_SQLiteReader.sqlite.bif.bro.rst | 16 - .../Bro_SQLiteWriter.sqlite.bif.bro.rst | 16 - .../bif/plugins/Bro_SSH.events.bif.bro.rst | 330 - .../bif/plugins/Bro_SSH.types.bif.bro.rst | 16 - .../bif/plugins/Bro_SSL.events.bif.bro.rst | 816 - .../bif/plugins/Bro_SSL.functions.bif.bro.rst | 34 - .../bif/plugins/Bro_SSL.types.bif.bro.rst | 16 - .../Bro_SteppingStone.events.bif.bro.rst | 57 - .../bif/plugins/Bro_Syslog.events.bif.bro.rst | 47 - .../bif/plugins/Bro_TCP.events.bif.bro.rst | 495 - .../bif/plugins/Bro_TCP.functions.bif.bro.rst | 122 - .../bif/plugins/Bro_Teredo.events.bif.bro.rst | 103 - .../bif/plugins/Bro_UDP.events.bif.bro.rst | 99 - .../plugins/Bro_Unified2.events.bif.bro.rst | 52 - .../plugins/Bro_Unified2.types.bif.bro.rst | 15 - .../bif/plugins/Bro_X509.events.bif.bro.rst | 145 - .../plugins/Bro_X509.functions.bif.bro.rst | 212 - .../plugins/Bro_X509.ocsp_events.bif.bro.rst | 194 - .../bif/plugins/Bro_X509.types.bif.bro.rst | 15 - .../bif/plugins/Bro_XMPP.events.bif.bro.rst | 34 - doc/scripts/base/bif/plugins/__load__.bro.rst | 14 - doc/scripts/base/bif/plugins/index.rst | 358 - doc/scripts/base/bif/reporter.bif.bro.rst | 250 - doc/scripts/base/bif/stats.bif.bro.rst | 327 - doc/scripts/base/bif/store.bif.bro.rst | 150 - doc/scripts/base/bif/strings.bif.bro.rst | 815 - doc/scripts/base/bif/top-k.bif.bro.rst | 206 - doc/scripts/base/bif/types.bif.bro.rst | 347 - .../base/files/extract/__load__.bro.rst | 14 - doc/scripts/base/files/extract/index.rst | 13 - doc/scripts/base/files/extract/main.bro.rst | 84 - doc/scripts/base/files/hash/__load__.bro.rst | 14 - doc/scripts/base/files/hash/index.rst | 13 - doc/scripts/base/files/hash/main.bro.rst | 22 - doc/scripts/base/files/pe/__load__.bro.rst | 14 - doc/scripts/base/files/pe/consts.bro.rst | 281 - doc/scripts/base/files/pe/index.rst | 16 - doc/scripts/base/files/pe/main.bro.rst | 115 - .../base/files/unified2/__load__.bro.rst | 14 - doc/scripts/base/files/unified2/index.rst | 13 - doc/scripts/base/files/unified2/main.bro.rst | 166 - doc/scripts/base/files/x509/__load__.bro.rst | 14 - doc/scripts/base/files/x509/index.rst | 14 - doc/scripts/base/files/x509/main.bro.rst | 78 - .../base/frameworks/analyzer/__load__.bro.rst | 14 - .../base/frameworks/analyzer/index.rst | 26 - .../base/frameworks/analyzer/main.bro.rst | 246 - .../base/frameworks/broker/__load__.bro.rst | 14 - doc/scripts/base/frameworks/broker/index.rst | 22 - .../base/frameworks/broker/log.bro.rst | 67 - .../base/frameworks/broker/main.bro.rst | 718 - .../base/frameworks/broker/store.bro.rst | 1199 -- .../base/frameworks/cluster/__load__.bro.rst | 14 - doc/scripts/base/frameworks/cluster/index.rst | 26 - .../base/frameworks/cluster/main.bro.rst | 529 - .../base/frameworks/cluster/pools.bro.rst | 291 - .../base/frameworks/config/__load__.bro.rst | 14 - doc/scripts/base/frameworks/config/index.rst | 25 - .../base/frameworks/config/input.bro.rst | 54 - .../base/frameworks/config/main.bro.rst | 101 - .../base/frameworks/config/weird.bro.rst | 17 - .../base/frameworks/control/__load__.bro.rst | 14 - doc/scripts/base/frameworks/control/index.rst | 18 - .../base/frameworks/control/main.bro.rst | 218 - .../base/frameworks/dpd/__load__.bro.rst | 14 - doc/scripts/base/frameworks/dpd/index.rst | 16 - doc/scripts/base/frameworks/dpd/main.bro.rst | 99 - .../base/frameworks/files/__load__.bro.rst | 14 - doc/scripts/base/frameworks/files/index.rst | 20 - .../frameworks/files/magic/__load__.bro.rst | 13 - .../base/frameworks/files/magic/index.rst | 9 - .../base/frameworks/files/main.bro.rst | 580 - .../base/frameworks/input/__load__.bro.rst | 14 - doc/scripts/base/frameworks/input/index.rst | 50 - .../base/frameworks/input/main.bro.rst | 419 - .../frameworks/input/readers/ascii.bro.rst | 104 - .../input/readers/benchmark.bro.rst | 77 - .../frameworks/input/readers/binary.bro.rst | 32 - .../frameworks/input/readers/config.bro.rst | 95 - .../base/frameworks/input/readers/raw.bro.rst | 58 - .../frameworks/input/readers/sqlite.bro.rst | 59 - .../base/frameworks/intel/__load__.bro.rst | 14 - .../base/frameworks/intel/files.bro.rst | 27 - doc/scripts/base/frameworks/intel/index.rst | 29 - .../base/frameworks/intel/input.bro.rst | 36 - .../base/frameworks/intel/main.bro.rst | 531 - .../base/frameworks/logging/__load__.bro.rst | 14 - doc/scripts/base/frameworks/logging/index.rst | 87 - .../base/frameworks/logging/main.bro.rst | 1111 -- .../logging/postprocessors/__load__.bro.rst | 14 - .../logging/postprocessors/index.rst | 44 - .../logging/postprocessors/scp.bro.rst | 121 - .../logging/postprocessors/sftp.bro.rst | 124 - .../frameworks/logging/writers/ascii.bro.rst | 162 - .../frameworks/logging/writers/none.bro.rst | 40 - .../frameworks/logging/writers/sqlite.bro.rst | 60 - .../frameworks/netcontrol/__load__.bro.rst | 14 - .../netcontrol/catch-and-release.bro.rst | 343 - .../base/frameworks/netcontrol/drop.bro.rst | 140 - .../base/frameworks/netcontrol/index.rst | 81 - .../base/frameworks/netcontrol/main.bro.rst | 544 - .../frameworks/netcontrol/non-cluster.bro.rst | 16 - .../base/frameworks/netcontrol/plugin.bro.rst | 137 - .../netcontrol/plugins/__load__.bro.rst | 14 - .../netcontrol/plugins/acld.bro.rst | 162 - .../netcontrol/plugins/broker.bro.rst | 129 - .../netcontrol/plugins/debug.bro.rst | 37 - .../frameworks/netcontrol/plugins/index.rst | 36 - .../netcontrol/plugins/openflow.bro.rst | 158 - .../netcontrol/plugins/packetfilter.bro.rst | 34 - .../base/frameworks/netcontrol/shunt.bro.rst | 93 - .../base/frameworks/netcontrol/types.bro.rst | 296 - .../base/frameworks/notice/__load__.bro.rst | 14 - .../notice/actions/add-geodata.bro.rst | 45 - .../frameworks/notice/actions/drop.bro.rst | 25 - .../notice/actions/email_admin.bro.rst | 26 - .../frameworks/notice/actions/page.bro.rst | 41 - .../notice/actions/pp-alarms.bro.rst | 92 - doc/scripts/base/frameworks/notice/index.rst | 62 - .../base/frameworks/notice/main.bro.rst | 1063 -- .../base/frameworks/notice/weird.bro.rst | 412 - .../base/frameworks/openflow/__load__.bro.rst | 14 - .../base/frameworks/openflow/consts.bro.rst | 564 - .../base/frameworks/openflow/index.rst | 50 - .../base/frameworks/openflow/main.bro.rst | 265 - .../frameworks/openflow/non-cluster.bro.rst | 16 - .../openflow/plugins/__load__.bro.rst | 14 - .../openflow/plugins/broker.bro.rst | 72 - .../frameworks/openflow/plugins/index.rst | 23 - .../frameworks/openflow/plugins/log.bro.rst | 91 - .../frameworks/openflow/plugins/ryu.bro.rst | 50 - .../base/frameworks/openflow/types.bro.rst | 264 - .../frameworks/packet-filter/__load__.bro.rst | 14 - .../base/frameworks/packet-filter/index.rst | 27 - .../frameworks/packet-filter/main.bro.rst | 223 - .../frameworks/packet-filter/netstats.bro.rst | 40 - .../frameworks/packet-filter/utils.bro.rst | 73 - .../base/frameworks/reporter/__load__.bro.rst | 14 - .../base/frameworks/reporter/index.rst | 26 - .../base/frameworks/reporter/main.bro.rst | 64 - .../frameworks/signatures/__load__.bro.rst | 14 - .../base/frameworks/signatures/index.rst | 19 - .../base/frameworks/signatures/main.bro.rst | 272 - .../base/frameworks/software/__load__.bro.rst | 14 - .../base/frameworks/software/index.rst | 21 - .../base/frameworks/software/main.bro.rst | 359 - .../base/frameworks/sumstats/__load__.bro.rst | 14 - .../base/frameworks/sumstats/index.rst | 67 - .../base/frameworks/sumstats/main.bro.rst | 484 - .../frameworks/sumstats/non-cluster.bro.rst | 16 - .../sumstats/plugins/__load__.bro.rst | 14 - .../sumstats/plugins/average.bro.rst | 24 - .../sumstats/plugins/hll_unique.bro.rst | 26 - .../frameworks/sumstats/plugins/index.rst | 54 - .../frameworks/sumstats/plugins/last.bro.rst | 39 - .../frameworks/sumstats/plugins/max.bro.rst | 24 - .../frameworks/sumstats/plugins/min.bro.rst | 24 - .../sumstats/plugins/sample.bro.rst | 26 - .../sumstats/plugins/std-dev.bro.rst | 24 - .../frameworks/sumstats/plugins/sum.bro.rst | 24 - .../frameworks/sumstats/plugins/topk.bro.rst | 25 - .../sumstats/plugins/unique.bro.rst | 26 - .../sumstats/plugins/variance.bro.rst | 25 - .../base/frameworks/tunnels/__load__.bro.rst | 14 - doc/scripts/base/frameworks/tunnels/index.rst | 20 - .../base/frameworks/tunnels/main.bro.rst | 184 - doc/scripts/base/init-bare.bro.rst | 9395 ---------- doc/scripts/base/init-default.bro.rst | 19 - .../base/init-frameworks-and-bifs.bro.rst | 14 - .../misc/find-checksum-offloading.bro.rst | 38 - .../base/misc/find-filtered-trace.bro.rst | 37 - doc/scripts/base/misc/version.bro.rst | 119 - .../base/protocols/conn/__load__.bro.rst | 14 - .../base/protocols/conn/contents.bro.rst | 63 - .../base/protocols/conn/inactivity.bro.rst | 60 - doc/scripts/base/protocols/conn/index.rst | 51 - doc/scripts/base/protocols/conn/main.bro.rst | 201 - .../base/protocols/conn/polling.bro.rst | 51 - .../base/protocols/conn/thresholds.bro.rst | 172 - .../base/protocols/dce-rpc/__load__.bro.rst | 14 - .../base/protocols/dce-rpc/consts.bro.rst | 1433 -- doc/scripts/base/protocols/dce-rpc/index.rst | 17 - .../base/protocols/dce-rpc/main.bro.rst | 123 - .../base/protocols/dhcp/__load__.bro.rst | 14 - .../base/protocols/dhcp/consts.bro.rst | 227 - doc/scripts/base/protocols/dhcp/index.rst | 23 - doc/scripts/base/protocols/dhcp/main.bro.rst | 257 - .../base/protocols/dnp3/__load__.bro.rst | 14 - .../base/protocols/dnp3/consts.bro.rst | 73 - doc/scripts/base/protocols/dnp3/index.rst | 17 - doc/scripts/base/protocols/dnp3/main.bro.rst | 72 - .../base/protocols/dns/__load__.bro.rst | 14 - doc/scripts/base/protocols/dns/consts.bro.rst | 264 - doc/scripts/base/protocols/dns/index.rst | 20 - doc/scripts/base/protocols/dns/main.bro.rst | 268 - .../base/protocols/ftp/__load__.bro.rst | 14 - doc/scripts/base/protocols/ftp/files.bro.rst | 44 - .../base/protocols/ftp/gridftp.bro.rst | 129 - doc/scripts/base/protocols/ftp/index.rst | 54 - doc/scripts/base/protocols/ftp/info.bro.rst | 132 - doc/scripts/base/protocols/ftp/main.bro.rst | 128 - .../base/protocols/ftp/utils-commands.bro.rst | 397 - doc/scripts/base/protocols/ftp/utils.bro.rst | 57 - .../base/protocols/http/__load__.bro.rst | 14 - .../base/protocols/http/entities.bro.rst | 40 - doc/scripts/base/protocols/http/files.bro.rst | 37 - doc/scripts/base/protocols/http/index.rst | 27 - doc/scripts/base/protocols/http/main.bro.rst | 342 - doc/scripts/base/protocols/http/utils.bro.rst | 78 - .../base/protocols/imap/__load__.bro.rst | 14 - doc/scripts/base/protocols/imap/index.rst | 17 - doc/scripts/base/protocols/imap/main.bro.rst | 21 - .../base/protocols/irc/__load__.bro.rst | 14 - .../base/protocols/irc/dcc-send.bro.rst | 29 - doc/scripts/base/protocols/irc/files.bro.rst | 37 - doc/scripts/base/protocols/irc/index.rst | 29 - doc/scripts/base/protocols/irc/main.bro.rst | 99 - .../base/protocols/krb/__load__.bro.rst | 14 - doc/scripts/base/protocols/krb/consts.bro.rst | 134 - doc/scripts/base/protocols/krb/files.bro.rst | 43 - doc/scripts/base/protocols/krb/index.rst | 21 - doc/scripts/base/protocols/krb/main.bro.rst | 164 - .../base/protocols/modbus/__load__.bro.rst | 14 - .../base/protocols/modbus/consts.bro.rst | 101 - doc/scripts/base/protocols/modbus/index.rst | 17 - .../base/protocols/modbus/main.bro.rst | 73 - .../base/protocols/mysql/__load__.bro.rst | 14 - .../base/protocols/mysql/consts.bro.rst | 67 - doc/scripts/base/protocols/mysql/index.rst | 17 - doc/scripts/base/protocols/mysql/main.bro.rst | 77 - .../base/protocols/ntlm/__load__.bro.rst | 14 - doc/scripts/base/protocols/ntlm/index.rst | 13 - doc/scripts/base/protocols/ntlm/main.bro.rst | 71 - .../base/protocols/pop3/__load__.bro.rst | 13 - doc/scripts/base/protocols/pop3/index.rst | 10 - .../base/protocols/radius/__load__.bro.rst | 14 - .../base/protocols/radius/consts.bro.rst | 15 - doc/scripts/base/protocols/radius/index.rst | 17 - .../base/protocols/radius/main.bro.rst | 98 - .../base/protocols/rdp/__load__.bro.rst | 14 - doc/scripts/base/protocols/rdp/consts.bro.rst | 440 - doc/scripts/base/protocols/rdp/index.rst | 17 - doc/scripts/base/protocols/rdp/main.bro.rst | 159 - .../base/protocols/rfb/__load__.bro.rst | 14 - doc/scripts/base/protocols/rfb/index.rst | 13 - doc/scripts/base/protocols/rfb/main.bro.rst | 92 - .../base/protocols/sip/__load__.bro.rst | 14 - doc/scripts/base/protocols/sip/index.rst | 16 - doc/scripts/base/protocols/sip/main.bro.rst | 180 - .../base/protocols/smb/__load__.bro.rst | 14 - .../protocols/smb/const-dos-error.bro.rst | 22 - .../protocols/smb/const-nt-status.bro.rst | 22 - doc/scripts/base/protocols/smb/consts.bro.rst | 2388 --- doc/scripts/base/protocols/smb/files.bro.rst | 37 - doc/scripts/base/protocols/smb/index.rst | 31 - doc/scripts/base/protocols/smb/main.bro.rst | 289 - .../base/protocols/smb/smb1-main.bro.rst | 22 - .../base/protocols/smb/smb2-main.bro.rst | 22 - .../base/protocols/smtp/__load__.bro.rst | 14 - .../base/protocols/smtp/entities.bro.rst | 45 - doc/scripts/base/protocols/smtp/files.bro.rst | 43 - doc/scripts/base/protocols/smtp/index.rst | 20 - doc/scripts/base/protocols/smtp/main.bro.rst | 196 - .../base/protocols/snmp/__load__.bro.rst | 14 - doc/scripts/base/protocols/snmp/index.rst | 14 - doc/scripts/base/protocols/snmp/main.bro.rst | 123 - .../base/protocols/socks/__load__.bro.rst | 14 - .../base/protocols/socks/consts.bro.rst | 102 - doc/scripts/base/protocols/socks/index.rst | 16 - doc/scripts/base/protocols/socks/main.bro.rst | 108 - .../base/protocols/ssh/__load__.bro.rst | 14 - doc/scripts/base/protocols/ssh/index.rst | 14 - doc/scripts/base/protocols/ssh/main.bro.rst | 228 - .../base/protocols/ssl/__load__.bro.rst | 14 - doc/scripts/base/protocols/ssl/consts.bro.rst | 3549 ---- .../base/protocols/ssl/ct-list.bro.rst | 22 - doc/scripts/base/protocols/ssl/files.bro.rst | 43 - doc/scripts/base/protocols/ssl/index.rst | 27 - doc/scripts/base/protocols/ssl/main.bro.rst | 596 - .../protocols/ssl/mozilla-ca-list.bro.rst | 22 - .../base/protocols/syslog/__load__.bro.rst | 14 - .../base/protocols/syslog/consts.bro.rst | 83 - doc/scripts/base/protocols/syslog/index.rst | 19 - .../base/protocols/syslog/main.bro.rst | 61 - .../base/protocols/tunnels/__load__.bro.rst | 13 - doc/scripts/base/protocols/tunnels/index.rst | 11 - .../base/protocols/xmpp/__load__.bro.rst | 14 - doc/scripts/base/protocols/xmpp/index.rst | 17 - doc/scripts/base/protocols/xmpp/main.bro.rst | 21 - doc/scripts/base/utils/active-http.bro.rst | 116 - doc/scripts/base/utils/addrs.bro.rst | 190 - doc/scripts/base/utils/conn-ids.bro.rst | 54 - doc/scripts/base/utils/dir.bro.rst | 62 - .../base/utils/directions-and-hosts.bro.rst | 109 - doc/scripts/base/utils/email.bro.rst | 78 - doc/scripts/base/utils/exec.bro.rst | 91 - doc/scripts/base/utils/files.bro.rst | 39 - doc/scripts/base/utils/geoip-distance.bro.rst | 42 - doc/scripts/base/utils/hash_hrw.bro.rst | 92 - doc/scripts/base/utils/json.bro.rst | 39 - doc/scripts/base/utils/numbers.bro.rst | 37 - doc/scripts/base/utils/paths.bro.rst | 96 - doc/scripts/base/utils/patterns.bro.rst | 89 - doc/scripts/base/utils/queue.bro.rst | 164 - doc/scripts/base/utils/site.bro.rst | 188 - doc/scripts/base/utils/strings.bro.rst | 82 - doc/scripts/base/utils/thresholds.bro.rst | 107 - doc/scripts/base/utils/time.bro.rst | 29 - doc/scripts/base/utils/urls.bro.rst | 103 - doc/scripts/broxygen/__load__.bro.rst | 14 - doc/scripts/broxygen/example.bro.rst | 248 - doc/scripts/broxygen/index.rst | 37 - .../policy/files/x509/log-ocsp.bro.rst | 85 - .../frameworks/control/controllee.bro.rst | 24 - .../frameworks/control/controller.bro.rst | 23 - .../frameworks/dpd/detect-protocols.bro.rst | 125 - .../dpd/packet-segment-logging.bro.rst | 43 - .../frameworks/files/detect-MHR.bro.rst | 69 - .../files/entropy-test-all-files.bro.rst | 21 - .../files/extract-all-files.bro.rst | 15 - .../frameworks/files/hash-all-files.bro.rst | 15 - .../policy/frameworks/intel/do_expire.bro.rst | 23 - .../policy/frameworks/intel/do_notice.bro.rst | 24 - .../frameworks/intel/seen/__load__.bro.rst | 14 - .../intel/seen/conn-established.bro.rst | 14 - .../policy/frameworks/intel/seen/dns.bro.rst | 14 - .../frameworks/intel/seen/file-hashes.bro.rst | 14 - .../frameworks/intel/seen/file-names.bro.rst | 14 - .../intel/seen/http-headers.bro.rst | 14 - .../frameworks/intel/seen/http-url.bro.rst | 14 - .../policy/frameworks/intel/seen/index.rst | 46 - .../intel/seen/pubkey-hashes.bro.rst | 14 - .../intel/seen/smtp-url-extraction.bro.rst | 14 - .../policy/frameworks/intel/seen/smtp.bro.rst | 14 - .../policy/frameworks/intel/seen/ssl.bro.rst | 14 - .../intel/seen/where-locations.bro.rst | 20 - .../policy/frameworks/intel/seen/x509.bro.rst | 32 - .../policy/frameworks/intel/whitelist.bro.rst | 23 - .../policy/frameworks/notice/__load__.bro.rst | 14 - .../notice/extend-email/hostnames.bro.rst | 20 - .../policy/frameworks/notice/index.rst | 16 - .../frameworks/packet-filter/shunt.bro.rst | 99 - .../software/version-changes.bro.rst | 45 - .../frameworks/software/vulnerable.bro.rst | 99 - .../windows-version-detection.bro.rst | 89 - .../integration/barnyard2/__load__.bro.rst | 14 - .../policy/integration/barnyard2/index.rst | 21 - .../policy/integration/barnyard2/main.bro.rst | 65 - .../integration/barnyard2/types.bro.rst | 89 - .../collective-intel/__load__.bro.rst | 14 - .../integration/collective-intel/index.rst | 16 - .../integration/collective-intel/main.bro.rst | 23 - doc/scripts/policy/misc/capture-loss.bro.rst | 94 - .../misc/detect-traceroute/__load__.bro.rst | 14 - .../policy/misc/detect-traceroute/index.rst | 17 - .../misc/detect-traceroute/main.bro.rst | 114 - doc/scripts/policy/misc/dump-events.bro.rst | 49 - .../policy/misc/load-balancing.bro.rst | 58 - .../policy/misc/loaded-scripts.bro.rst | 41 - doc/scripts/policy/misc/profiling.bro.rst | 25 - doc/scripts/policy/misc/scan.bro.rst | 95 - doc/scripts/policy/misc/stats.bro.rst | 149 - .../policy/misc/trim-trace-file.bro.rst | 55 - doc/scripts/policy/misc/weird-stats.bro.rst | 75 - .../policy/protocols/conn/known-hosts.bro.rst | 176 - .../protocols/conn/known-services.bro.rst | 199 - .../policy/protocols/conn/mac-logging.bro.rst | 23 - .../protocols/conn/vlan-logging.bro.rst | 23 - .../policy/protocols/conn/weirds.bro.rst | 27 - .../protocols/dhcp/deprecated_events.bro.rst | 354 - .../policy/protocols/dhcp/msg-orig.bro.rst | 26 - .../policy/protocols/dhcp/software.bro.rst | 24 - .../policy/protocols/dhcp/sub-opts.bro.rst | 22 - .../policy/protocols/dns/auth-addl.bro.rst | 28 - .../dns/detect-external-names.bro.rst | 26 - .../protocols/ftp/detect-bruteforcing.bro.rst | 53 - .../policy/protocols/ftp/detect.bro.rst | 23 - .../policy/protocols/ftp/software.bro.rst | 23 - .../policy/protocols/http/detect-sqli.bro.rst | 98 - .../protocols/http/detect-webapps.bro.rst | 25 - .../protocols/http/header-names.bro.rst | 50 - .../http/software-browser-plugins.bro.rst | 25 - .../policy/protocols/http/software.bro.rst | 43 - .../http/var-extraction-cookies.bro.rst | 23 - .../protocols/http/var-extraction-uri.bro.rst | 24 - .../protocols/krb/ticket-logging.bro.rst | 23 - .../modbus/known-masters-slaves.bro.rst | 91 - .../protocols/modbus/track-memmap.bro.rst | 126 - .../policy/protocols/mysql/software.bro.rst | 23 - .../policy/protocols/rdp/indicate_ssl.bro.rst | 24 - .../policy/protocols/smb/__load__.bro.rst | 14 - doc/scripts/policy/protocols/smb/index.rst | 14 - .../policy/protocols/smb/log-cmds.bro.rst | 46 - .../policy/protocols/smtp/blocklists.bro.rst | 42 - .../smtp/detect-suspicious-orig.bro.rst | 49 - .../protocols/smtp/entities-excerpt.bro.rst | 43 - .../policy/protocols/smtp/software.bro.rst | 71 - .../protocols/ssh/detect-bruteforcing.bro.rst | 66 - .../policy/protocols/ssh/geo-data.bro.rst | 48 - .../ssh/interesting-hostnames.bro.rst | 47 - .../policy/protocols/ssh/software.bro.rst | 24 - .../protocols/ssl/expiring-certs.bro.rst | 57 - .../protocols/ssl/extract-certs-pem.bro.rst | 45 - .../policy/protocols/ssl/heartbleed.bro.rst | 25 - .../policy/protocols/ssl/known-certs.bro.rst | 195 - .../protocols/ssl/log-hostcerts-only.bro.rst | 25 - .../policy/protocols/ssl/notary.bro.rst | 60 - .../protocols/ssl/validate-certs.bro.rst | 99 - .../protocols/ssl/validate-ocsp.bro.rst | 24 - .../policy/protocols/ssl/validate-sct.bro.rst | 90 - .../policy/protocols/ssl/weak-keys.bro.rst | 94 - doc/scripts/policy/tuning/__load__.bro.rst | 15 - .../policy/tuning/defaults/__load__.bro.rst | 14 - .../defaults/extracted_file_limits.bro.rst | 20 - doc/scripts/policy/tuning/defaults/index.rst | 23 - .../tuning/defaults/packet-fragments.bro.rst | 20 - .../policy/tuning/defaults/warnings.bro.rst | 17 - doc/scripts/policy/tuning/index.rst | 34 - doc/scripts/policy/tuning/json-logs.bro.rst | 21 - .../policy/tuning/track-all-assets.bro.rst | 23 - doc/scripts/test-all-policy.bro.rst | 14 - testing/scripts/gen-broxygen-docs.sh | 10 +- 693 files changed, 26 insertions(+), 105609 deletions(-) delete mode 100644 .readthedocs.yml mode change 100644 => 120000 NEWS create mode 160000 doc delete mode 100644 doc/.gitignore delete mode 100644 doc/CMakeLists.txt delete mode 100644 doc/LICENSE delete mode 100644 doc/README delete mode 100644 doc/_templates/breadcrumbs.html delete mode 100644 doc/_templates/layout.html delete mode 100644 doc/cluster/index.rst delete mode 120000 doc/components/binpac/README.rst delete mode 120000 doc/components/bro-aux/README.rst delete mode 120000 doc/components/broctl/README.rst delete mode 120000 doc/components/broker/README.rst delete mode 120000 doc/components/btest/README.rst delete mode 120000 doc/components/capstats/README.rst delete mode 100644 doc/components/index.rst delete mode 120000 doc/components/pysubnettree/README.rst delete mode 120000 doc/components/trace-summary/README.rst delete mode 100644 doc/conf.py delete mode 100644 doc/configuration/index.rst delete mode 100644 doc/devel/plugins.rst delete mode 100644 doc/examples/httpmonitor/file_extraction.bro delete mode 100644 doc/examples/httpmonitor/http_proxy_01.bro delete mode 100644 doc/examples/httpmonitor/http_proxy_02.bro delete mode 100644 doc/examples/httpmonitor/http_proxy_03.bro delete mode 100644 doc/examples/httpmonitor/http_proxy_04.bro delete mode 100644 doc/examples/httpmonitor/index.rst delete mode 100644 doc/examples/ids/index.rst delete mode 100644 doc/examples/index.rst delete mode 100644 doc/examples/logs/index.rst delete mode 100644 doc/examples/mimestats/index.rst delete mode 100644 doc/examples/mimestats/mimestats.bro delete mode 100644 doc/examples/scripting/connection_record_01.bro delete mode 100644 doc/examples/scripting/connection_record_02.bro delete mode 100644 doc/examples/scripting/data_struct_record_01.bro delete mode 100644 doc/examples/scripting/data_struct_record_02.bro delete mode 100644 doc/examples/scripting/data_struct_set_declaration.bro delete mode 100644 doc/examples/scripting/data_struct_table_complex.bro delete mode 100644 doc/examples/scripting/data_struct_table_declaration.bro delete mode 100644 doc/examples/scripting/data_struct_vector.bro delete mode 100644 doc/examples/scripting/data_struct_vector_declaration.bro delete mode 100644 doc/examples/scripting/data_struct_vector_iter.bro delete mode 100644 doc/examples/scripting/data_type_const.bro delete mode 100644 doc/examples/scripting/data_type_const_simple.bro delete mode 100644 doc/examples/scripting/data_type_declaration.bro delete mode 100644 doc/examples/scripting/data_type_interval.bro delete mode 100644 doc/examples/scripting/data_type_local.bro delete mode 100644 doc/examples/scripting/data_type_pattern_01.bro delete mode 100644 doc/examples/scripting/data_type_pattern_02.bro delete mode 100644 doc/examples/scripting/data_type_record.bro delete mode 100644 doc/examples/scripting/data_type_subnets.bro delete mode 100644 doc/examples/scripting/data_type_time.bro delete mode 100644 doc/examples/scripting/framework_logging_factorial_01.bro delete mode 100644 doc/examples/scripting/framework_logging_factorial_02.bro delete mode 100644 doc/examples/scripting/framework_logging_factorial_03.bro delete mode 100644 doc/examples/scripting/framework_logging_factorial_04.bro delete mode 100644 doc/examples/scripting/framework_notice_hook_01.bro delete mode 100644 doc/examples/scripting/framework_notice_hook_suppression_01.bro delete mode 100644 doc/examples/scripting/framework_notice_shortcuts_01.bro delete mode 100644 doc/examples/scripting/framework_notice_shortcuts_02.bro delete mode 100644 doc/examples/scripting/http_main.bro delete mode 100644 doc/examples/scripting/index.rst delete mode 100644 doc/examples/scripting/using_bro_sandbox_01 delete mode 100644 doc/examples/scripting/using_bro_sandbox_02 delete mode 100644 doc/ext/bro.py delete mode 100644 doc/frameworks/broker.rst delete mode 100644 doc/frameworks/broker/cluster-layout.png delete mode 100644 doc/frameworks/broker/cluster-layout.xml delete mode 100644 doc/frameworks/broker/connecting-connector.bro delete mode 100644 doc/frameworks/broker/connecting-listener.bro delete mode 100644 doc/frameworks/broker/events-connector.bro delete mode 100644 doc/frameworks/broker/events-listener.bro delete mode 100644 doc/frameworks/broker/logs-connector.bro delete mode 100644 doc/frameworks/broker/logs-listener.bro delete mode 100644 doc/frameworks/broker/stores-connector.bro delete mode 100644 doc/frameworks/broker/stores-listener.bro delete mode 100644 doc/frameworks/broker/testlog.bro delete mode 100644 doc/frameworks/configuration.rst delete mode 100644 doc/frameworks/file-analysis.rst delete mode 100644 doc/frameworks/file_analysis_01.bro delete mode 100644 doc/frameworks/file_analysis_02.bro delete mode 100644 doc/frameworks/file_analysis_03.bro delete mode 100644 doc/frameworks/geoip.rst delete mode 100644 doc/frameworks/index.rst delete mode 100644 doc/frameworks/input.rst delete mode 100644 doc/frameworks/intel.rst delete mode 100644 doc/frameworks/logging-input-sqlite.rst delete mode 100644 doc/frameworks/logging.rst delete mode 100644 doc/frameworks/netcontrol-1-drop-with-debug.bro delete mode 100644 doc/frameworks/netcontrol-10-use-skeleton.bro delete mode 100644 doc/frameworks/netcontrol-2-ssh-guesser.bro delete mode 100644 doc/frameworks/netcontrol-3-ssh-guesser.bro delete mode 100644 doc/frameworks/netcontrol-4-drop.bro delete mode 100644 doc/frameworks/netcontrol-5-hook.bro delete mode 100644 doc/frameworks/netcontrol-6-find.bro delete mode 100644 doc/frameworks/netcontrol-7-catch-release.bro delete mode 100644 doc/frameworks/netcontrol-8-multiple.bro delete mode 100644 doc/frameworks/netcontrol-9-skeleton.bro delete mode 100644 doc/frameworks/netcontrol-architecture.png delete mode 100644 doc/frameworks/netcontrol-openflow.png delete mode 100644 doc/frameworks/netcontrol-rules.png delete mode 100644 doc/frameworks/netcontrol.rst delete mode 100644 doc/frameworks/notice.rst delete mode 100644 doc/frameworks/notice_ssh_guesser.bro delete mode 100644 doc/frameworks/signatures.rst delete mode 100644 doc/frameworks/sqlite-conn-filter.bro delete mode 100644 doc/frameworks/sqlite-read-events.bro delete mode 100644 doc/frameworks/sqlite-read-table.bro delete mode 100644 doc/frameworks/sumstats-countconns.bro delete mode 100644 doc/frameworks/sumstats-toy-scan.bro delete mode 100644 doc/frameworks/sumstats.rst delete mode 100644 doc/images/deployment.png delete mode 100644 doc/index.rst delete mode 120000 doc/install/CHANGES-binpac.txt delete mode 120000 doc/install/CHANGES-bro-aux.txt delete mode 120000 doc/install/CHANGES-bro.txt delete mode 120000 doc/install/CHANGES-broctl.txt delete mode 120000 doc/install/CHANGES-broker.txt delete mode 120000 doc/install/CHANGES-btest.txt delete mode 120000 doc/install/CHANGES-capstats.txt delete mode 120000 doc/install/CHANGES-pysubnettree.txt delete mode 120000 doc/install/CHANGES-trace-summary.txt delete mode 120000 doc/install/NEWS.rst delete mode 100644 doc/install/changes.rst delete mode 100644 doc/install/cross-compiling.rst delete mode 100644 doc/install/guidelines.rst delete mode 100644 doc/install/index.rst delete mode 100644 doc/install/install.rst delete mode 100644 doc/install/release-notes.rst delete mode 100644 doc/install/upgrade.rst delete mode 100644 doc/intro/architecture.png delete mode 100644 doc/intro/bro-eyes.png delete mode 100644 doc/intro/history.png delete mode 100644 doc/intro/index.rst delete mode 100644 doc/quickstart/conditional-notice.bro delete mode 100644 doc/quickstart/index.rst delete mode 100644 doc/script-reference/attributes.rst delete mode 100644 doc/script-reference/autogenerated-file-analyzer-index.rst delete mode 100644 doc/script-reference/autogenerated-package-index.rst delete mode 100644 doc/script-reference/autogenerated-protocol-analyzer-index.rst delete mode 100644 doc/script-reference/autogenerated-script-index.rst delete mode 100644 doc/script-reference/directives.rst delete mode 100644 doc/script-reference/file-analyzers.rst delete mode 100644 doc/script-reference/index.rst delete mode 100644 doc/script-reference/log-files.rst delete mode 100644 doc/script-reference/notices.rst delete mode 100644 doc/script-reference/operators.rst delete mode 100644 doc/script-reference/packages.rst delete mode 100644 doc/script-reference/proto-analyzers.rst delete mode 100644 doc/script-reference/scripts.rst delete mode 100644 doc/script-reference/statements.rst delete mode 100644 doc/script-reference/types.rst delete mode 100644 doc/scripts/base/bif/__load__.bro.rst delete mode 100644 doc/scripts/base/bif/analyzer.bif.bro.rst delete mode 100644 doc/scripts/base/bif/bloom-filter.bif.bro.rst delete mode 100644 doc/scripts/base/bif/bro.bif.bro.rst delete mode 100644 doc/scripts/base/bif/broxygen.bif.bro.rst delete mode 100644 doc/scripts/base/bif/cardinality-counter.bif.bro.rst delete mode 100644 doc/scripts/base/bif/comm.bif.bro.rst delete mode 100644 doc/scripts/base/bif/const.bif.bro.rst delete mode 100644 doc/scripts/base/bif/data.bif.bro.rst delete mode 100644 doc/scripts/base/bif/event.bif.bro.rst delete mode 100644 doc/scripts/base/bif/file_analysis.bif.bro.rst delete mode 100644 doc/scripts/base/bif/index.rst delete mode 100644 doc/scripts/base/bif/input.bif.bro.rst delete mode 100644 doc/scripts/base/bif/logging.bif.bro.rst delete mode 100644 doc/scripts/base/bif/messaging.bif.bro.rst delete mode 100644 doc/scripts/base/bif/option.bif.bro.rst delete mode 100644 doc/scripts/base/bif/pcap.bif.bro.rst delete mode 100644 doc/scripts/base/bif/plugins/Bro_ARP.events.bif.bro.rst delete mode 100644 doc/scripts/base/bif/plugins/Bro_AsciiReader.ascii.bif.bro.rst delete mode 100644 doc/scripts/base/bif/plugins/Bro_AsciiWriter.ascii.bif.bro.rst delete mode 100644 doc/scripts/base/bif/plugins/Bro_BackDoor.events.bif.bro.rst delete mode 100644 doc/scripts/base/bif/plugins/Bro_BenchmarkReader.benchmark.bif.bro.rst delete mode 100644 doc/scripts/base/bif/plugins/Bro_BinaryReader.binary.bif.bro.rst delete mode 100644 doc/scripts/base/bif/plugins/Bro_BitTorrent.events.bif.bro.rst delete mode 100644 doc/scripts/base/bif/plugins/Bro_ConfigReader.config.bif.bro.rst delete mode 100644 doc/scripts/base/bif/plugins/Bro_ConnSize.events.bif.bro.rst delete mode 100644 doc/scripts/base/bif/plugins/Bro_ConnSize.functions.bif.bro.rst delete mode 100644 doc/scripts/base/bif/plugins/Bro_DCE_RPC.consts.bif.bro.rst delete mode 100644 doc/scripts/base/bif/plugins/Bro_DCE_RPC.events.bif.bro.rst delete mode 100644 doc/scripts/base/bif/plugins/Bro_DCE_RPC.types.bif.bro.rst delete mode 100644 doc/scripts/base/bif/plugins/Bro_DHCP.events.bif.bro.rst delete mode 100644 doc/scripts/base/bif/plugins/Bro_DHCP.types.bif.bro.rst delete mode 100644 doc/scripts/base/bif/plugins/Bro_DNP3.events.bif.bro.rst delete mode 100644 doc/scripts/base/bif/plugins/Bro_DNS.events.bif.bro.rst delete mode 100644 doc/scripts/base/bif/plugins/Bro_FTP.events.bif.bro.rst delete mode 100644 doc/scripts/base/bif/plugins/Bro_FTP.functions.bif.bro.rst delete mode 100644 doc/scripts/base/bif/plugins/Bro_File.events.bif.bro.rst delete mode 100644 doc/scripts/base/bif/plugins/Bro_FileEntropy.events.bif.bro.rst delete mode 100644 doc/scripts/base/bif/plugins/Bro_FileExtract.events.bif.bro.rst delete mode 100644 doc/scripts/base/bif/plugins/Bro_FileExtract.functions.bif.bro.rst delete mode 100644 doc/scripts/base/bif/plugins/Bro_FileHash.events.bif.bro.rst delete mode 100644 doc/scripts/base/bif/plugins/Bro_Finger.events.bif.bro.rst delete mode 100644 doc/scripts/base/bif/plugins/Bro_GSSAPI.events.bif.bro.rst delete mode 100644 doc/scripts/base/bif/plugins/Bro_GTPv1.events.bif.bro.rst delete mode 100644 doc/scripts/base/bif/plugins/Bro_Gnutella.events.bif.bro.rst delete mode 100644 doc/scripts/base/bif/plugins/Bro_HTTP.events.bif.bro.rst delete mode 100644 doc/scripts/base/bif/plugins/Bro_HTTP.functions.bif.bro.rst delete mode 100644 doc/scripts/base/bif/plugins/Bro_ICMP.events.bif.bro.rst delete mode 100644 doc/scripts/base/bif/plugins/Bro_IMAP.events.bif.bro.rst delete mode 100644 doc/scripts/base/bif/plugins/Bro_IRC.events.bif.bro.rst delete mode 100644 doc/scripts/base/bif/plugins/Bro_Ident.events.bif.bro.rst delete mode 100644 doc/scripts/base/bif/plugins/Bro_InterConn.events.bif.bro.rst delete mode 100644 doc/scripts/base/bif/plugins/Bro_KRB.events.bif.bro.rst delete mode 100644 doc/scripts/base/bif/plugins/Bro_KRB.types.bif.bro.rst delete mode 100644 doc/scripts/base/bif/plugins/Bro_Login.events.bif.bro.rst delete mode 100644 doc/scripts/base/bif/plugins/Bro_Login.functions.bif.bro.rst delete mode 100644 doc/scripts/base/bif/plugins/Bro_MIME.events.bif.bro.rst delete mode 100644 doc/scripts/base/bif/plugins/Bro_Modbus.events.bif.bro.rst delete mode 100644 doc/scripts/base/bif/plugins/Bro_MySQL.events.bif.bro.rst delete mode 100644 doc/scripts/base/bif/plugins/Bro_NCP.consts.bif.bro.rst delete mode 100644 doc/scripts/base/bif/plugins/Bro_NCP.events.bif.bro.rst delete mode 100644 doc/scripts/base/bif/plugins/Bro_NTLM.events.bif.bro.rst delete mode 100644 doc/scripts/base/bif/plugins/Bro_NTLM.types.bif.bro.rst delete mode 100644 doc/scripts/base/bif/plugins/Bro_NTP.events.bif.bro.rst delete mode 100644 doc/scripts/base/bif/plugins/Bro_NetBIOS.events.bif.bro.rst delete mode 100644 doc/scripts/base/bif/plugins/Bro_NetBIOS.functions.bif.bro.rst delete mode 100644 doc/scripts/base/bif/plugins/Bro_NoneWriter.none.bif.bro.rst delete mode 100644 doc/scripts/base/bif/plugins/Bro_PE.events.bif.bro.rst delete mode 100644 doc/scripts/base/bif/plugins/Bro_POP3.events.bif.bro.rst delete mode 100644 doc/scripts/base/bif/plugins/Bro_RADIUS.events.bif.bro.rst delete mode 100644 doc/scripts/base/bif/plugins/Bro_RDP.events.bif.bro.rst delete mode 100644 doc/scripts/base/bif/plugins/Bro_RDP.types.bif.bro.rst delete mode 100644 doc/scripts/base/bif/plugins/Bro_RFB.events.bif.bro.rst delete mode 100644 doc/scripts/base/bif/plugins/Bro_RPC.events.bif.bro.rst delete mode 100644 doc/scripts/base/bif/plugins/Bro_RawReader.raw.bif.bro.rst delete mode 100644 doc/scripts/base/bif/plugins/Bro_SIP.events.bif.bro.rst delete mode 100644 doc/scripts/base/bif/plugins/Bro_SMB.consts.bif.bro.rst delete mode 100644 doc/scripts/base/bif/plugins/Bro_SMB.events.bif.bro.rst delete mode 100644 doc/scripts/base/bif/plugins/Bro_SMB.smb1_com_check_directory.bif.bro.rst delete mode 100644 doc/scripts/base/bif/plugins/Bro_SMB.smb1_com_close.bif.bro.rst delete mode 100644 doc/scripts/base/bif/plugins/Bro_SMB.smb1_com_create_directory.bif.bro.rst delete mode 100644 doc/scripts/base/bif/plugins/Bro_SMB.smb1_com_echo.bif.bro.rst delete mode 100644 doc/scripts/base/bif/plugins/Bro_SMB.smb1_com_logoff_andx.bif.bro.rst delete mode 100644 doc/scripts/base/bif/plugins/Bro_SMB.smb1_com_negotiate.bif.bro.rst delete mode 100644 doc/scripts/base/bif/plugins/Bro_SMB.smb1_com_nt_cancel.bif.bro.rst delete mode 100644 doc/scripts/base/bif/plugins/Bro_SMB.smb1_com_nt_create_andx.bif.bro.rst delete mode 100644 doc/scripts/base/bif/plugins/Bro_SMB.smb1_com_query_information.bif.bro.rst delete mode 100644 doc/scripts/base/bif/plugins/Bro_SMB.smb1_com_read_andx.bif.bro.rst delete mode 100644 doc/scripts/base/bif/plugins/Bro_SMB.smb1_com_session_setup_andx.bif.bro.rst delete mode 100644 doc/scripts/base/bif/plugins/Bro_SMB.smb1_com_transaction.bif.bro.rst delete mode 100644 doc/scripts/base/bif/plugins/Bro_SMB.smb1_com_transaction2.bif.bro.rst delete mode 100644 doc/scripts/base/bif/plugins/Bro_SMB.smb1_com_transaction2_secondary.bif.bro.rst delete mode 100644 doc/scripts/base/bif/plugins/Bro_SMB.smb1_com_transaction_secondary.bif.bro.rst delete mode 100644 doc/scripts/base/bif/plugins/Bro_SMB.smb1_com_tree_connect_andx.bif.bro.rst delete mode 100644 doc/scripts/base/bif/plugins/Bro_SMB.smb1_com_tree_disconnect.bif.bro.rst delete mode 100644 doc/scripts/base/bif/plugins/Bro_SMB.smb1_com_write_andx.bif.bro.rst delete mode 100644 doc/scripts/base/bif/plugins/Bro_SMB.smb1_events.bif.bro.rst delete mode 100644 doc/scripts/base/bif/plugins/Bro_SMB.smb2_com_close.bif.bro.rst delete mode 100644 doc/scripts/base/bif/plugins/Bro_SMB.smb2_com_create.bif.bro.rst delete mode 100644 doc/scripts/base/bif/plugins/Bro_SMB.smb2_com_negotiate.bif.bro.rst delete mode 100644 doc/scripts/base/bif/plugins/Bro_SMB.smb2_com_read.bif.bro.rst delete mode 100644 doc/scripts/base/bif/plugins/Bro_SMB.smb2_com_session_setup.bif.bro.rst delete mode 100644 doc/scripts/base/bif/plugins/Bro_SMB.smb2_com_set_info.bif.bro.rst delete mode 100644 doc/scripts/base/bif/plugins/Bro_SMB.smb2_com_tree_connect.bif.bro.rst delete mode 100644 doc/scripts/base/bif/plugins/Bro_SMB.smb2_com_tree_disconnect.bif.bro.rst delete mode 100644 doc/scripts/base/bif/plugins/Bro_SMB.smb2_com_write.bif.bro.rst delete mode 100644 doc/scripts/base/bif/plugins/Bro_SMB.smb2_events.bif.bro.rst delete mode 100644 doc/scripts/base/bif/plugins/Bro_SMB.types.bif.bro.rst delete mode 100644 doc/scripts/base/bif/plugins/Bro_SMTP.events.bif.bro.rst delete mode 100644 doc/scripts/base/bif/plugins/Bro_SMTP.functions.bif.bro.rst delete mode 100644 doc/scripts/base/bif/plugins/Bro_SNMP.events.bif.bro.rst delete mode 100644 doc/scripts/base/bif/plugins/Bro_SNMP.types.bif.bro.rst delete mode 100644 doc/scripts/base/bif/plugins/Bro_SOCKS.events.bif.bro.rst delete mode 100644 doc/scripts/base/bif/plugins/Bro_SQLiteReader.sqlite.bif.bro.rst delete mode 100644 doc/scripts/base/bif/plugins/Bro_SQLiteWriter.sqlite.bif.bro.rst delete mode 100644 doc/scripts/base/bif/plugins/Bro_SSH.events.bif.bro.rst delete mode 100644 doc/scripts/base/bif/plugins/Bro_SSH.types.bif.bro.rst delete mode 100644 doc/scripts/base/bif/plugins/Bro_SSL.events.bif.bro.rst delete mode 100644 doc/scripts/base/bif/plugins/Bro_SSL.functions.bif.bro.rst delete mode 100644 doc/scripts/base/bif/plugins/Bro_SSL.types.bif.bro.rst delete mode 100644 doc/scripts/base/bif/plugins/Bro_SteppingStone.events.bif.bro.rst delete mode 100644 doc/scripts/base/bif/plugins/Bro_Syslog.events.bif.bro.rst delete mode 100644 doc/scripts/base/bif/plugins/Bro_TCP.events.bif.bro.rst delete mode 100644 doc/scripts/base/bif/plugins/Bro_TCP.functions.bif.bro.rst delete mode 100644 doc/scripts/base/bif/plugins/Bro_Teredo.events.bif.bro.rst delete mode 100644 doc/scripts/base/bif/plugins/Bro_UDP.events.bif.bro.rst delete mode 100644 doc/scripts/base/bif/plugins/Bro_Unified2.events.bif.bro.rst delete mode 100644 doc/scripts/base/bif/plugins/Bro_Unified2.types.bif.bro.rst delete mode 100644 doc/scripts/base/bif/plugins/Bro_X509.events.bif.bro.rst delete mode 100644 doc/scripts/base/bif/plugins/Bro_X509.functions.bif.bro.rst delete mode 100644 doc/scripts/base/bif/plugins/Bro_X509.ocsp_events.bif.bro.rst delete mode 100644 doc/scripts/base/bif/plugins/Bro_X509.types.bif.bro.rst delete mode 100644 doc/scripts/base/bif/plugins/Bro_XMPP.events.bif.bro.rst delete mode 100644 doc/scripts/base/bif/plugins/__load__.bro.rst delete mode 100644 doc/scripts/base/bif/plugins/index.rst delete mode 100644 doc/scripts/base/bif/reporter.bif.bro.rst delete mode 100644 doc/scripts/base/bif/stats.bif.bro.rst delete mode 100644 doc/scripts/base/bif/store.bif.bro.rst delete mode 100644 doc/scripts/base/bif/strings.bif.bro.rst delete mode 100644 doc/scripts/base/bif/top-k.bif.bro.rst delete mode 100644 doc/scripts/base/bif/types.bif.bro.rst delete mode 100644 doc/scripts/base/files/extract/__load__.bro.rst delete mode 100644 doc/scripts/base/files/extract/index.rst delete mode 100644 doc/scripts/base/files/extract/main.bro.rst delete mode 100644 doc/scripts/base/files/hash/__load__.bro.rst delete mode 100644 doc/scripts/base/files/hash/index.rst delete mode 100644 doc/scripts/base/files/hash/main.bro.rst delete mode 100644 doc/scripts/base/files/pe/__load__.bro.rst delete mode 100644 doc/scripts/base/files/pe/consts.bro.rst delete mode 100644 doc/scripts/base/files/pe/index.rst delete mode 100644 doc/scripts/base/files/pe/main.bro.rst delete mode 100644 doc/scripts/base/files/unified2/__load__.bro.rst delete mode 100644 doc/scripts/base/files/unified2/index.rst delete mode 100644 doc/scripts/base/files/unified2/main.bro.rst delete mode 100644 doc/scripts/base/files/x509/__load__.bro.rst delete mode 100644 doc/scripts/base/files/x509/index.rst delete mode 100644 doc/scripts/base/files/x509/main.bro.rst delete mode 100644 doc/scripts/base/frameworks/analyzer/__load__.bro.rst delete mode 100644 doc/scripts/base/frameworks/analyzer/index.rst delete mode 100644 doc/scripts/base/frameworks/analyzer/main.bro.rst delete mode 100644 doc/scripts/base/frameworks/broker/__load__.bro.rst delete mode 100644 doc/scripts/base/frameworks/broker/index.rst delete mode 100644 doc/scripts/base/frameworks/broker/log.bro.rst delete mode 100644 doc/scripts/base/frameworks/broker/main.bro.rst delete mode 100644 doc/scripts/base/frameworks/broker/store.bro.rst delete mode 100644 doc/scripts/base/frameworks/cluster/__load__.bro.rst delete mode 100644 doc/scripts/base/frameworks/cluster/index.rst delete mode 100644 doc/scripts/base/frameworks/cluster/main.bro.rst delete mode 100644 doc/scripts/base/frameworks/cluster/pools.bro.rst delete mode 100644 doc/scripts/base/frameworks/config/__load__.bro.rst delete mode 100644 doc/scripts/base/frameworks/config/index.rst delete mode 100644 doc/scripts/base/frameworks/config/input.bro.rst delete mode 100644 doc/scripts/base/frameworks/config/main.bro.rst delete mode 100644 doc/scripts/base/frameworks/config/weird.bro.rst delete mode 100644 doc/scripts/base/frameworks/control/__load__.bro.rst delete mode 100644 doc/scripts/base/frameworks/control/index.rst delete mode 100644 doc/scripts/base/frameworks/control/main.bro.rst delete mode 100644 doc/scripts/base/frameworks/dpd/__load__.bro.rst delete mode 100644 doc/scripts/base/frameworks/dpd/index.rst delete mode 100644 doc/scripts/base/frameworks/dpd/main.bro.rst delete mode 100644 doc/scripts/base/frameworks/files/__load__.bro.rst delete mode 100644 doc/scripts/base/frameworks/files/index.rst delete mode 100644 doc/scripts/base/frameworks/files/magic/__load__.bro.rst delete mode 100644 doc/scripts/base/frameworks/files/magic/index.rst delete mode 100644 doc/scripts/base/frameworks/files/main.bro.rst delete mode 100644 doc/scripts/base/frameworks/input/__load__.bro.rst delete mode 100644 doc/scripts/base/frameworks/input/index.rst delete mode 100644 doc/scripts/base/frameworks/input/main.bro.rst delete mode 100644 doc/scripts/base/frameworks/input/readers/ascii.bro.rst delete mode 100644 doc/scripts/base/frameworks/input/readers/benchmark.bro.rst delete mode 100644 doc/scripts/base/frameworks/input/readers/binary.bro.rst delete mode 100644 doc/scripts/base/frameworks/input/readers/config.bro.rst delete mode 100644 doc/scripts/base/frameworks/input/readers/raw.bro.rst delete mode 100644 doc/scripts/base/frameworks/input/readers/sqlite.bro.rst delete mode 100644 doc/scripts/base/frameworks/intel/__load__.bro.rst delete mode 100644 doc/scripts/base/frameworks/intel/files.bro.rst delete mode 100644 doc/scripts/base/frameworks/intel/index.rst delete mode 100644 doc/scripts/base/frameworks/intel/input.bro.rst delete mode 100644 doc/scripts/base/frameworks/intel/main.bro.rst delete mode 100644 doc/scripts/base/frameworks/logging/__load__.bro.rst delete mode 100644 doc/scripts/base/frameworks/logging/index.rst delete mode 100644 doc/scripts/base/frameworks/logging/main.bro.rst delete mode 100644 doc/scripts/base/frameworks/logging/postprocessors/__load__.bro.rst delete mode 100644 doc/scripts/base/frameworks/logging/postprocessors/index.rst delete mode 100644 doc/scripts/base/frameworks/logging/postprocessors/scp.bro.rst delete mode 100644 doc/scripts/base/frameworks/logging/postprocessors/sftp.bro.rst delete mode 100644 doc/scripts/base/frameworks/logging/writers/ascii.bro.rst delete mode 100644 doc/scripts/base/frameworks/logging/writers/none.bro.rst delete mode 100644 doc/scripts/base/frameworks/logging/writers/sqlite.bro.rst delete mode 100644 doc/scripts/base/frameworks/netcontrol/__load__.bro.rst delete mode 100644 doc/scripts/base/frameworks/netcontrol/catch-and-release.bro.rst delete mode 100644 doc/scripts/base/frameworks/netcontrol/drop.bro.rst delete mode 100644 doc/scripts/base/frameworks/netcontrol/index.rst delete mode 100644 doc/scripts/base/frameworks/netcontrol/main.bro.rst delete mode 100644 doc/scripts/base/frameworks/netcontrol/non-cluster.bro.rst delete mode 100644 doc/scripts/base/frameworks/netcontrol/plugin.bro.rst delete mode 100644 doc/scripts/base/frameworks/netcontrol/plugins/__load__.bro.rst delete mode 100644 doc/scripts/base/frameworks/netcontrol/plugins/acld.bro.rst delete mode 100644 doc/scripts/base/frameworks/netcontrol/plugins/broker.bro.rst delete mode 100644 doc/scripts/base/frameworks/netcontrol/plugins/debug.bro.rst delete mode 100644 doc/scripts/base/frameworks/netcontrol/plugins/index.rst delete mode 100644 doc/scripts/base/frameworks/netcontrol/plugins/openflow.bro.rst delete mode 100644 doc/scripts/base/frameworks/netcontrol/plugins/packetfilter.bro.rst delete mode 100644 doc/scripts/base/frameworks/netcontrol/shunt.bro.rst delete mode 100644 doc/scripts/base/frameworks/netcontrol/types.bro.rst delete mode 100644 doc/scripts/base/frameworks/notice/__load__.bro.rst delete mode 100644 doc/scripts/base/frameworks/notice/actions/add-geodata.bro.rst delete mode 100644 doc/scripts/base/frameworks/notice/actions/drop.bro.rst delete mode 100644 doc/scripts/base/frameworks/notice/actions/email_admin.bro.rst delete mode 100644 doc/scripts/base/frameworks/notice/actions/page.bro.rst delete mode 100644 doc/scripts/base/frameworks/notice/actions/pp-alarms.bro.rst delete mode 100644 doc/scripts/base/frameworks/notice/index.rst delete mode 100644 doc/scripts/base/frameworks/notice/main.bro.rst delete mode 100644 doc/scripts/base/frameworks/notice/weird.bro.rst delete mode 100644 doc/scripts/base/frameworks/openflow/__load__.bro.rst delete mode 100644 doc/scripts/base/frameworks/openflow/consts.bro.rst delete mode 100644 doc/scripts/base/frameworks/openflow/index.rst delete mode 100644 doc/scripts/base/frameworks/openflow/main.bro.rst delete mode 100644 doc/scripts/base/frameworks/openflow/non-cluster.bro.rst delete mode 100644 doc/scripts/base/frameworks/openflow/plugins/__load__.bro.rst delete mode 100644 doc/scripts/base/frameworks/openflow/plugins/broker.bro.rst delete mode 100644 doc/scripts/base/frameworks/openflow/plugins/index.rst delete mode 100644 doc/scripts/base/frameworks/openflow/plugins/log.bro.rst delete mode 100644 doc/scripts/base/frameworks/openflow/plugins/ryu.bro.rst delete mode 100644 doc/scripts/base/frameworks/openflow/types.bro.rst delete mode 100644 doc/scripts/base/frameworks/packet-filter/__load__.bro.rst delete mode 100644 doc/scripts/base/frameworks/packet-filter/index.rst delete mode 100644 doc/scripts/base/frameworks/packet-filter/main.bro.rst delete mode 100644 doc/scripts/base/frameworks/packet-filter/netstats.bro.rst delete mode 100644 doc/scripts/base/frameworks/packet-filter/utils.bro.rst delete mode 100644 doc/scripts/base/frameworks/reporter/__load__.bro.rst delete mode 100644 doc/scripts/base/frameworks/reporter/index.rst delete mode 100644 doc/scripts/base/frameworks/reporter/main.bro.rst delete mode 100644 doc/scripts/base/frameworks/signatures/__load__.bro.rst delete mode 100644 doc/scripts/base/frameworks/signatures/index.rst delete mode 100644 doc/scripts/base/frameworks/signatures/main.bro.rst delete mode 100644 doc/scripts/base/frameworks/software/__load__.bro.rst delete mode 100644 doc/scripts/base/frameworks/software/index.rst delete mode 100644 doc/scripts/base/frameworks/software/main.bro.rst delete mode 100644 doc/scripts/base/frameworks/sumstats/__load__.bro.rst delete mode 100644 doc/scripts/base/frameworks/sumstats/index.rst delete mode 100644 doc/scripts/base/frameworks/sumstats/main.bro.rst delete mode 100644 doc/scripts/base/frameworks/sumstats/non-cluster.bro.rst delete mode 100644 doc/scripts/base/frameworks/sumstats/plugins/__load__.bro.rst delete mode 100644 doc/scripts/base/frameworks/sumstats/plugins/average.bro.rst delete mode 100644 doc/scripts/base/frameworks/sumstats/plugins/hll_unique.bro.rst delete mode 100644 doc/scripts/base/frameworks/sumstats/plugins/index.rst delete mode 100644 doc/scripts/base/frameworks/sumstats/plugins/last.bro.rst delete mode 100644 doc/scripts/base/frameworks/sumstats/plugins/max.bro.rst delete mode 100644 doc/scripts/base/frameworks/sumstats/plugins/min.bro.rst delete mode 100644 doc/scripts/base/frameworks/sumstats/plugins/sample.bro.rst delete mode 100644 doc/scripts/base/frameworks/sumstats/plugins/std-dev.bro.rst delete mode 100644 doc/scripts/base/frameworks/sumstats/plugins/sum.bro.rst delete mode 100644 doc/scripts/base/frameworks/sumstats/plugins/topk.bro.rst delete mode 100644 doc/scripts/base/frameworks/sumstats/plugins/unique.bro.rst delete mode 100644 doc/scripts/base/frameworks/sumstats/plugins/variance.bro.rst delete mode 100644 doc/scripts/base/frameworks/tunnels/__load__.bro.rst delete mode 100644 doc/scripts/base/frameworks/tunnels/index.rst delete mode 100644 doc/scripts/base/frameworks/tunnels/main.bro.rst delete mode 100644 doc/scripts/base/init-bare.bro.rst delete mode 100644 doc/scripts/base/init-default.bro.rst delete mode 100644 doc/scripts/base/init-frameworks-and-bifs.bro.rst delete mode 100644 doc/scripts/base/misc/find-checksum-offloading.bro.rst delete mode 100644 doc/scripts/base/misc/find-filtered-trace.bro.rst delete mode 100644 doc/scripts/base/misc/version.bro.rst delete mode 100644 doc/scripts/base/protocols/conn/__load__.bro.rst delete mode 100644 doc/scripts/base/protocols/conn/contents.bro.rst delete mode 100644 doc/scripts/base/protocols/conn/inactivity.bro.rst delete mode 100644 doc/scripts/base/protocols/conn/index.rst delete mode 100644 doc/scripts/base/protocols/conn/main.bro.rst delete mode 100644 doc/scripts/base/protocols/conn/polling.bro.rst delete mode 100644 doc/scripts/base/protocols/conn/thresholds.bro.rst delete mode 100644 doc/scripts/base/protocols/dce-rpc/__load__.bro.rst delete mode 100644 doc/scripts/base/protocols/dce-rpc/consts.bro.rst delete mode 100644 doc/scripts/base/protocols/dce-rpc/index.rst delete mode 100644 doc/scripts/base/protocols/dce-rpc/main.bro.rst delete mode 100644 doc/scripts/base/protocols/dhcp/__load__.bro.rst delete mode 100644 doc/scripts/base/protocols/dhcp/consts.bro.rst delete mode 100644 doc/scripts/base/protocols/dhcp/index.rst delete mode 100644 doc/scripts/base/protocols/dhcp/main.bro.rst delete mode 100644 doc/scripts/base/protocols/dnp3/__load__.bro.rst delete mode 100644 doc/scripts/base/protocols/dnp3/consts.bro.rst delete mode 100644 doc/scripts/base/protocols/dnp3/index.rst delete mode 100644 doc/scripts/base/protocols/dnp3/main.bro.rst delete mode 100644 doc/scripts/base/protocols/dns/__load__.bro.rst delete mode 100644 doc/scripts/base/protocols/dns/consts.bro.rst delete mode 100644 doc/scripts/base/protocols/dns/index.rst delete mode 100644 doc/scripts/base/protocols/dns/main.bro.rst delete mode 100644 doc/scripts/base/protocols/ftp/__load__.bro.rst delete mode 100644 doc/scripts/base/protocols/ftp/files.bro.rst delete mode 100644 doc/scripts/base/protocols/ftp/gridftp.bro.rst delete mode 100644 doc/scripts/base/protocols/ftp/index.rst delete mode 100644 doc/scripts/base/protocols/ftp/info.bro.rst delete mode 100644 doc/scripts/base/protocols/ftp/main.bro.rst delete mode 100644 doc/scripts/base/protocols/ftp/utils-commands.bro.rst delete mode 100644 doc/scripts/base/protocols/ftp/utils.bro.rst delete mode 100644 doc/scripts/base/protocols/http/__load__.bro.rst delete mode 100644 doc/scripts/base/protocols/http/entities.bro.rst delete mode 100644 doc/scripts/base/protocols/http/files.bro.rst delete mode 100644 doc/scripts/base/protocols/http/index.rst delete mode 100644 doc/scripts/base/protocols/http/main.bro.rst delete mode 100644 doc/scripts/base/protocols/http/utils.bro.rst delete mode 100644 doc/scripts/base/protocols/imap/__load__.bro.rst delete mode 100644 doc/scripts/base/protocols/imap/index.rst delete mode 100644 doc/scripts/base/protocols/imap/main.bro.rst delete mode 100644 doc/scripts/base/protocols/irc/__load__.bro.rst delete mode 100644 doc/scripts/base/protocols/irc/dcc-send.bro.rst delete mode 100644 doc/scripts/base/protocols/irc/files.bro.rst delete mode 100644 doc/scripts/base/protocols/irc/index.rst delete mode 100644 doc/scripts/base/protocols/irc/main.bro.rst delete mode 100644 doc/scripts/base/protocols/krb/__load__.bro.rst delete mode 100644 doc/scripts/base/protocols/krb/consts.bro.rst delete mode 100644 doc/scripts/base/protocols/krb/files.bro.rst delete mode 100644 doc/scripts/base/protocols/krb/index.rst delete mode 100644 doc/scripts/base/protocols/krb/main.bro.rst delete mode 100644 doc/scripts/base/protocols/modbus/__load__.bro.rst delete mode 100644 doc/scripts/base/protocols/modbus/consts.bro.rst delete mode 100644 doc/scripts/base/protocols/modbus/index.rst delete mode 100644 doc/scripts/base/protocols/modbus/main.bro.rst delete mode 100644 doc/scripts/base/protocols/mysql/__load__.bro.rst delete mode 100644 doc/scripts/base/protocols/mysql/consts.bro.rst delete mode 100644 doc/scripts/base/protocols/mysql/index.rst delete mode 100644 doc/scripts/base/protocols/mysql/main.bro.rst delete mode 100644 doc/scripts/base/protocols/ntlm/__load__.bro.rst delete mode 100644 doc/scripts/base/protocols/ntlm/index.rst delete mode 100644 doc/scripts/base/protocols/ntlm/main.bro.rst delete mode 100644 doc/scripts/base/protocols/pop3/__load__.bro.rst delete mode 100644 doc/scripts/base/protocols/pop3/index.rst delete mode 100644 doc/scripts/base/protocols/radius/__load__.bro.rst delete mode 100644 doc/scripts/base/protocols/radius/consts.bro.rst delete mode 100644 doc/scripts/base/protocols/radius/index.rst delete mode 100644 doc/scripts/base/protocols/radius/main.bro.rst delete mode 100644 doc/scripts/base/protocols/rdp/__load__.bro.rst delete mode 100644 doc/scripts/base/protocols/rdp/consts.bro.rst delete mode 100644 doc/scripts/base/protocols/rdp/index.rst delete mode 100644 doc/scripts/base/protocols/rdp/main.bro.rst delete mode 100644 doc/scripts/base/protocols/rfb/__load__.bro.rst delete mode 100644 doc/scripts/base/protocols/rfb/index.rst delete mode 100644 doc/scripts/base/protocols/rfb/main.bro.rst delete mode 100644 doc/scripts/base/protocols/sip/__load__.bro.rst delete mode 100644 doc/scripts/base/protocols/sip/index.rst delete mode 100644 doc/scripts/base/protocols/sip/main.bro.rst delete mode 100644 doc/scripts/base/protocols/smb/__load__.bro.rst delete mode 100644 doc/scripts/base/protocols/smb/const-dos-error.bro.rst delete mode 100644 doc/scripts/base/protocols/smb/const-nt-status.bro.rst delete mode 100644 doc/scripts/base/protocols/smb/consts.bro.rst delete mode 100644 doc/scripts/base/protocols/smb/files.bro.rst delete mode 100644 doc/scripts/base/protocols/smb/index.rst delete mode 100644 doc/scripts/base/protocols/smb/main.bro.rst delete mode 100644 doc/scripts/base/protocols/smb/smb1-main.bro.rst delete mode 100644 doc/scripts/base/protocols/smb/smb2-main.bro.rst delete mode 100644 doc/scripts/base/protocols/smtp/__load__.bro.rst delete mode 100644 doc/scripts/base/protocols/smtp/entities.bro.rst delete mode 100644 doc/scripts/base/protocols/smtp/files.bro.rst delete mode 100644 doc/scripts/base/protocols/smtp/index.rst delete mode 100644 doc/scripts/base/protocols/smtp/main.bro.rst delete mode 100644 doc/scripts/base/protocols/snmp/__load__.bro.rst delete mode 100644 doc/scripts/base/protocols/snmp/index.rst delete mode 100644 doc/scripts/base/protocols/snmp/main.bro.rst delete mode 100644 doc/scripts/base/protocols/socks/__load__.bro.rst delete mode 100644 doc/scripts/base/protocols/socks/consts.bro.rst delete mode 100644 doc/scripts/base/protocols/socks/index.rst delete mode 100644 doc/scripts/base/protocols/socks/main.bro.rst delete mode 100644 doc/scripts/base/protocols/ssh/__load__.bro.rst delete mode 100644 doc/scripts/base/protocols/ssh/index.rst delete mode 100644 doc/scripts/base/protocols/ssh/main.bro.rst delete mode 100644 doc/scripts/base/protocols/ssl/__load__.bro.rst delete mode 100644 doc/scripts/base/protocols/ssl/consts.bro.rst delete mode 100644 doc/scripts/base/protocols/ssl/ct-list.bro.rst delete mode 100644 doc/scripts/base/protocols/ssl/files.bro.rst delete mode 100644 doc/scripts/base/protocols/ssl/index.rst delete mode 100644 doc/scripts/base/protocols/ssl/main.bro.rst delete mode 100644 doc/scripts/base/protocols/ssl/mozilla-ca-list.bro.rst delete mode 100644 doc/scripts/base/protocols/syslog/__load__.bro.rst delete mode 100644 doc/scripts/base/protocols/syslog/consts.bro.rst delete mode 100644 doc/scripts/base/protocols/syslog/index.rst delete mode 100644 doc/scripts/base/protocols/syslog/main.bro.rst delete mode 100644 doc/scripts/base/protocols/tunnels/__load__.bro.rst delete mode 100644 doc/scripts/base/protocols/tunnels/index.rst delete mode 100644 doc/scripts/base/protocols/xmpp/__load__.bro.rst delete mode 100644 doc/scripts/base/protocols/xmpp/index.rst delete mode 100644 doc/scripts/base/protocols/xmpp/main.bro.rst delete mode 100644 doc/scripts/base/utils/active-http.bro.rst delete mode 100644 doc/scripts/base/utils/addrs.bro.rst delete mode 100644 doc/scripts/base/utils/conn-ids.bro.rst delete mode 100644 doc/scripts/base/utils/dir.bro.rst delete mode 100644 doc/scripts/base/utils/directions-and-hosts.bro.rst delete mode 100644 doc/scripts/base/utils/email.bro.rst delete mode 100644 doc/scripts/base/utils/exec.bro.rst delete mode 100644 doc/scripts/base/utils/files.bro.rst delete mode 100644 doc/scripts/base/utils/geoip-distance.bro.rst delete mode 100644 doc/scripts/base/utils/hash_hrw.bro.rst delete mode 100644 doc/scripts/base/utils/json.bro.rst delete mode 100644 doc/scripts/base/utils/numbers.bro.rst delete mode 100644 doc/scripts/base/utils/paths.bro.rst delete mode 100644 doc/scripts/base/utils/patterns.bro.rst delete mode 100644 doc/scripts/base/utils/queue.bro.rst delete mode 100644 doc/scripts/base/utils/site.bro.rst delete mode 100644 doc/scripts/base/utils/strings.bro.rst delete mode 100644 doc/scripts/base/utils/thresholds.bro.rst delete mode 100644 doc/scripts/base/utils/time.bro.rst delete mode 100644 doc/scripts/base/utils/urls.bro.rst delete mode 100644 doc/scripts/broxygen/__load__.bro.rst delete mode 100644 doc/scripts/broxygen/example.bro.rst delete mode 100644 doc/scripts/broxygen/index.rst delete mode 100644 doc/scripts/policy/files/x509/log-ocsp.bro.rst delete mode 100644 doc/scripts/policy/frameworks/control/controllee.bro.rst delete mode 100644 doc/scripts/policy/frameworks/control/controller.bro.rst delete mode 100644 doc/scripts/policy/frameworks/dpd/detect-protocols.bro.rst delete mode 100644 doc/scripts/policy/frameworks/dpd/packet-segment-logging.bro.rst delete mode 100644 doc/scripts/policy/frameworks/files/detect-MHR.bro.rst delete mode 100644 doc/scripts/policy/frameworks/files/entropy-test-all-files.bro.rst delete mode 100644 doc/scripts/policy/frameworks/files/extract-all-files.bro.rst delete mode 100644 doc/scripts/policy/frameworks/files/hash-all-files.bro.rst delete mode 100644 doc/scripts/policy/frameworks/intel/do_expire.bro.rst delete mode 100644 doc/scripts/policy/frameworks/intel/do_notice.bro.rst delete mode 100644 doc/scripts/policy/frameworks/intel/seen/__load__.bro.rst delete mode 100644 doc/scripts/policy/frameworks/intel/seen/conn-established.bro.rst delete mode 100644 doc/scripts/policy/frameworks/intel/seen/dns.bro.rst delete mode 100644 doc/scripts/policy/frameworks/intel/seen/file-hashes.bro.rst delete mode 100644 doc/scripts/policy/frameworks/intel/seen/file-names.bro.rst delete mode 100644 doc/scripts/policy/frameworks/intel/seen/http-headers.bro.rst delete mode 100644 doc/scripts/policy/frameworks/intel/seen/http-url.bro.rst delete mode 100644 doc/scripts/policy/frameworks/intel/seen/index.rst delete mode 100644 doc/scripts/policy/frameworks/intel/seen/pubkey-hashes.bro.rst delete mode 100644 doc/scripts/policy/frameworks/intel/seen/smtp-url-extraction.bro.rst delete mode 100644 doc/scripts/policy/frameworks/intel/seen/smtp.bro.rst delete mode 100644 doc/scripts/policy/frameworks/intel/seen/ssl.bro.rst delete mode 100644 doc/scripts/policy/frameworks/intel/seen/where-locations.bro.rst delete mode 100644 doc/scripts/policy/frameworks/intel/seen/x509.bro.rst delete mode 100644 doc/scripts/policy/frameworks/intel/whitelist.bro.rst delete mode 100644 doc/scripts/policy/frameworks/notice/__load__.bro.rst delete mode 100644 doc/scripts/policy/frameworks/notice/extend-email/hostnames.bro.rst delete mode 100644 doc/scripts/policy/frameworks/notice/index.rst delete mode 100644 doc/scripts/policy/frameworks/packet-filter/shunt.bro.rst delete mode 100644 doc/scripts/policy/frameworks/software/version-changes.bro.rst delete mode 100644 doc/scripts/policy/frameworks/software/vulnerable.bro.rst delete mode 100644 doc/scripts/policy/frameworks/software/windows-version-detection.bro.rst delete mode 100644 doc/scripts/policy/integration/barnyard2/__load__.bro.rst delete mode 100644 doc/scripts/policy/integration/barnyard2/index.rst delete mode 100644 doc/scripts/policy/integration/barnyard2/main.bro.rst delete mode 100644 doc/scripts/policy/integration/barnyard2/types.bro.rst delete mode 100644 doc/scripts/policy/integration/collective-intel/__load__.bro.rst delete mode 100644 doc/scripts/policy/integration/collective-intel/index.rst delete mode 100644 doc/scripts/policy/integration/collective-intel/main.bro.rst delete mode 100644 doc/scripts/policy/misc/capture-loss.bro.rst delete mode 100644 doc/scripts/policy/misc/detect-traceroute/__load__.bro.rst delete mode 100644 doc/scripts/policy/misc/detect-traceroute/index.rst delete mode 100644 doc/scripts/policy/misc/detect-traceroute/main.bro.rst delete mode 100644 doc/scripts/policy/misc/dump-events.bro.rst delete mode 100644 doc/scripts/policy/misc/load-balancing.bro.rst delete mode 100644 doc/scripts/policy/misc/loaded-scripts.bro.rst delete mode 100644 doc/scripts/policy/misc/profiling.bro.rst delete mode 100644 doc/scripts/policy/misc/scan.bro.rst delete mode 100644 doc/scripts/policy/misc/stats.bro.rst delete mode 100644 doc/scripts/policy/misc/trim-trace-file.bro.rst delete mode 100644 doc/scripts/policy/misc/weird-stats.bro.rst delete mode 100644 doc/scripts/policy/protocols/conn/known-hosts.bro.rst delete mode 100644 doc/scripts/policy/protocols/conn/known-services.bro.rst delete mode 100644 doc/scripts/policy/protocols/conn/mac-logging.bro.rst delete mode 100644 doc/scripts/policy/protocols/conn/vlan-logging.bro.rst delete mode 100644 doc/scripts/policy/protocols/conn/weirds.bro.rst delete mode 100644 doc/scripts/policy/protocols/dhcp/deprecated_events.bro.rst delete mode 100644 doc/scripts/policy/protocols/dhcp/msg-orig.bro.rst delete mode 100644 doc/scripts/policy/protocols/dhcp/software.bro.rst delete mode 100644 doc/scripts/policy/protocols/dhcp/sub-opts.bro.rst delete mode 100644 doc/scripts/policy/protocols/dns/auth-addl.bro.rst delete mode 100644 doc/scripts/policy/protocols/dns/detect-external-names.bro.rst delete mode 100644 doc/scripts/policy/protocols/ftp/detect-bruteforcing.bro.rst delete mode 100644 doc/scripts/policy/protocols/ftp/detect.bro.rst delete mode 100644 doc/scripts/policy/protocols/ftp/software.bro.rst delete mode 100644 doc/scripts/policy/protocols/http/detect-sqli.bro.rst delete mode 100644 doc/scripts/policy/protocols/http/detect-webapps.bro.rst delete mode 100644 doc/scripts/policy/protocols/http/header-names.bro.rst delete mode 100644 doc/scripts/policy/protocols/http/software-browser-plugins.bro.rst delete mode 100644 doc/scripts/policy/protocols/http/software.bro.rst delete mode 100644 doc/scripts/policy/protocols/http/var-extraction-cookies.bro.rst delete mode 100644 doc/scripts/policy/protocols/http/var-extraction-uri.bro.rst delete mode 100644 doc/scripts/policy/protocols/krb/ticket-logging.bro.rst delete mode 100644 doc/scripts/policy/protocols/modbus/known-masters-slaves.bro.rst delete mode 100644 doc/scripts/policy/protocols/modbus/track-memmap.bro.rst delete mode 100644 doc/scripts/policy/protocols/mysql/software.bro.rst delete mode 100644 doc/scripts/policy/protocols/rdp/indicate_ssl.bro.rst delete mode 100644 doc/scripts/policy/protocols/smb/__load__.bro.rst delete mode 100644 doc/scripts/policy/protocols/smb/index.rst delete mode 100644 doc/scripts/policy/protocols/smb/log-cmds.bro.rst delete mode 100644 doc/scripts/policy/protocols/smtp/blocklists.bro.rst delete mode 100644 doc/scripts/policy/protocols/smtp/detect-suspicious-orig.bro.rst delete mode 100644 doc/scripts/policy/protocols/smtp/entities-excerpt.bro.rst delete mode 100644 doc/scripts/policy/protocols/smtp/software.bro.rst delete mode 100644 doc/scripts/policy/protocols/ssh/detect-bruteforcing.bro.rst delete mode 100644 doc/scripts/policy/protocols/ssh/geo-data.bro.rst delete mode 100644 doc/scripts/policy/protocols/ssh/interesting-hostnames.bro.rst delete mode 100644 doc/scripts/policy/protocols/ssh/software.bro.rst delete mode 100644 doc/scripts/policy/protocols/ssl/expiring-certs.bro.rst delete mode 100644 doc/scripts/policy/protocols/ssl/extract-certs-pem.bro.rst delete mode 100644 doc/scripts/policy/protocols/ssl/heartbleed.bro.rst delete mode 100644 doc/scripts/policy/protocols/ssl/known-certs.bro.rst delete mode 100644 doc/scripts/policy/protocols/ssl/log-hostcerts-only.bro.rst delete mode 100644 doc/scripts/policy/protocols/ssl/notary.bro.rst delete mode 100644 doc/scripts/policy/protocols/ssl/validate-certs.bro.rst delete mode 100644 doc/scripts/policy/protocols/ssl/validate-ocsp.bro.rst delete mode 100644 doc/scripts/policy/protocols/ssl/validate-sct.bro.rst delete mode 100644 doc/scripts/policy/protocols/ssl/weak-keys.bro.rst delete mode 100644 doc/scripts/policy/tuning/__load__.bro.rst delete mode 100644 doc/scripts/policy/tuning/defaults/__load__.bro.rst delete mode 100644 doc/scripts/policy/tuning/defaults/extracted_file_limits.bro.rst delete mode 100644 doc/scripts/policy/tuning/defaults/index.rst delete mode 100644 doc/scripts/policy/tuning/defaults/packet-fragments.bro.rst delete mode 100644 doc/scripts/policy/tuning/defaults/warnings.bro.rst delete mode 100644 doc/scripts/policy/tuning/index.rst delete mode 100644 doc/scripts/policy/tuning/json-logs.bro.rst delete mode 100644 doc/scripts/policy/tuning/track-all-assets.bro.rst delete mode 100644 doc/scripts/test-all-policy.bro.rst diff --git a/.gitmodules b/.gitmodules index fc8176a6a6..5efc3b0fb8 100644 --- a/.gitmodules +++ b/.gitmodules @@ -28,3 +28,6 @@ [submodule "aux/bifcl"] path = aux/bifcl url = https://github.com/zeek/bifcl +[submodule "doc"] + path = doc + url = https://github.com/zeek/zeek-docs diff --git a/.readthedocs.yml b/.readthedocs.yml deleted file mode 100644 index 19b6eca259..0000000000 --- a/.readthedocs.yml +++ /dev/null @@ -1,4 +0,0 @@ -formats: [] - -python: - version: 3 diff --git a/CHANGES b/CHANGES index 2b67f7404a..ecf41099c1 100644 --- a/CHANGES +++ b/CHANGES @@ -1,4 +1,10 @@ +2.6-82 | 2019-01-17 14:09:29 -0600 + + * Change doc/ subdir into a git submodule (Jon Siwek, Corelight) + + The docs now live at https://github.com/zeek/zeek-docs + 2.6-81 | 2019-01-16 19:03:07 -0600 * Add Broker::peer_counts_as_iosource option (Jon Siwek, Corelight) diff --git a/CMakeLists.txt b/CMakeLists.txt index 8171ad5bf4..7781f547e9 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -307,7 +307,6 @@ include_directories(BEFORE ${CAF_INCLUDE_DIR_OPENSSL}) add_subdirectory(src) add_subdirectory(scripts) -add_subdirectory(doc) add_subdirectory(man) include(CheckOptionalBuildSources) diff --git a/Makefile b/Makefile index 21d99d8fca..8e9d77e3cf 100644 --- a/Makefile +++ b/Makefile @@ -23,15 +23,14 @@ install-aux: configured clean: configured docclean $(MAKE) -C $(BUILD) $@ -doc: configured - $(MAKE) -C $(BUILD) $@ +doc: + $(MAKE) -C doc $@ -docclean: configured - $(MAKE) -C $(BUILD) $@ +docclean: + (cd doc && make clean) livehtml: - @mkdir -p build/doc/html - sphinx-autobuild --ignore "testing/*" --ignore "*.git/*" --ignore "*.lock" --ignore "*.pyc" --ignore "*.swp" --ignore "*.swpx" --ignore "*.swx" -b html ./doc ./build/doc/html + $(MAKE) -C doc $@ dist: @test -e ../$(VERSION_FULL) && rm -ri ../$(VERSION_FULL) || true diff --git a/NEWS b/NEWS deleted file mode 100644 index 1c5bde47dc..0000000000 --- a/NEWS +++ /dev/null @@ -1,2371 +0,0 @@ - -This document summarizes the most important changes in the current Bro -release. For an exhaustive list of changes, see the ``CHANGES`` file -(note that submodules, such as Broker, come with their own ``CHANGES``.) - -Bro 2.7 -======= - -New Functionality ------------------ - -- Added support for DNSSEC resource records RRSIG, DNSKEY, DS, NSEC, and NSEC3. - The associated events are: - - - dns_RRSIG - - dns_DNSKEY - - dns_DS - - dns_NSEC - - dns_NSEC3 - -- Bro's Plugin framework now allows a patch version. If a patch version is not - provided, it will default to 0. To specify this, modify the plugin - Configuration class in your ``src/Plugin.cc` and set - ``config.version.patch``. Note that the default plugin skeleton - includes a unit test whose Baseline has the plugin version number in - it and that will now fail due to the version number now including a - patch number. For those that want to keep the unit test, simply adapt - the unit test/baseline to include the new plugin patch number. - -Changed Functionality ---------------------- - -- The for-loop index variable for vectors has been changed from - 'int' to 'count' type. It's unlikely this would alter/break any - script behavior unless they were explicitly inspecting the variable's - type (and there's typically no reason to do that). - -Removed Functionality ---------------------- - -Deprecated Functionality ------------------------- - -Bro 2.6 -======= - -New Functionality ------------------ - -- Bro has switched to using the new Broker library for all its - communication. Broker's API has been completely redesigned (compared - to the version in 2.5), and much of its implementation has been - redone. There's a new script-level "broker" framework that - supersedes the old "communication" framework, which is now - deprecated. All scripts that ship with Bro have been ported to use - Broker. BroControl has likewise been ported to use Broker. - - For more about the new Broker framework, see - https://www.bro.org/sphinx-git/frameworks/broker.html. There's also - a guide there for porting existing Bro scripts to Broker. For more - about Broker itself, including its API for external applications, - see https://bro-broker.readthedocs.io/en/stable - - When using BroControl, the function of proxies has changed with - Broker. If you are upgrading and have configured more than one proxy - currenty, we recommend going back down to a single proxy node now. - That should be fine unless you are using custom scripts doing - significant data distribution through the new cluster framework. - - A side effect of the switch to using Broker is that each Bro node now runs - as a single process instead of two. Also, the number of file descriptors - being polled in Bro's main event loop has been reduced (1 per worker - versus 5). This should increase the number of workers one can - use before reaching the common 1024 file descriptor limitation of - "select()". - -- Bro now has new "is" and "as" script operators for dynamic - type-checking and casting. - - - "v as T" casts a value v into a value of type T, assuming that's - possible (if not, it triggers a runtime error). - - - "v is T" returns a boolean indicating whether value v can be - casted into type T (i.e., if true then "v as T" will succeed). - - This casting supports three cases currently: (1) a value of - declared type "any" can be casted to its actual underlying type; - (2) Broker values can be casted to their corresponding script - types; and (3) all values can be casted to their declared types - (i.e., a no-op). - - Example for "any":: - - # cat a.bro - function check(a: any) - { - local s: string = "default"; - - if ( a is string ) - s = (a as string); - - print fmt("s=%s", s); - } - - event bro_init() - { - check("Foo"); - check(1); - } - - # bro a.bro - s=Foo - s=default - -- The existing "switch" statement got extended to now also support switching by - type rather than value. The new syntax supports two type-based versions - of "case": - - - "case type T: ...": Take branch if operand can be casted to type T. - - - "case type T as x: ... ": Take branch if operand can be casted - to type T, and make the casted value available through ID "x". - - Multiple types can be listed per branch, separated by commas. - However, one cannot mix cases with expressions and types inside a - single switch statement. - - Example:: - - function switch_one(v: any) - { - switch (v) { - case type string: - print "It's a string!"; - break; - - case type count as c: - print "It's a count!", c; - break; - - case type bool, type addr: - print "It's a bool or address!"; - break; - - default: - print "Something else!"; - break; - } - } - -- Bro now comes with a new "configuration framework" that allows - updating script options dynamically at runtime. This functionality - consists of three larger pieces working together: - - - Option variables: The new "option" keyword allows variables to be - declared as runtime options. Such variables cannot be changed - using normal assignments. Instead, they can be changed using the - new function "Config::set_value". This function will automatically - apply the change to all nodes in a cluster. Note that options can also - be changed using the new function "Option::set", but this function will - not send the change to any other nodes, so Config::set_value should - typically be used instead of Option::set. - - Various redef-able constants in the standard Bro scripts have - been converted to runtime options. This change will not affect any - user scripts because the initial value of runtime options can still be - redefined with a "redef" declaration. Example:: - - option testvar = "old value"; - redef testvar = "new value"; - - It is possible to "subscribe" to an option through - "Option::set_change_handler", which will trigger a handler callback - when an option changes. Change handlers can optionally modify - values before they are applied by returning the desired value, or - reject updates by returning the old value. Priorities can be - specified if there are several handlers for one option. - - Example script:: - - option testbool: bool = T; - - function option_changed(ID: string, new_value: bool): bool - { - print fmt("Value of %s changed from %s to %s", ID, testbool, new_value); - return new_value; - } - - event bro_init() - { - print "Old value", testbool; - Option::set_change_handler("testbool", option_changed); - Option::set("testbool", F); - print "New value", testbool; - } - - - Script-level configuration framework: The new script framework - base/framework/config facilitates reading in new option values - from external files at runtime. The format for these files looks - like this:: - - [option name][tab/spaces][new variable value] - - Configuration files to read can be specified by adding them to - "Config::config_files". - - Usage example:: - - redef Config::config_files += { "/path/to/config.dat" }; - - module TestConfig; - - export { - option testbool: bool = F; - } - - The specified file will now be monitored continuously for changes, so - that writing "TestConfig::testbool T" into ``/path/to/config.dat`` will - automatically update the option's value accordingly. - - The configuration framework creates a ``config.log`` that shows all - value changes that took place. - - - Config reader: Internally, the configuration framework uses a new - type of input reader to read such configuration files into Bro. - The reader uses the option name to look up the type that variable - has, converts the read value to the correct type, and then updates - the option's value. Example script use:: - - type Idx: record { - option_name: string; - }; - - type Val: record { - option_val: string; - }; - - global currconfig: table[string] of string = table(); - - event InputConfig::new_value(name: string, source: string, id: string, value: any) - { - print id, value; - } - - event bro_init() - { - Input::add_table([$reader=Input::READER_CONFIG, $source="../configfile", $name="configuration", $idx=Idx, $val=Val, $destination=currconfig, $want_record=F]); - } - -- Support for OCSP and Signed Certificate Timestamp. This adds the - following events and BIFs: - - - Events: - - - ocsp_request - - ocsp_request_certificate - - ocsp_response_status - - ocsp_response_bytes - - ocsp_response_certificate - - ocsp_extension - - x509_ocsp_ext_signed_certificate_timestamp - - ssl_extension_signed_certificate_timestamp - - - Functions: - - - sct_verify - - x509_subject_name_hash - - x509_issuer_name_hash - - x509_spki_hash - -- The SSL scripts provide a new hook "ssl_finishing(c: connection)" - to trigger actions after the handshake has concluded. - -- New functionality has been added to the TLS parser, adding several - events. These events mostly extract information from the server and client - key exchange messages. The new events are: - - - ssl_ecdh_server_params - - ssl_dh_server_params - - ssl_server_signature - - ssl_ecdh_client_params - - ssl_dh_client_params - - ssl_rsa_client_pms - - Since "ssl_ecdh_server_params" contains more information than the old - "ssl_server_curve" event, "ssl_server_curve" is now marked as deprecated. - -- The "ssl_application_data" event was retired and replaced with - "ssl_plaintext_data". - -- Some SSL events were changed and now provide additional data. These events - are: - - - ssl_client_hello - - ssl_server_hello - - ssl_encrypted_data - - If you use these events, you can make your scripts work on old and new - versions of Bro by wrapping the event definition in an "@if", for example:: - - @if ( Version::at_least("2.6") || ( Version::number == 20500 && Version::info$commit >= 944 ) ) - event ssl_client_hello(c: connection, version: count, record_version: count, possible_ts: time, client_random: string, session_id: string, ciphers: index_vec, comp_methods: index_vec) - @else - event ssl_client_hello(c: connection, version: count, possible_ts: time, client_random: string, session_id: string, ciphers: index_vec) - @endif - -- Functions for retrieving files by their ID have been added: - - - Files::file_exists - - Files::lookup_File - -- New functions in the logging API: - - - Log::get_filter_names - - Log::enable_stream - -- HTTP now recognizes and skips upgraded/websocket connections. A new event, - "http_connection_upgrade", is raised in such cases. - -- A new hook, HTTP::sqli_policy, may be used to whitelist requests that - could otherwise be counted as SQL injection attempts. - -- Added a MOUNT3 protocol parser - - - This is not enabled by default (no ports are registered and no - DPD signatures exist, so no connections will end up attaching the - new Mount analyzer). If it were to be activated by users, the - following events are available: - - - mount_proc_null - - mount_proc_mnt - - mount_proc_umnt - - mount_proc_umnt_all - - mount_proc_not_implemented - - mount_reply_status - -- Added new NFS events: - - - nfs_proc_symlink - - nfs_proc_link - - nfs_proc_sattr - -- The SMB scripts in ``policy/protocols/smb`` are now moved into - ``base/protocols/smb`` and loaded/enabled by default. If you previously - loaded these scripts from their ``policy/`` location (in local.bro or - other custom scripts) you may now remove/change those although they - should still work since ``policy/protocols/smb`` is simply a placeholder - script that redirects to the new ``base/`` location. - -- Added new SMB events: - - - smb1_transaction_secondary_request - - smb1_transaction2_secondary_request - - smb1_transaction_response - -- Bro can now decrypt Kerberos tickets, and retrieve the authentication from - them, given a suitable keytab file. - -- Added support for bitwise operations on "count" values. '&', '|' and - '^' are binary "and", "or" and "xor" operators, and '~' is a unary - ones-complement operator. - -- The '&' and '|' operators can apply to patterns, too. p1 & p2 yields - a pattern that represents matching p1 followed by p2, and p1 | p2 yields - a pattern representing matching p1 or p2. The p1 | p2 functionality was - semi-present in previous versions of Bro, but required constants as - its operands; now you can use any pattern-valued expressions. - -- You can now specify that a pattern matches in a case-insensitive - fashion by adding 'i' to the end of its specification. So for example - /fOO/i == "Foo" yields T, as does /fOO/i in "xFoObar". - - You can achieve the same functionality for a subpattern enclosed in - parentheses by adding "?i:" to the open parenthesis. So for example - /foo|(?i:bar)/ will match "BaR", but not "FoO". - - For both ways of specifying case-insensitivity, characters enclosed in - double quotes remain case-sensitive. So for example /"foo"/i will not - match "Foo", but it will match "foo". - -- "make install" now installs Bro's include headers (and more) into - "--prefix" so that compiling plugins no longer needs access to a - source/build tree. For OS distributions, this also facilitates - creating "bro-devel" packages providing all files necessary to build - plugins. - -- Bro now supports PPPoE over QinQ. - -- Bro now supports OpenSSL 1.1. - -- The new connection/conn.log history character 'W' indicates that - the originator ('w' = responder) advertised a TCP zero window - (instructing the peer to not send any data until receiving a - non-zero window). - -- The connection/conn.log history characters 'C' (checksum error seen), - 'T' (retransmission seen), and 'W' (zero window advertised) are now - repeated in a logarithmic fashion upon seeing multiple instances - of the corresponding behavior. Thus a connection with 2 C's in its - history means that the originator sent >= 10 packets with checksum - errors; 3 C's means >= 100, etc. - -- The above connection history behaviors occurring multiple times - (i.e., starting at 10 instances, than again for 100 instances, - etc.) generate corresponding events: - - - tcp_multiple_checksum_errors - - udp_multiple_checksum_errors - - tcp_multiple_zero_windows - - tcp_multiple_retransmissions - - Each has the same form, e.g.:: - - event tcp_multiple_retransmissions(c: connection, is_orig: bool, - threshold: count); - -- Added support for set union, intersection, difference, and comparison - operations. The corresponding operators for the first three are - "s1 | s2", "s1 & s2", and "s1 - s2". Relationals are in terms - of subsets, so "s1 < s2" yields true if s1 is a proper subset of s2 - and "s1 == s2" if the two sets have exactly the same elements. - "s1 <= s2" holds for subsets or equality, and similarly "s1 != s2", - "s1 > s2", and "s1 >= s2" have the expected meanings in terms - of non-equality, proper superset, and superset-or-equal. - -- An expression of the form "v += e" will append the value of the expression - "e" to the end of the vector "v" (of course assuming type-compatibility). - "redef v += { a, b, c }" will similarly extend a vector previously declared - with &redef by appending the result of expressions "a", "b", and "c" to - the vector at initialization-time. - -- A new "@deprecated" directive was added. It marks a script-file as - deprecated. - -Changed Functionality ---------------------- - -- All communication is now handled through Broker, requiring changes - to existing scripts to port them over to the new API. The Broker - framework documentation comes with a porting guide. - -- The DHCP analyzer and its script-layer interface have been rewritten. - - - Supports more DHCP options than before. - - - The DHCP log now represents DHCP sessions based on transaction ID - and works on Bro cluster deployments. - - - Removed the ``policy/protocols/dhcp/known-devices-and-hostnames.bro`` - script since it's generally less relevant now with the updated log. - - - Removed the ``base/protocols/dhcp/utils.bro`` script and thus the - "reverse_ip" function. - - - Replaced all DHCP events with the single "dhcp_message" event. - The list of removed events includes: - - - dhcp_discover - - dhcp_offer - - dhcp_request - - dhcp_decline - - dhcp_ack - - dhcp_nak - - dhcp_release - - dhcp_inform - - - A new script, ``policy/protocols/dhcp/deprecated_events.bro``, may be - loaded to aid those transitioning away from the list of "removed" - events above. The script provides definitions for the old events - and automatically generates them from a "dhcp_message" handler, thus - providing equivalent functionality to the previous Bro release. - Such usage emits deprecation warnings. - -- Removed ``policy/misc/known-devices.bro`` script and thus - ``known_devices.log`` will no longer be created. - -- The "--with-binpac" configure option has changed to mean "path - to the binpac executable" instead of "path to binpac installation root". - -- The MIME types used to identify X.509 certificates in SSL - connections changed from "application/pkix-cert" to - "application/x-x509-user-cert" for host certificates and - "application/x-x509-ca-cert" for CA certificates. - -- The "ssl_server_curve" event is considered deprecated and will be removed - in the future. See the new "ssl_ecdh_server_params" event for a - replacement. - -- The Socks analyzer no longer logs passwords by default. This - brings its behavior in line with the FTP/HTTP analyzers which also - do not log passwords by default. - - To restore the previous behavior and log Socks passwords, use:: - - redef SOCKS::default_capture_password = T; - -- The DNS base scripts no longer generate some noisy and annoying - weirds: - - - dns_unmatched_msg - - dns_unmatched_msg_quantity - - dns_unmatched_reply - -- The "tunnel_parents" field of ``conn.log`` is now marked ``&optional``, so, - in the default configuration of logs, this field will show "-" - instead of "(empty)" for connections that lack any tunneling. - -- SMB event argument changes: - - - "smb1_transaction_request" now has two additional arguments, "parameters" - and "data" strings - - - "smb1_transaction2_request" now has an additional "args" record argument - -- The "SMB::write_cmd_log" option has been removed and the corresponding - logic moving to ``policy/protocols/smb/log-cmds.bro`` which can simply - be loaded to produce the same effect of toggling the old flag on. - -- SSL event argument changes: - - - "ssl_server_signature" now has an additional argument - "signature_and_hashalgorithm". - -- The "dnp3_header_block" event no longer has the "start" parameter. - -- The "string_to_pattern()" and now-deprecated "merge_pattern()" - built-ins are no longer restricted to only be called at initialization time. - -- GeoIP Legacy Database support has been replaced with GeoIP2 MaxMind DB - format support. - - - This updates the "lookup_location" and "lookup_asn" BIFs to use - libmaxminddb. The motivation for this is that MaxMind is discontinuing - GeoLite Legacy databases: no updates after April 1, 2018, no downloads - after January 2, 2019. It's also noted that all GeoIP Legacy databases - may be discontinued as they are superseded by GeoIP2. - -- "Weird" events are now generally suppressed/sampled by default according to - some tunable parameters: - - - Weird::sampling_whitelist - - Weird::sampling_threshold - - Weird::sampling_rate - - Weird::sampling_duration - - Those options can be changed if one needs the previous behavior of - a "net_weird", "flow_weird", or "conn_weird" event being raised for - every single event. - - The original ``weird.log`` may not differ much with these changes, - except in the cases where a particular weird type exceeds the - sampling threshold. - - Otherwise, there is a new ``weird_stats.log`` generated via - ``policy/misc/weird-stats.bro`` which contains concise summaries - of weird counts per type per time period. - -- Improved DCE-RPC analysis via tracking of context identifier mappings - - - These DCE-RPC events now contain an additional context-id argument: - - - dce_rpc_bind - - dce_rpc_request - - dce_rpc_response - - - Added new events: - - - dce_rpc_alter_context - - dce_rpc_alter_context_resp - -- The default value of ``Pcap::snaplen`` changed from 8192 to 9216 bytes - to better accommodate jumbo frames. - -- Improvements to ``ntlm.log`` to fix incorrect reporting of login - success/failure. Also, the "status" field was removed and - "server_nb_computer_name", "server_dns_computer_name", and - "server_tree_name" fields added. - -- BroControl: The output of the broctl "top" command has changed slightly. - The "Proc" column has been removed from the output. This column previously - indicated whether each Bro process was the "parent" or "child", but this - is no longer relevant because each Bro node now runs as a single process. - -- The ``DNP3::function_codes`` name for request 0x21 has been corrected from - "AUTHENTICATE_ERR" to "AUTHENTICATE_REQ_NR". - -- The ``DNS::query_types`` names for resource records 41 and 100 have been - corrected from "EDNS" to "OPT" and "DINFO" to "UINFO", respectively. - -Removed Functionality ---------------------- - -- We no longer maintain any Bro plugins as part of the Bro - distribution. Most of the plugins that used to be in aux/plugins have - been moved over to use the Bro Package Manager instead. See - https://packages.bro.org for a list of Bro packages currently - available. - -- The "ocsp_request" event no longer has "requestorName" parameter. - -- The node-specific ``site/local-*.bro`` scripts have been removed. - -- BroControl: The "IPv6Comm" and "ZoneID" options are no longer - available (though Broker should be able to handle IPv6 automatically). - -Deprecated Functionality ------------------------- - -- The old communication system is now deprecated and scheduled for - removal with the next Bro release. This includes the "communication" - framework, the ``&sychronized`` attributes, and the existing - communication-related BiFs. Use Broker instead. - -- The infrastructure for serializing Bro values into a binary - representation is now deprecated and scheduled for removal with the - next Bro release. This includes the ``&persistent`` attribute, as well - as BIFs like "send_id()". Use Broker data stores and the new - configuration framework instead. - -- Mixing of scalars and vectors, such as "v + e" yielding a vector - corresponding to the vector v with the scalar e added to each of - its elements, has been deprecated. - -- The built-in function "merge_pattern()" has been deprecated. It will - be replaced by the '&' operator for patterns. - -- The undocumented feature of using "&&" and "||" operators for patterns - has been deprecated. - -- BroControl: The "update" command is deprecated and scheduled for - removal with the next Bro release. Bro's new configuration framework - is taking its place. - -Bro 2.5.5 -========= - -Bro 2.5.5 primarily addresses security issues. - -- Fix array bounds checking in BinPAC: for arrays that are fields within - a record, the bounds check was based on a pointer to the start of the - record rather than the start of the array field, potentially resulting - in a buffer over-read. - -- Fix SMTP command string comparisons: the number of bytes compared was - based on the user-supplied string length and can lead to incorrect - matches. e.g. giving a command of "X" incorrectly matched - "X-ANONYMOUSTLS" (and an empty commands match anything). - -The following changes address potential vectors for Denial of Service -reported by Christian Titze & Jan Grashöfer of Karlsruhe Institute of -Technology: - -- "Weird" events are now generally suppressed/sampled by default according - to some tunable parameters: - - - Weird::sampling_whitelist - - Weird::sampling_threshold - - Weird::sampling_rate - - Weird::sampling_duration - - Those options can be changed if one needs the previous behavior of - a "net_weird", "flow_weird", or "conn_weird" event being raised for - every single event. Otherwise, there is a new weird_stats.log which - contains concise summaries of weird counts per type per time period - and the original weird.log may not differ much either, except in - the cases where a particular weird type exceeds the sampling threshold. - These changes help improve performance issues resulting from excessive - numbers of weird events. - -- Improved handling of empty lines in several text protocol analyzers - that can cause performance issues when seen in long sequences. - -- Add 'smtp_excessive_pending_cmds' weird which serves as a notification - for when the "pending command" queue has reached an upper limit and - been cleared to prevent one from attempting to slowly exhaust memory. - -Bro 2.5.4 -========= - -Bro 2.5.4 primarily fixes security issues: - -* Multiple fixes and improvements to BinPAC generated code related to - array parsing, with potential impact to all Bro's BinPAC-generated - analyzers in the form of buffer over-reads or other invalid memory - accesses depending on whether a particular analyzer incorrectly - assumed that the evaulated-array-length expression is actually the - number of elements that were parsed out from the input. - -* The NCP analyzer (not enabled by default and also updated to actually - work with newer Bro APIs in the release) performed a memory allocation - based directly on a field in the input packet and using signed integer - storage. This could result in a signed integer overflow and memory - allocations of negative or very large size, leading to a crash or - memory exhaustion. The new NCP::max_frame_size tuning option now - limits the maximum amount of memory that can be allocated. - -There's also the following bug fixes: - -* A memory leak in the SMBv1 analyzer. - -* The MySQL analyzer was generally not working as intended, for example, - it now is able to parse responses that contain multiple results/rows. - -Bro 2.5.3 -========= - -Bro 2.5.3 fixes a security issue in Binpac generated code. In some cases -the code generated by binpac could lead to an integer overflow which can -lead to out of bound reads and allow a remote attacker to crash Bro; there -is also a possibility that this can be exploited in other ways. - -Bro 2.5.2 -========= - -Bro 2.5.2 fixes a security issue in the ContentLine analyzer. In rare cases -a bug in the ContentLine analyzer can lead to an out of bound write of a single -byte. This allows a remote attacker to crash Bro; there also is a possibility -this can be exploited in other ways. CVE-2017-1000458 has been assigned to this -issue. - -Bro 2.5.1 -========= - -New Functionality ------------------ - -- Bro now includes bifs for rename, unlink, and rmdir. - -- Bro now includes events for two extensions used by TLS 1.3: - ssl_extension_supported_versions and ssl_extension_psk_key_exchange_modes - -- Bro now includes hooks that can be used to interact with log processing - on the C++ level. - -- Bro now supports ERSPAN. Currently this ignores the ethernet header that is - carried over the tunnel; if a MAC is logged currently only the outer MAC - is returned. - -- Added a new BroControl option CrashExpireInterval to enable - "broctl cron" to remove crash directories that are older than the - specified number of days (the default value is 0, which means crash - directories never expire). - -- Added a new BroControl option MailReceivingPackets to control - whether or not "broctl cron" will mail a warning when it notices - that no packets were seen on an interface. - -- There is a new broctl command-line option "--version" which outputs - the BroControl version. - -Changed Functionality ---------------------- - -- The input framework's Ascii reader is now more resilient. If an input - is marked to reread a file when it changes and the file didn't exist - during a check Bro would stop watching the file in previous versions. - The same could happen with bad data in a line of a file. These - situations do not cause Bro to stop watching input files anymore. The - old behavior is available through settings in the Ascii reader. - -- The RADIUS scripts have been reworked. Requests are now logged even if - there is no response. The new framed_addr field in the log indicates - if the radius server is hinting at an address for the client. The ttl - field indicates how quickly the server is replying to the network access - server. - -- With the introduction of the Bro package manager, the Bro plugin repository - is considered deprecated. The af_packet, postgresql, and tcprs plugins have - already been removed and are available via bro-pkg. - -Bro 2.5 -======= - -New Dependencies ----------------- - -- Bro now requires a compiler with C++11 support for building the - source code. - -- Bro now requires Python instead of Perl to compile the source code. - -- When enabling Broker (which is disabled by default), Bro now requires - version 0.14 of the C++ Actor Framework. - -New Functionality ------------------ - -- SMB analyzer. This is the rewrite that has been in development for - several years. The scripts are currently not loaded by default and - must be loaded manually by loading policy/protocols/smb. The next - release will load the smb scripts by default. - - - Implements SMB1+2. - - Fully integrated with the file analysis framework so that files - transferred over SMB can be analyzed. - - Includes GSSAPI and NTLM analyzer and reimplements the DCE-RPC - analyzer. - - New logs: smb_cmd.log, smb_files.log, smb_mapping.log, ntlm.log, - and dce_rpc.log - - Not every possible SMB command or functionality is implemented, but - generally, file handling should work whenever files are transferred. - Please speak up on the mailing list if there is an obvious oversight. - -- Bro now includes the NetControl framework. The framework allows for easy - interaction of Bro with hard- and software switches, firewalls, etc. - New log files: netcontrol.log, netcontrol_catch_release.log, - netcontrol_drop.log, and netcontrol_shunt.log. - -- Bro now includes the OpenFlow framework which exposes the data structures - necessary to interface to OpenFlow capable hardware. - -- Bro's Intelligence Framework was refactored and new functionality - has been added: - - - The framework now supports the new indicator type Intel::SUBNET. - As subnets are matched against seen addresses, the new field 'matched' - in intel.log was introduced to indicate which indicator type(s) caused - the hit. - - - The new function remove() allows to delete intelligence items. - - - The intel framework now supports expiration of intelligence items. - Expiration can be configured using the new Intel::item_expiration constant - and can be handled by using the item_expired() hook. The new script - do_expire.bro removes expired items. - - - The new hook extend_match() allows extending the framework. The new - policy script whitelist.bro uses the hook to implement whitelisting. - - - Intel notices are now suppressible and mails for intel notices now - list the identified services as well as the intel source. - -- There is a new file entropy analyzer for files. - -- Bro now supports the remote framebuffer protocol (RFB) that is used by - VNC servers for remote graphical displays. New log file: rfb.log. - -- Bro now supports the Radiotap header for 802.11 frames. - -- Bro now has rudimentary IMAP and XMPP analyzers examining the initial - phases of the protocol. Right now these analyzers only identify - STARTTLS sessions, handing them over to TLS analysis. These analyzers - do not yet analyze any further IMAP/XMPP content. - -- New funtionality has been added to the SSL/TLS analyzer: - - - Bro now supports (draft) TLS 1.3. - - - The new event ssl_extension_signature_algorithm() allows access to the - TLS signature_algorithms extension that lists client supported signature - and hash algorithm pairs. - - - The new event ssl_extension_key_share gives access to the supported named - groups in TLS 1.3. - - - The new event ssl_application_data gives information about application data - that is exchanged before encryption fully starts. This is used to detect - when encryption starts in TLS 1.3. - -- Bro now tracks VLAN IDs. To record them inside the connection log, - load protocols/conn/vlan-logging.bro. - -- A new dns_CAA_reply() event gives access to DNS Certification Authority - Authorization replies. - -- A new per-packet event raw_packet() provides access to layer 2 - information. Use with care, generating events per packet is - expensive. - -- A new built-in function, decode_base64_conn() for Base64 decoding. - It works like decode_base64() but receives an additional connection - argument that will be used for decoding errors into weird.log - (instead of reporter.log). - -- A new get_current_packet_header() bif returns the headers of the current - packet. - -- Three new built-in functions for handling set[subnet] and table[subnet]: - - - check_subnet(subnet, table) checks if a specific subnet is a member - of a set/table. This is different from the "in" operator, which always - performs a longest prefix match. - - - matching_subnets(subnet, table) returns all subnets of the set or table - that contain the given subnet. - - - filter_subnet_table(subnet, table) works like matching_subnets, but returns - a table containing all matching entries. - -- Several built-in functions for handling IP addresses and subnets were added: - - - is_v4_subnet(subnet) checks whether a subnet specification is IPv4. - - - is_v6_subnet(subnet) checks whether a subnet specification is IPv6. - - - addr_to_subnet(addr) converts an IP address to a /32 subnet. - - - subnet_to_addr(subnet) returns the IP address part of a subnet. - - - subnet_width(subnet) returns the width of a subnet. - -- The IRC analyzer now recognizes StartTLS sessions and enables the SSL - analyzer for them. - -- The misc/stats.bro script is now loaded by default and logs more Bro - execution statistics to the stats.log file than it did previously. It - now also uses the standard Bro log format. - -- A set of new built-in functions for gathering execution statistics: - - get_net_stats(), get_conn_stats(), get_proc_stats(), - get_event_stats(), get_reassembler_stats(), get_dns_stats(), - get_timer_stats(), get_file_analysis_stats(), get_thread_stats(), - get_gap_stats(), get_matcher_stats() - -- Two new functions haversine_distance() and haversine_distance_ip() - for calculating geographic distances. The latter function requires that Bro - be built with libgeoip. - -- Table expiration timeout expressions are evaluated dynamically as - timestamps are updated. - -- The pcap buffer size can be set through the new option Pcap::bufsize. - -- Input framework readers stream types Table and Event can now define a custom - event (specified by the new "error_ev" field) to receive error messages - emitted by the input stream. This can, e.g., be used to raise notices in - case errors occur when reading an important input source. - -- The logging framework now supports user-defined record separators, - renaming of column names, as well as extension data columns that can - be added to specific or all logfiles (e.g., to add new names). - -- The new "bro-config" script can be used to determine the Bro installation - paths. - -- New BroControl functionality in aux/broctl: - - - There is a new node type "logger" that can be specified in - node.cfg (that file has a commented-out example). The purpose of - this new node type is to receive logs from all nodes in a cluster - in order to reduce the load on the manager node. However, if - there is no "logger" node, then the manager node will handle - logging as usual. - - - The post-terminate script will send email if it fails to archive - any log files. These mails can be turned off by changing the - value of the new BroControl option MailArchiveLogFail. - - - Added the ability for "broctl deploy" to reload the BroControl - configuration (both broctl.cfg and node.cfg). This happens - automatically if broctl detects any changes to those config files - since the last time the config was loaded. Note that this feature - is relevant only when using the BroControl shell interactively. - - - The BroControl plugin API has a new function "broctl_config". - This gives plugin authors the ability to add their own script code - to the autogenerated broctl-config.bro script. - - - There is a new BroControl plugin for custom load balancing. This - plugin can be used by setting "lb_method=custom" for your worker - nodes in node.cfg. To support packet source plugins, it allows - configuration of a prefix and suffix for the interface name. - -- New Bro plugins in aux/plugins: - - - af_packet: Native AF_PACKET support. - - kafka : Log writer interfacing to Kafka. - - myricom: Native Myricom SNF v3 support. - - pf_ring: Native PF_RING support. - - postgresql: A PostgreSQL reader/writer. - - redis: An experimental log writer for Redis. - - tcprs: A TCP-level analyzer detecting retransmissions, reordering, and more. - -Changed Functionality ---------------------- - -- Log changes: - - - Connections - - The 'history' field gains two new flags: '^' indicates that - Bro heuristically flipped the direction of the connection. - 't/T' indicates the first TCP payload retransmission from - originator or responder, respectively. - - - Intelligence - - New field 'matched' to indicate which indicator type(s) caused the hit. - - - DNS - - New 'rtt' field to indicate the round trip time between when a - request was sent and when a reply started. - - - SMTP - - New 'cc' field which includes the 'Cc' header from MIME - messages sent over SMTP. - - Changes in 'mailfrom' and 'rcptto' fields to remove some - non-address cruft that will tend to be found. The main - example is the change from ``""`` to - ``"user@domain.com"``. - - - HTTP - - Removed 'filename' field (which was seldomly used). - - New 'orig_filenames' and 'resp_filenames' fields which each - contain a vector of filenames seen in entities transferred. - - - stats.log - - The following fields have been added: active_tcp_conns, - active_udp_conns, active_icmp_conns, tcp_conns, udp_conns, - icmp_conns, timers, active_timers, files, active_files, dns_requests, - active_dns_requests, reassem_tcp_size, reassem_file_size, - reassem_frag_size, reassem_unknown_size. - - The following fields have been renamed: lag -> pkt_lag. - - The following fields have been removed: pkts_recv. - -- The BrokerComm and BrokerStore namespaces were renamed to Broker. - The Broker "print()" function was renamed to Broker::send_print(), and - the "event()" function was renamed to Broker::send_event(). - -- The constant ``SSH::skip_processing_after_detection`` was removed. The - functionality was replaced by the new constant - ``SSH::disable_analyzer_after_detection``. - -- The ``net_stats()`` and ``resource_usage()`` functions have been - removed, and their functionality is now provided by the new execution - statistics functions (see above). - -- Some script-level identifiers have changed their names: - - - snaplen -> Pcap::snaplen - - precompile_pcap_filter() -> Pcap::precompile_pcap_filter() - - install_pcap_filter() -> Pcap::install_pcap_filter() - - pcap_error() -> Pcap::error() - -- TCP analysis was changed to process connections without the initial - SYN packet. In the past, connections without a full handshake were - treated as partial, meaning that most application-layer analyzers - would refuse to inspect the payload. Now, Bro will consider these - connections as complete and all analyzers will process them normally. - -- The ``policy/misc/capture-loss.bro`` script is now loaded by default. - -- The traceroute detection script package ``policy/misc/detect-traceroute`` - is no longer loaded by default. - -- Changed BroControl functionality in aux/broctl: - - - The networks.cfg file now contains private IP space 172.16.0.0/12 - by default. - - - Upon startup, if broctl can't get IP addresses from the "ifconfig" - command for any reason, then broctl will now also try to use the - "ip" command. - - - BroControl will now automatically search the Bro plugin directory - for BroControl plugins (in addition to all the other places where - BroControl searches). This enables automatic loading of - BroControl plugins that are provided by a Bro plugin. - - - Changed the default value of the StatusCmdShowAll option so that - the "broctl status" command runs faster. This also means that - there is no longer a "Peers" column in the status output by - default. - - - Users can now specify a more granular log expiration interval. The - BroControl option LogExpireInterval can be set to an arbitrary - time interval instead of just an integer number of days. The time - interval is specified as an integer followed by a time unit: - "day", "hr", or "min". For backward compatibility, an integer - value without a time unit is still interpreted as a number of - days. - - - Changed the text of crash report emails. Now crash reports tell - the user to forward the mail to the Bro team only when a backtrace - is included in the crash report. If there is no backtrace, then - the crash report includes instructions on how to get backtraces - included in future crash reports. - - - There is a new option SitePolicyScripts that replaces SitePolicyStandalone - (the old option is still available, but will be removed in the next - release). - -Removed Functionality ---------------------- - -- The app-stats scripts have been removed because they weren't - being maintained and they were becoming inaccurate (as a result, the - app_stats.log is also gone). They were also prone to needing more regular - updates as the internet changed and will likely be more relevant if - maintained externally. - -- The event ack_above_hole() has been removed, as it was a subset - of content_gap() and led to plenty of noise. - -- The command line options ``--analyze``, ``--set-seed``, and - ``--md5-hashkey`` have been removed. - -- The packaging scripts pkg/make-\*-packages are gone. They aren't - used anymore for the binary Bro packages that the project - distributes; haven't been supported in a while; and have - problems. - -Deprecated Functionality ------------------------- - -- The built-in functions decode_base64_custom() and - encode_base64_custom() are no longer needed and will be removed - in the future. Their functionality is now provided directly by - decode_base64() and encode_base64(), which take an optional - parameter to change the Base64 alphabet. - -Bro 2.4 -======= - -New Functionality ------------------ - -- Bro now has support for external plugins that can extend its core - functionality, like protocol/file analysis, via shared libraries. - Plugins can be developed and distributed externally, and will be - pulled in dynamically at startup (the environment variables - BRO_PLUGIN_PATH and BRO_PLUGIN_ACTIVATE can be used to specify the - locations and names of plugins to activate). Currently, a plugin - can provide custom protocol analyzers, file analyzers, log writers, - input readers, packet sources and dumpers, and new built-in functions. - A plugin can furthermore hook into Bro's processing at a number of - places to add custom logic. - - See https://www.bro.org/sphinx-git/devel/plugins.html for more - information on writing plugins. - -- Bro now has support for the MySQL wire protocol. Activity gets - logged into mysql.log. - -- Bro now parses DTLS traffic. Activity gets logged into ssl.log. - -- Bro now has support for the Kerberos KRB5 protocol over TCP and - UDP. Activity gets logged into kerberos.log. - -- Bro now has an RDP analyzer. Activity gets logged into rdp.log. - -- Bro now has a file analyzer for Portable Executables. Activity gets - logged into pe.log. - -- Bro now has support for the SIP protocol over UDP. Activity gets - logged into sip.log. - -- Bro now features a completely rewritten, enhanced SSH analyzer. The - new analyzer is able to determine if logins failed or succeeded in - most circumstances, logs a lot more more information about SSH - sessions, supports v1, and introduces the intelligence type - ``Intel::PUBKEY_HASH`` and location ``SSH::IN_SERVER_HOST_KEY``. The - analayzer also generates a set of additional events - (``ssh_auth_successful``, ``ssh_auth_failed``, ``ssh_auth_attempted``, - ``ssh_auth_result``, ``ssh_capabilities``, ``ssh2_server_host_key``, - ``ssh1_server_host_key``, ``ssh_encrypted_packet``, - ``ssh2_dh_server_params``, ``ssh2_gss_error``, ``ssh2_ecc_key``). See - next section for incompatible SSH changes. - -- Bro's file analysis now supports reassembly of files that are not - transferred/seen sequentially. The default file reassembly buffer - size is set with the ``Files::reassembly_buffer_size`` variable. - -- Bro's file type identification has been greatly improved (new file types, - bug fixes, and performance improvements). - -- Bro's scripting language now has a ``while`` statement:: - - while ( i < 5 ) - print ++i; - - ``next`` and ``break`` can be used inside the loop's body just like - with ``for`` loops. - -- Bro now integrates Broker, a new communication library. See - aux/broker/README for more information on Broker, and - doc/frameworks/broker.rst for the corresponding Bro script API. - - With Broker, Bro has the similar capabilities of exchanging events and - logs with remote peers (either another Bro process or some other - application that uses Broker). It also includes a key-value store - API that can be used to share state between peers and optionally - allow data to persist on disk for longer-term storage. - - Broker support is by default off for now; it can be enabled at - configure time with --enable-broker. It requires CAF version 0.13+ - (https://github.com/actor-framework/actor-framework) as well as a - C++11 compiler (e.g. GCC 4.8+ or Clang 3.3+). - - Broker will become a mandatory dependency in future Bro versions and - replace the current communication and serialization system. - -- Add --enable-c++11 configure flag to compile Bro's source code in - C++11 mode with a corresponding compiler. Note that 2.4 will be the - last version of Bro that compiles without C++11 support. - -- The SSL analysis now alerts when encountering SSL connections with - old protocol versions or unsafe cipher suites. It also gained - extended reporting of weak keys, caching of already validated - certificates, and full support for TLS record defragmentation. SSL generally - became much more robust and added several fields to ssl.log (while - removing some others). - -- A new icmp_sent_payload event provides access to ICMP payload. - -- The input framework's raw reader now supports seeking by adding an - option "offset" to the config map. Positive offsets are interpreted - to be from the beginning of the file, negative from the end of the - file (-1 is end of file). - -- One can now raise events when a connection crosses a given size - threshold in terms of packets or bytes. The primary API for that - functionality is in base/protocols/conn/thresholds.bro. - -- There is a new command-line option -Q/--time that prints Bro's execution - time and memory usage to stderr. - -- BroControl now has a new command "deploy" which is equivalent to running - the "check", "install", "stop", and "start" commands (in that order). - -- BroControl now has a new option "StatusCmdShowAll" that controls whether - or not the broctl "status" command gathers all of the status information. - This option can be used to make the "status" command run significantly - faster (in this case, the "Peers" column will not be shown in the output). - -- BroControl now has a new option "StatsLogEnable" that controls whether - or not broctl will record information to the "stats.log" file. This option - can be used to make the "broctl cron" command run slightly faster (in this - case, "broctl cron" will also no longer send email about not seeing any - packets on the monitoring interfaces). - -- BroControl now has a new option "MailHostUpDown" which controls whether or - not the "broctl cron" command will send email when it notices that a host - in the cluster is up or down. - -- BroControl now has a new option "CommandTimeout" which specifies the number - of seconds to wait for a command that broctl ran to return results. - -Changed Functionality ---------------------- - -- bro-cut has been rewritten in C, and is hence much faster. - -- File analysis - - * Removed ``fa_file`` record's ``mime_type`` and ``mime_types`` - fields. The event ``file_sniff`` has been added which provides - the same information. The ``mime_type`` field of ``Files::Info`` - also still has this info. - - * The earliest point that new mime type information is available is - in the ``file_sniff`` event which comes after the ``file_new`` and - ``file_over_new_connection`` events. Scripts which inspected mime - type info within those events will need to be adapted. (Note: for - users that worked w/ versions of Bro from git, for a while there was - also an event called ``file_mime_type`` which is now replaced with - the ``file_sniff`` event). - - * Removed ``Files::add_analyzers_for_mime_type`` function. - - * Removed ``offset`` parameter of the ``file_extraction_limit`` - event. Since file extraction now internally depends on file - reassembly for non-sequential files, "offset" can be obtained - with other information already available -- adding together - ``seen_bytes`` and ``missed_bytes`` fields of the ``fa_file`` - record gives how many bytes have been written so far (i.e. - the "offset"). - -- The SSH changes come with a few incompatibilities. The following - events have been renamed: - - * ``SSH::heuristic_failed_login`` to ``ssh_auth_failed`` - * ``SSH::heuristic_successful_login`` to ``ssh_auth_successful`` - - The ``SSH::Info`` status field has been removed and replaced with - the ``auth_success`` field. This field has been changed from a - string that was previously ``success``, ``failure`` or - ``undetermined`` to a boolean. a boolean that is ``T``, ``F``, or - unset. - -- The has_valid_octets function now uses a string_vec parameter instead of - string_array. - -- conn.log gained a new field local_resp that works like local_orig, - just for the responder address of the connection. - -- GRE tunnels are now identified as ``Tunnel::GRE`` instead of - ``Tunnel::IP``. - -- The default name for extracted files changed from extract-protocol-id - to extract-timestamp-protocol-id. - -- The weird named "unmatched_HTTP_reply" has been removed since it can - be detected at the script-layer and is handled correctly by the - default HTTP scripts. - -- When adding a logging filter to a stream, the filter can now inherit - a default ``path`` field from the associated ``Log::Stream`` record. - -- When adding a logging filter to a stream, the - ``Log::default_path_func`` is now only automatically added to the - filter if it has neither a ``path`` nor a ``path_func`` already - explicitly set. Before, the default path function would always be set - for all filters which didn't specify their own ``path_func``. - -- BroControl now establishes only one ssh connection from the manager to - each remote host in a cluster configuration (previously, there would be - one ssh connection per remote Bro process). - -- BroControl now uses SQLite to record state information instead of a - plain text file (the file "spool/broctl.dat" is no longer used). - On FreeBSD, this means that there is a new dependency on the package - "py27-sqlite3". - -- BroControl now records the expected running state of each Bro node right - before each start or stop. The "broctl cron" command uses this info to - either start or stop Bro nodes as needed so that the actual state matches - the expected state (previously, "broctl cron" could only start nodes in - the "crashed" state, and could never stop a node). - -- BroControl now sends all normal command output (i.e., not error messages) - to stdout. Error messages are still sent to stderr, however. - -- The capability of processing NetFlow input has been removed for the - time being. Therefore, the -y/--flowfile and -Y/--netflow command-line - options have been removed, and the netflow_v5_header and netflow_v5_record - events have been removed. - -- The -D/--dfa-size command-line option has been removed. - -- The -L/--rule-benchmark command-line option has been removed. - -- The -O/--optimize command-line option has been removed. - -- The deprecated fields "hot" and "addl" have been removed from the - connection record. Likewise, the functions append_addl() and - append_addl_marker() have been removed. - -- Log files now escape non-printable characters consistently as "\xXX'. - Furthermore, backslashes are escaped as "\\", making the - representation fully reversible. - -Deprecated Functionality ------------------------- - -- The split* family of functions are to be replaced with alternate - versions that return a vector of strings rather than a table of - strings. This also allows deprecation for some related string - concatenation/extraction functions. Note that the new functions use - 0-based indexing, rather than 1-based. - - The full list of now deprecated functions is: - - * split: use split_string instead. - - * split1: use split_string1 instead. - - * split_all: use split_string_all instead. - - * split_n: use split_string_n instead. - - * cat_string_array: see join_string_vec instead. - - * cat_string_array_n: see join_string_vec instead. - - * join_string_array: see join_string_vec instead. - - * sort_string_array: use sort instead. - - * find_ip_addresses: use extract_ip_addresses instead. - -Bro 2.3 -======= - -Dependencies ------------- - -- Libmagic is no longer a dependency. - -New Functionality ------------------ - -- Support for GRE tunnel decapsulation, including enhanced GRE - headers. GRE tunnels are treated just like IP-in-IP tunnels by - parsing past the GRE header in between the delivery and payload IP - packets. - -- The DNS analyzer now actually generates the dns_SRV_reply() event. - It had been documented before, yet was never raised. - -- Bro now uses "file magic signatures" to identify file types. These - are defined via two new constructs in the signature rule parsing - grammar: "file-magic" gives a regular expression to match against, - and "file-mime" gives the MIME type string of content that matches - the magic and an optional strength value for the match. (See also - "Changed Functionality" below for changes due to switching from - using libmagic to such signatures.) - -- A new built-in function, "file_magic", can be used to get all file - magic matches and their corresponding strength against a given chunk - of data. - -- The SSL analyzer now supports heartbeats as well as a few - extensions, including server_name, alpn, and ec-curves. - -- The SSL analyzer comes with Heartbleed detector script in - protocols/ssl/heartbleed.bro. Note that loading this script changes - the default value of "SSL::disable_analyzer_after_detection" from true - to false to prevent encrypted heartbeats from being ignored. - -- StartTLS is now supported for SMTP and POP3. - -- The X509 analyzer can now perform OSCP validation. - -- Bro now has analyzers for SNMP and Radius, which produce corresponding - snmp.log and radius.log output (as well as various events of course). - -- BroControl has a new option "BroPort" which allows a user to specify - the starting port number for Bro. - -- BroControl has a new option "StatsLogExpireInterval" which allows a - user to specify when entries in the stats.log file expire. - -- BroControl has a new option "PFRINGClusterType" which allows a user - to specify a PF_RING cluster type. - -- BroControl now supports PF_RING+DNA. There is also a new option - "PFRINGFirstAppInstance" that allows a user to specify the starting - application instance number for processes running on a DNA cluster. - See the BroControl documentation for more details. - -- BroControl now warns a user to run "broctl install" if Bro has - been upgraded or if the broctl or node configuration has changed - since the most recent install. - -Changed Functionality ---------------------- - -- string slices now exclude the end index (e.g., "123"[1:2] returns - "2"). Generally, Bro's string slices now behave similar to Python. - -- ssl_client_hello() now receives a vector of ciphers, instead of a - set, to preserve their order. - -- Notice::end_suppression() has been removed. - -- Bro now parses X.509 extensions headers and, as a result, the - corresponding event got a new signature: - - event x509_extension(c: connection, is_orig: bool, cert: X509, ext: X509_extension_info); - -- In addition, there are several new, more specialized events for a - number of x509 extensions. - -- Generally, all x509 events and handling functions have changed their - signatures. - -- X509 certificate verification now returns the complete certificate - chain that was used for verification. - -- Bro no longer special-cases SYN/FIN/RST-filtered traces by not - reporting missing data. Instead, if Bro never sees any data segments - for analyzed TCP connections, the new - base/misc/find-filtered-trace.bro script will log a warning in - reporter.log and to stderr. The old behavior can be reverted by - redef'ing "detect_filtered_trace". - -- We have removed the packet sorter component. - -- Bro no longer uses libmagic to identify file types but instead now - comes with its own signature library (which initially is still - derived from libmagic's database). This leads to a number of further - changes with regards to MIME types: - - * The second parameter of the "identify_data" built-in function - can no longer be used to get verbose file type descriptions, - though it can still be used to get the strongest matching file - magic signature. - - * The "file_transferred" event's "descr" parameter no longer - contains verbose file type descriptions. - - * The BROMAGIC environment variable no longer changes any behavior - in Bro as magic databases are no longer used/installed. - - * Removed "binary" and "octet-stream" mime type detections. They - don't provide any more information than an uninitialized - mime_type field. - - * The "fa_file" record now contains a "mime_types" field that - contains all magic signatures that matched the file content - (where the "mime_type" field is just a shortcut for the - strongest match). - -- dns_TXT_reply() now supports more than one string entry by receiving - a vector of strings. - -- BroControl now runs the "exec" and "df" broctl commands only once - per host, instead of once per Bro node. The output of these - commands has been changed slightly to include both the host and - node names. - -- Several performance improvements were made. Particular emphasis - was put on the File Analysis system, which generally will now emit - far fewer file handle request events due to protocol analyzers now - caching that information internally. - -Bro 2.2 -======= - -New Functionality ------------------ - -- A completely overhauled intelligence framework for consuming - external intelligence data. It provides an abstracted mechanism - for feeding data into the framework to be matched against the - data available. It also provides a function named ``Intel::match`` - which makes any hits on intelligence data available to the - scripting language. - - Using input framework, the intel framework can load data from - text files. It can also update and add data if changes are - made to the file being monitored. Files to monitor for - intelligence can be provided by redef-ing the - ``Intel::read_files`` variable. - - The intel framework is cluster-ready. On a cluster, the - manager is the only node that needs to load in data from disk, - the cluster support will distribute the data across a cluster - automatically. - - Scripts are provided at ``policy/frameworks/intel/seen`` that - provide a broad set of sources of data to feed into the intel - framwork to be matched. - -- A new file analysis framework moves most of the processing of file - content from script-land into the core, where it belongs. See - ``doc/file-analysis.rst``, or the online documentation, for more - information. - - Much of this is an internal change, but the framework also comes - with the following user-visible functionality (some of that was - already available before but is done differently, and more - efficiently, now): - - - HTTP: - - * Identify MIME type of messages. - * Extract messages to disk. - * Compute MD5 for messages. - - - SMTP: - - * Identify MIME type of messages. - * Extract messages to disk. - * Compute MD5 for messages. - * Provide access to start of entity data. - - - FTP data transfers: - - * Identify MIME types of data. - * Record to disk. - - - IRC DCC transfers: Record to disk. - - - Support for analyzing data transferred via HTTP range requests. - - - A binary input reader interfaces the input framework with the - file analysis, allowing to inject files on disk into Bro's - content processing. - -- A new framework for computing a wide array of summary statistics, - such as counters and thresholds checks, standard deviation and mean, - set cardinality, top K, and more. The framework operates in - real-time, independent of the underlying data, and can aggregate - information from many independent monitoring points (including - clusters). It provides a transparent, easy-to-use user interface, - and can optionally deploy a set of probabilistic data structures for - memory-efficient operation. The framework is located in - ``scripts/base/frameworks/sumstats``. - - A number of new applications now ship with Bro that are built on top - of the summary statistics framework: - - * Scan detection: Detectors for port and address scans. See - ``policy/misc/scan.bro`` (these scan detectors used to exist in - Bro versions <2.0; it's now back, but quite different). - - * Tracerouter detector: ``policy/misc/detect-traceroute.bro`` - - * Web application detection/measurement: - ``policy/misc/app-stats/*`` - - * FTP and SSH brute-forcing detector: - ``policy/protocols/ftp/detect-bruteforcing.bro``, - ``policy/protocols/ssh/detect-bruteforcing.bro`` - - * HTTP-based SQL injection detector: - ``policy/protocols/http/detect-sqli.bro`` (existed before, but - now ported to the new framework) - -- GridFTP support. This is an extension to the standard FTP analyzer - and includes: - - - An analyzer for the GSI mechanism of GSSAPI FTP AUTH method. - GSI authentication involves an encoded TLS/SSL handshake over - the FTP control session. For FTP sessions that attempt GSI - authentication, the ``service`` field of the connection log - will include ``gridftp`` (as well as also ``ftp`` and - ``ssl``). - - - An example of a GridFTP data channel detection script. It - relies on the heuristics of GridFTP data channels commonly - default to SSL mutual authentication with a NULL bulk cipher - and that they usually transfer large datasets (default - threshold of script is 1 GB). For identified GridFTP data - channels, the ``services`` fields of the connection log will - include ``gridftp-data``. - -- Modbus and DNP3 support. Script-level support is only basic at this - point but see ``src/analyzer/protocol/{modbus,dnp3}/events.bif``, or - the online documentation, for the events Bro generates. For Modbus, - there are also some example policies in - ``policy/protocols/modbus/*``. - -- The documentation now includes a new introduction to writing Bro - scripts. See ``doc/scripting/index.rst`` or, much better, the online - version. There's also the beginning of a chapter on "Using Bro" in - ``doc/using/index.rst``. - -- GPRS Tunnelling Protocol (GTPv1) decapsulation. - -- The scripting language now provide "hooks", a new flavor of - functions that share characteristics of both standard functions and - events. They are like events in that multiple bodies can be defined - for the same hook identifier. They are more like functions in the - way they are invoked/called, because, unlike events, their execution - is immediate and they do not get scheduled through an event queue. - Also, a unique feature of a hook is that a given hook handler body - can short-circuit the execution of remaining hook handlers simply by - exiting from the body as a result of a ``break`` statement (as - opposed to a ``return`` or just reaching the end of the body). See - ``doc/scripts/builtins.rst``, or the online documentation, for more - informatin. - -- Bro's language now has a working ``switch`` statement that generally - behaves like C-style switches (except that case labels can be - comprised of multiple literal constants delimited by commas). Only - atomic types are allowed for now. Case label bodies that don't - execute a ``return`` or ``break`` statement will fall through to - subsequent cases. A ``default`` case label is supported. - -- Bro's language now has a new set of types ``opaque of X``. Opaque - values can be passed around like other values but they can only be - manipulated with BiF functions, not with other operators. Currently, - the following opaque types are supported:: - - opaque of md5 - opaque of sha1 - opaque of sha256 - opaque of cardinality - opaque of topk - opaque of bloomfilter - - These go along with the corrsponding BiF functions ``md5_*``, - ``sha1_*``, ``sha256_*``, ``entropy_*``, etc. . Note that where - these functions existed before, they have changed their signatures - to work with opaques types rather than global state. - -- The scripting language now supports constructing sets, tables, - vectors, and records by name:: - - type MyRecordType: record { - c: count; - s: string &optional; - }; - - global r: MyRecordType = record($c = 7); - - type MySet: set[MyRec]; - global s = MySet([$c=1], [$c=2]); - -- Strings now support the subscript operator to extract individual - characters and substrings (e.g., ``s[4]``, ``s[1:5]``). The index - expression can take up to two indices for the start and end index of - the substring to return (e.g. ``mystring[1:3]``). - -- Functions now support default parameters, e.g.:: - - global foo: function(s: string, t: string &default="abc", u: count &default=0); - -- Scripts can now use two new "magic constants" ``@DIR`` and - ``@FILENAME`` that expand to the directory path of the current - script and just the script file name without path, respectively. - -- ``ssl.log`` now also records the subject client and issuer - certificates. - -- The ASCII writer can now output CSV files on a per filter basis. - -- New SQLite reader and writer plugins for the logging framework allow - to read/write persistent data from on disk SQLite databases. - -- A new packet filter framework supports BPF-based load-balancing, - shunting, and sampling; plus plugin support to customize filters - dynamically. - -- Bro now provides Bloom filters of two kinds: basic Bloom filters - supporting membership tests, and counting Bloom filters that track - the frequency of elements. The corresponding functions are:: - - bloomfilter_basic_init(fp: double, capacity: count, name: string &default=""): opaque of bloomfilter - bloomfilter_basic_init2(k: count, cells: count, name: string &default=""): opaque of bloomfilter - bloomfilter_counting_init(k: count, cells: count, max: count, name: string &default=""): opaque of bloomfilter - bloomfilter_add(bf: opaque of bloomfilter, x: any) - bloomfilter_lookup(bf: opaque of bloomfilter, x: any): count - bloomfilter_merge(bf1: opaque of bloomfilter, bf2: opaque of bloomfilter): opaque of bloomfilter - bloomfilter_clear(bf: opaque of bloomfilter) - - See ``src/probabilistic/bloom-filter.bif``, or the online - documentation, for full documentation. - -- Bro now provides a probabilistic data structure for computing - "top k" elements. The corresponding functions are:: - - topk_init(size: count): opaque of topk - topk_add(handle: opaque of topk, value: any) - topk_get_top(handle: opaque of topk, k: count) - topk_count(handle: opaque of topk, value: any): count - topk_epsilon(handle: opaque of topk, value: any): count - topk_size(handle: opaque of topk): count - topk_sum(handle: opaque of topk): count - topk_merge(handle1: opaque of topk, handle2: opaque of topk) - topk_merge_prune(handle1: opaque of topk, handle2: opaque of topk) - - See ``src/probabilistic/top-k.bif``, or the online documentation, - for full documentation. - -- Bro now provides a probabilistic data structure for computing set - cardinality, using the HyperLogLog algorithm. The corresponding - functions are:: - - hll_cardinality_init(err: double, confidence: double): opaque of cardinality - hll_cardinality_add(handle: opaque of cardinality, elem: any): bool - hll_cardinality_merge_into(handle1: opaque of cardinality, handle2: opaque of cardinality): bool - hll_cardinality_estimate(handle: opaque of cardinality): double - hll_cardinality_copy(handle: opaque of cardinality): opaque of cardinality - - See ``src/probabilistic/cardinality-counter.bif``, or the online - documentation, for full documentation. - -- ``base/utils/exec.bro`` provides a module to start external - processes asynchronously and retrieve their output on termination. - ``base/utils/dir.bro`` uses it to monitor a directory for changes, - and ``base/utils/active-http.bro`` for providing an interface for - querying remote web servers. - -- BroControl can now pin Bro processes to CPUs on supported platforms: - To use CPU pinning, a new per-node option ``pin_cpus`` can be - specified in node.cfg if the OS is either Linux or FreeBSD. - -- BroControl now returns useful exit codes. Most BroControl commands - return 0 if everything was OK, and 1 otherwise. However, there are - a few exceptions. The "status" and "top" commands return 0 if all Bro - nodes are running, and 1 if not all nodes are running. The "cron" - command always returns 0 (but it still sends email if there were any - problems). Any command provided by a plugin always returns 0. - -- BroControl now has an option "env_vars" to set Bro environment variables. - The value of this option is a comma-separated list of environment variable - assignments (e.g., "VAR1=value, VAR2=another"). The "env_vars" option - can apply to all Bro nodes (by setting it in broctl.cfg), or can be - node-specific (by setting it in node.cfg). Environment variables in - node.cfg have priority over any specified in broctl.cfg. - -- BroControl now supports load balancing with PF_RING while sniffing - multiple interfaces. Rather than assigning the same PF_RING cluster ID - to all workers on a host, cluster ID assignment is now based on which - interface a worker is sniffing (i.e., all workers on a host that sniff - the same interface will share a cluster ID). This is handled by - BroControl automatically. - -- BroControl has several new options: MailConnectionSummary (for - disabling the sending of connection summary report emails), - MailAlarmsInterval (for specifying a different interval to send alarm - summary emails), CompressCmd (if archived log files will be compressed, - this specifies the command that will be used to compress them), - CompressExtension (if archived log files will be compressed, this - specifies the file extension to use). - -- BroControl comes with its own test-suite now. ``make test`` in - ``aux/broctl`` will run it. - -In addition to these, Bro 2.2 comes with a large set of smaller -extensions, tweaks, and fixes across the whole code base, including -most submodules. - -Changed Functionality ---------------------- - -- Previous versions of ``$prefix/share/bro/site/local.bro`` (where - "$prefix" indicates the installation prefix of Bro), aren't compatible - with Bro 2.2. This file won't be overwritten when installing over a - previous Bro installation to prevent clobbering users' modifications, - but an example of the new version is located in - ``$prefix/share/bro/site/local.bro.example``. So if no modification - has been done to the previous local.bro, just copy the new example - version over it, else merge in the differences. For reference, - a common error message when attempting to use an outdated local.bro - looks like:: - - fatal error in /usr/local/bro/share/bro/policy/frameworks/software/vulnerable.bro, line 41: BroType::AsRecordType (table/record) (set[record { min:record { major:count; minor:count; minor2:count; minor3:count; addl:string; }; max:record { major:count; minor:count; minor2:count; minor3:count; addl:string; }; }]) - -- The type of ``Software::vulnerable_versions`` changed to allow - more flexibility and range specifications. An example usage: - - .. code:: bro - - const java_1_6_vuln = Software::VulnerableVersionRange( - $max = Software::Version($major = 1, $minor = 6, $minor2 = 0, $minor3 = 44) - ); - - const java_1_7_vuln = Software::VulnerableVersionRange( - $min = Software::Version($major = 1, $minor = 7), - $max = Software::Version($major = 1, $minor = 7, $minor2 = 0, $minor3 = 20) - ); - - redef Software::vulnerable_versions += { - ["Java"] = set(java_1_6_vuln, java_1_7_vuln) - }; - -- The interface to extracting content from application-layer protocols - (including HTTP, SMTP, FTP) has changed significantly due to the - introduction of the new file analysis framework (see above). - -- Removed the following, already deprecated, functionality: - - * Scripting language: - - ``&disable_print_hook attribute``. - - * BiF functions: - - ``parse_dotted_addr()``, ``dump_config()``, - ``make_connection_persistent()``, ``generate_idmef()``, - ``split_complete()`` - - - ``md5_*``, ``sha1_*``, ``sha256_*``, and ``entropy_*`` have - all changed their signatures to work with opaque types (see - above). - -- Removed a now unused argument from ``do_split`` helper function. - -- ``this`` is no longer a reserved keyword. - -- The Input Framework's ``update_finished`` event has been renamed to - ``end_of_data``. It will now not only fire after table-reads have - been completed, but also after the last event of a whole-file-read - (or whole-db-read, etc.). - -- Renamed the option defining the frequency of alarm summary mails to - ``Logging::default_alarm_mail_interval``. When using BroControl, the - value can now be set with the new broctl.cfg option - ``MailAlarmsInterval``. - -- We have completely rewritten the ``notice_policy`` mechanism. It now - no longer uses a record of policy items but a ``hook``, a new - language element that's roughly equivalent to a function with - multiple bodies (see above). For existing code, the two main changes - are: - - - What used to be a ``redef`` of ``Notice::policy`` now becomes a - hook implementation. Example: - - Old:: - - redef Notice::policy += { - [$pred(n: Notice::Info) = { - return n$note == SSH::Login && n$id$resp_h == 10.0.0.1; - }, - $action = Notice::ACTION_EMAIL] - }; - - New:: - - hook Notice::policy(n: Notice::Info) - { - if ( n$note == SSH::Login && n$id$resp_h == 10.0.0.1 ) - add n$actions[Notice::ACTION_EMAIL]; - } - - - notice() is now likewise a hook, no longer an event. If you - have handlers for that event, you'll likely just need to change - the type accordingly. Example: - - Old:: - - event notice(n: Notice::Info) { ... } - - New:: - - hook notice(n: Notice::Info) { ... } - -- The ``notice_policy.log`` is gone. That's a result of the new notice - policy setup. - -- Removed the ``byte_len()`` and ``length()`` bif functions. Use the - ``|...|`` operator instead. - -- The ``SSH::Login`` notice has been superseded by an corresponding - intelligence framework observation (``SSH::SUCCESSFUL_LOGIN``). - -- ``PacketFilter::all_packets`` has been replaced with - ``PacketFilter::enable_auto_protocol_capture_filters``. - -- We removed the BitTorrent DPD signatures pending further updates to - that analyzer. - -- In previous versions of BroControl, running "broctl cron" would create - a file ``$prefix/logs/stats/www`` (where "$prefix" indicates the - installation prefix of Bro). Now, it is created as a directory. - Therefore, if you perform an upgrade install and you're using BroControl, - then you may see an email (generated by "broctl cron") containing an - error message: "error running update-stats". To fix this problem, - either remove that file (it is not needed) or rename it. - -- Due to lack of maintenance the Ruby bindings for Broccoli are now - deprecated, and the build process no longer includes them by - default. For the time being, they can still be enabled by - configuring with ``--enable-ruby``, however we plan to remove - Broccoli's Ruby support with the next Bro release. - -Bro 2.1 -======= - -New Functionality ------------------ - -- Bro now comes with extensive IPv6 support. Past versions offered - only basic IPv6 functionality that was rarely used in practice as it - had to be enabled explicitly. IPv6 support is now fully integrated - into all parts of Bro including protocol analysis and the scripting - language. It's on by default and no longer requires any special - configuration. - - Some of the most significant enhancements include support for IPv6 - fragment reassembly, support for following IPv6 extension header - chains, and support for tunnel decapsulation (6to4 and Teredo). The - DNS analyzer now handles AAAA records properly, and DNS lookups that - Bro itself performs now include AAAA queries, so that, for example, - the result returned by script-level lookups is a set that can - contain both IPv4 and IPv6 addresses. Support for the most common - ICMPv6 message types has been added. Also, the FTP EPSV and EPRT - commands are now handled properly. Internally, the way IP addresses - are stored has been improved, so Bro can handle both IPv4 - and IPv6 by default without any special configuration. - - In addition to Bro itself, the other Bro components have also been - made IPv6-aware by default. In particular, significant changes were - made to trace-summary, PySubnetTree, and Broccoli to support IPv6. - -- Bro now decapsulates tunnels via its new tunnel framework located in - scripts/base/frameworks/tunnels. It currently supports Teredo, - AYIYA, IP-in-IP (both IPv4 and IPv6), and SOCKS. For all these, it - logs the outer tunnel connections in both conn.log and tunnel.log, - and then proceeds to analyze the inner payload as if it were not - tunneled, including also logging that session in conn.log. For - SOCKS, it generates a new socks.log in addition with more - information. - -- Bro now features a flexible input framework that allows users to - integrate external information in real-time into Bro while it's - processing network traffic. The most direct use-case at the moment - is reading data from ASCII files into Bro tables, with updates - picked up automatically when the file changes during runtime. See - doc/input.rst for more information. - - Internally, the input framework is structured around the notion of - "reader plugins" that make it easy to interface to different data - sources. We will add more in the future. - -- BroControl now has built-in support for host-based load-balancing - when using either PF_RING, Myricom cards, or individual interfaces. - Instead of adding a separate worker entry in node.cfg for each Bro - worker process on each worker host, it is now possible to just - specify the number of worker processes on each host and BroControl - configures everything correctly (including any neccessary enviroment - variables for the balancers). - - This change adds three new keywords to the node.cfg file (to be used - with worker entries): lb_procs (specifies number of workers on a - host), lb_method (specifies what type of load balancing to use: - pf_ring, myricom, or interfaces), and lb_interfaces (used only with - "lb_method=interfaces" to specify which interfaces to load-balance - on). - -- Bro's default ASCII log format is not exactly the most efficient way - for storing and searching large volumes of data. An alternatives, - Bro now comes with experimental support for two alternative output - formats: - - * DataSeries: an efficient binary format for recording structured - bulk data. DataSeries is developed and maintained at HP Labs. - See doc/logging-dataseries for more information. - - * ElasticSearch: a distributed RESTful, storage engine and search - engine built on top of Apache Lucene. It scales very well, both - for distributed indexing and distributed searching. See - doc/logging-elasticsearch.rst for more information. - - Note that at this point, we consider Bro's support for these two - formats as prototypes for collecting experience with alternative - outputs. We do not yet recommend them for production (but welcome - feedback!) - - -Changed Functionality ---------------------- - -The following summarizes the most important differences in existing -functionality. Note that this list is not complete, see CHANGES for -the full set. - -- Changes in dependencies: - - * Bro now requires CMake >= 2.6.3. - - * On Linux, Bro now links in tcmalloc (part of Google perftools) - if found at configure time. Doing so can significantly improve - memory and CPU use. - - On the other platforms, the new configure option - --enable-perftools can be used to enable linking to tcmalloc. - (Note that perftools's support for non-Linux platforms may be - less reliable). - -- The configure switch --enable-brov6 is gone. - -- DNS name lookups performed by Bro now also query AAAA records. The - results of the A and AAAA queries for a given hostname are combined - such that at the scripting layer, the name resolution can yield a - set with both IPv4 and IPv6 addresses. - -- The connection compressor was already deprecated in 2.0 and has now - been removed from the code base. - -- We removed the "match" statement, which was no longer used by any of - the default scripts, nor was it likely to be used by anybody anytime - soon. With that, "match" and "using" are no longer reserved keywords. - -- The syntax for IPv6 literals changed from "2607:f8b0:4009:802::1012" - to "[2607:f8b0:4009:802::1012]". When an IP address variable or IP - address literal is enclosed in pipes (for example, - ``|[fe80::db15]|``) the result is now the size of the address in - bits (32 for IPv4 and 128 for IPv6). - -- Bro now spawns threads for doing its logging. From a user's - perspective not much should change, except that the OS may now show - a bunch of Bro threads. - -- We renamed the configure option --enable-perftools to - --enable-perftools-debug to indicate that the switch is only relevant - for debugging the heap. - -- Bro's ICMP analyzer now handles both IPv4 and IPv6 messages with a - joint set of events. The `icmp_conn` record got a new boolean field - 'v6' that indicates whether the ICMP message is v4 or v6. - -- Log postprocessor scripts get an additional argument indicating the - type of the log writer in use (e.g., "ascii"). - -- BroControl's make-archive-name script also receives the writer - type, but as its 2nd(!) argument. If you're using a custom version - of that script, you need to adapt it. See the shipped version for - details. - -- Signature files can now be loaded via the new "@load-sigs" - directive. In contrast to the existing (and still supported) - signature_files constant, this can be used to load signatures - relative to the current script (e.g., "@load-sigs ./foo.sig"). - -- The options "tunnel_port" and "parse_udp_tunnels" have been removed. - Bro now supports decapsulating tunnels directly for protocols it - understands. - -- ASCII logs now record the time when they were opened/closed at the - beginning and end of the file, respectively (wall clock). The - options LogAscii::header_prefix and LogAscii::include_header have - been renamed to LogAscii::meta_prefix and LogAscii::include_meta, - respectively. - -- The ASCII writers "header_*" options have been renamed to "meta_*" - (because there's now also a footer). - -- Some built-in functions have been removed: "addr_to_count" (use - "addr_to_counts" instead), "bro_has_ipv6" (this is no longer - relevant because Bro now always supports IPv6), "active_connection" - (use "connection_exists" instead), and "connection_record" (use - "lookup_connection" instead). - -- The "NFS3::mode2string" built-in function has been renamed to - "file_mode". - -- Some built-in functions have been changed: "exit" (now takes the - exit code as a parameter), "to_port" (now takes a string as - parameter instead of a count and transport protocol, but - "count_to_port" is still available), "connect" (now takes an - additional string parameter specifying the zone of a non-global IPv6 - address), and "listen" (now takes three additional parameters to - enable listening on IPv6 addresses). - -- Some Bro script variables have been renamed: - "LogAscii::header_prefix" has been renamed to - "LogAscii::meta_prefix", "LogAscii::include_header" has been renamed - to "LogAscii::include_meta". - -- Some Bro script variables have been removed: "tunnel_port", - "parse_udp_tunnels", "use_connection_compressor", - "cc_handle_resets", "cc_handle_only_syns", and - "cc_instantiate_on_data". - -- A couple events have changed: the "icmp_redirect" event now includes - the target and destination addresses and any Neighbor Discovery - options in the message, and the last parameter of the - "dns_AAAA_reply" event has been removed because it was unused. - -- The format of the ASCII log files has changed very slightly. Two - new lines are automatically added, one to record the time when the - log was opened, and the other to record the time when the log was - closed. - -- In BroControl, the option (in broctl.cfg) "CFlowAddr" was renamed to - "CFlowAddress". - - -Bro 2.0 -======= - -As the version number jump from 1.5 suggests, Bro 2.0 is a major -upgrade and lots of things have changed. Most importantly, we have -rewritten almost all of Bro's default scripts from scratch, using -quite different structure now and focusing more on operational -deployment. The result is a system that works much better "out of the -box", even without much initial site-specific configuration. The -down-side is that 1.x configurations will need to be adapted to work -with the new version. The two rules of thumb are: - - (1) If you have written your own Bro scripts - that do not depend on any of the standard scripts formerly - found in ``policy/``, they will most likely just keep working - (although you might want to adapt them to use some of the new - features, like the new logging framework; see below). - - (2) If you have custom code that depends on specifics of 1.x - default scripts (including most configuration tuning), that is - unlikely to work with 2.x. We recommend to start by using just - the new scripts first, and then port over any customizations - incrementally as necessary (they may be much easier to do now, - or even unnecessary). Send mail to the Bro user mailing list - if you need help. - -Below we summarize changes from 1.x to 2.x in more detail. This list -isn't complete, see the ``CHANGES`` file in the distribution. -for the full story. - -Script Organization -------------------- - -In versions before 2.0, Bro scripts were all maintained in a flat -directory called ``policy/`` in the source tree. This directory is now -renamed to ``scripts/`` and contains major subdirectories ``base/``, -``policy/``, and ``site/``, each of which may also be subdivided -further. - -The contents of the new ``scripts/`` directory, like the old/flat -``policy/`` still gets installed under the ``share/bro`` -subdirectory of the installation prefix path just like previous -versions. For example, if Bro was compiled like ``./configure ---prefix=/usr/local/bro && make && make install``, then the script -hierarchy can be found in ``/usr/local/bro/share/bro``. - -The main -subdirectories of that hierarchy are as follows: - -- ``base/`` contains all scripts that are loaded by Bro by default - (unless the ``-b`` command line option is used to run Bro in a - minimal configuration). Note that is a major conceptual change: - rather than not loading anything by default, Bro now uses an - extensive set of default scripts out of the box. - - The scripts under this directory generally either accumulate/log - useful state/protocol information for monitored traffic, configure a - default/recommended mode of operation, or provide extra Bro - scripting-layer functionality that has no significant performance cost. - -- ``policy/`` contains all scripts that a user will need to explicitly - tell Bro to load. These are scripts that implement - functionality/analysis that not all users may want to use and may have - more significant performance costs. For a new installation, you - should go through these and see what appears useful to load. - -- ``site/`` remains a directory that can be used to store locally - developed scripts. It now comes with some preinstalled example - scripts that contain recommended default configurations going beyond - the ``base/`` setup. E.g. ``local.bro`` loads extra scripts from - ``policy/`` and does extra tuning. These files can be customized in - place without being overwritten by upgrades/reinstalls, unlike - scripts in other directories. - -With version 2.0, the default ``BROPATH`` is set to automatically -search for scripts in ``policy/``, ``site/`` and their parent -directory, but **not** ``base/``. Generally, everything under -``base/`` is loaded automatically, but for users of the ``-b`` option, -it's important to know that loading a script in that directory -requires the extra ``base/`` path qualification. For example, the -following two scripts: - -* ``$PREFIX/share/bro/base/protocols/ssl/main.bro`` -* ``$PREFIX/share/bro/policy/protocols/ssl/validate-certs.bro`` - -are referenced from another Bro script like: - -.. code:: bro - - @load base/protocols/ssl/main - @load protocols/ssl/validate-certs - -Notice how ``policy/`` can be omitted as a convenience in the second -case. ``@load`` can now also use relative path, e.g., ``@load -../main``. - - -Logging Framework ------------------ - -- The logs generated by scripts that ship with Bro are entirely redone - to use a standardized, machine parsable format via the new logging - framework. Generally, the log content has been restructured towards - making it more directly useful to operations. Also, several - analyzers have been significantly extended and thus now log more - information. Take a look at ``ssl.log``. - - * A particular format change that may be useful to note is that the - ``conn.log`` ``service`` field is derived from DPD instead of - well-known ports (while that was already possible in 1.5, it was - not the default). - - * Also, ``conn.log`` now reports raw number of packets/bytes per - endpoint. - -- The new logging framework makes it possible to extend, customize, - and filter logs very easily. - -- A common pattern found in the new scripts is to store logging stream - records for protocols inside the ``connection`` records so that - state can be collected until enough is seen to log a coherent unit - of information regarding the activity of that connection. This - state is now frequently seen/accessible in event handlers, for - example, like ``c$`` where ```` is replaced by - the name of the protocol. This field is added to the ``connection`` - record by ``redef``'ing it in a - ``base/protocols//main.bro`` script. - -- The logging code has been rewritten internally, with script-level - interface and output backend now clearly separated. While ASCII - logging is still the default, we will add further output types in - the future (binary format, direct database logging). - - -Notice Framework ----------------- - -The way users interact with "notices" has changed significantly in order -to make it easier to define a site policy and more extensible for adding -customized actions. - - -New Default Settings --------------------- - -- Dynamic Protocol Detection (DPD) is now enabled/loaded by default. - -- The default packet filter now examines all packets instead of - dynamically building a filter based on which protocol analysis scripts - are loaded. See ``PacketFilter::all_packets`` for how to revert to old - behavior. - -API Changes ------------ - -- The ``@prefixes`` directive works differently now. - Any added prefixes are now searched for and loaded *after* all input - files have been parsed. After all input files are parsed, Bro - searches ``BROPATH`` for prefixed, flattened versions of all of the - parsed input files. For example, if ``lcl`` is in ``@prefixes``, and - ``site.bro`` is loaded, then a file named ``lcl.site.bro`` that's in - ``BROPATH`` would end up being automatically loaded as well. Packages - work similarly, e.g. loading ``protocols/http`` means a file named - ``lcl.protocols.http.bro`` in ``BROPATH`` gets loaded automatically. - -- The ``make_addr`` BIF now returns a ``subnet`` versus an ``addr`` - - -Variable Naming ---------------- - -- ``Module`` is more widely used for namespacing. E.g. the new - ``site.bro`` exports the ``local_nets`` identifier (among other - things) into the ``Site`` module. - -- Identifiers may have been renamed to conform to new `scripting - conventions - `_ - - -Removed Functionality ---------------------- - -We have remove a bunch of functionality that was rarely used and/or -had not been maintained for a while already: - - - The ``net`` script data type. - - The ``alarm`` statement; use the notice framework instead. - - Trace rewriting. - - DFA state expiration in regexp engine. - - Active mapping. - - Native DAG support (may come back eventually) - - ClamAV support. - - The connection compressor is now disabled by default, and will - be removed in the future. - -BroControl Changes ------------------- - -BroControl looks pretty much similar to the version coming with Bro 1.x, -but has been cleaned up and streamlined significantly internally. - -BroControl has a new ``process`` command to process a trace on disk -offline using a similar configuration to what BroControl installs for -live analysis. - -BroControl now has an extensive plugin interface for adding new -commands and options. Note that this is still considered experimental. - -We have removed the ``analysis`` command, and BroControl currently -does not send daily alarm summaries anymore (this may be restored -later). - -Development Infrastructure --------------------------- - -Bro development has moved from using SVN to Git for revision control. -Users that want to use the latest Bro development snapshot by checking it out -from the source repositories should see the `development process -`_. Note that all the various -sub-components now reside in their own repositories. However, the -top-level Bro repository includes them as git submodules so it's easy -to check them all out simultaneously. - -Bro now uses `CMake `_ for its build system so -that is a new required dependency when building from source. - -Bro now comes with a growing suite of regression tests in -``testing/``. diff --git a/NEWS b/NEWS new file mode 120000 index 0000000000..318c73d3e5 --- /dev/null +++ b/NEWS @@ -0,0 +1 @@ +doc/install/NEWS.rst \ No newline at end of file diff --git a/VERSION b/VERSION index 24e9daef87..714e4d7406 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -2.6-81 +2.6-82 diff --git a/doc b/doc new file mode 160000 index 0000000000..c0092fab7b --- /dev/null +++ b/doc @@ -0,0 +1 @@ +Subproject commit c0092fab7b28c029eddb6b9b654f6096d8e4456a diff --git a/doc/.gitignore b/doc/.gitignore deleted file mode 100644 index 15972ee82a..0000000000 --- a/doc/.gitignore +++ /dev/null @@ -1,2 +0,0 @@ -html -*.pyc diff --git a/doc/CMakeLists.txt b/doc/CMakeLists.txt deleted file mode 100644 index 215b2c9e4f..0000000000 --- a/doc/CMakeLists.txt +++ /dev/null @@ -1,20 +0,0 @@ -set(html_output_dir ${CMAKE_CURRENT_BINARY_DIR}/html) - -add_custom_target(zeek-doc-html - COMMAND sphinx-build - -b html - -c ${CMAKE_CURRENT_SOURCE_DIR} - ${CMAKE_CURRENT_SOURCE_DIR} - ${html_output_dir} - # Create symlink to the html output directory for convenience. - COMMAND "${CMAKE_COMMAND}" -E create_symlink - ${html_output_dir} - ${CMAKE_BINARY_DIR}/html - WORKING_DIRECTORY ${CMAKE_CURRENT_BINARY_DIR} - COMMENT "[Sphinx] Generate Bro HTML documentation in ${html_output_dir}") - -if (NOT TARGET doc) - add_custom_target(doc) -endif () - -add_dependencies(doc zeek-doc-html) diff --git a/doc/LICENSE b/doc/LICENSE deleted file mode 100644 index 8bf6198f38..0000000000 --- a/doc/LICENSE +++ /dev/null @@ -1,5 +0,0 @@ -This work is licensed under the Creative Commons -Attribution 4.0 International License. To view a copy of this -license, visit https://creativecommons.org/licenses/by/4.0/ or send -a letter to Creative Commons, 444 Castro Street, Suite 900, Mountain -View, California, 94041, USA. diff --git a/doc/README b/doc/README deleted file mode 100644 index 79491b15b9..0000000000 --- a/doc/README +++ /dev/null @@ -1,28 +0,0 @@ - -Documentation -============= - -This directory contains documentation in reStructuredText format -(see http://docutils.sourceforge.net/rst.html). - -It is the root of a Sphinx source tree and can be modified to add more -documentation, style sheets, JavaScript, etc. The Sphinx config file -is ``conf.py``. - -There is also a custom Sphinx domain implemented in ``ext/bro.py`` -which adds some reST directives and roles that aid in generating useful -index entries and cross-references. Other extensions can be added in -a similar fashion. - -The ``make doc`` target in the top-level Makefile can be used to locally -render the reST files into HTML. That target depends on: - -* Python interpreter >= 2.7 -* `Sphinx `_ -* `Read the Docs Sphinx Theme `_ - -After the build completes, HTML documentation is symlinked in ``build/html``. - -There's also a ``make livehtml`` target in the top-level Makefile that -is useful for editing the reST files and seeing changes rendered out live -to a separate HTML browser. diff --git a/doc/_templates/breadcrumbs.html b/doc/_templates/breadcrumbs.html deleted file mode 100644 index 0ce26f7d04..0000000000 --- a/doc/_templates/breadcrumbs.html +++ /dev/null @@ -1,15 +0,0 @@ -{% extends "!breadcrumbs.html" %} - -{% block breadcrumbs_aside %} -
  • -{% if pagename != "search" %} - {% if display_github %} - {% if github_version == "master" %} - {{ _('Edit on GitHub') }} - {% endif %} - {% elif show_source and has_source and sourcename %} - {{ _('View page source') }} - {% endif %} -{% endif %} -
    • -{% endblock %} diff --git a/doc/_templates/layout.html b/doc/_templates/layout.html deleted file mode 100644 index cebb59637b..0000000000 --- a/doc/_templates/layout.html +++ /dev/null @@ -1,7 +0,0 @@ -{% extends "!layout.html" %} - -{% if READTHEDOCS and current_version %} - {% if current_version == "latest" or current_version == "stable" %} - {% set current_version = current_version ~ " (" ~ version ~ ")" %} - {% endif %} -{% endif %} diff --git a/doc/cluster/index.rst b/doc/cluster/index.rst deleted file mode 100644 index 2a45435831..0000000000 --- a/doc/cluster/index.rst +++ /dev/null @@ -1,188 +0,0 @@ - -==================== -Cluster Architecture -==================== - - -Bro is not multithreaded, so once the limitations of a single processor core -are reached the only option currently is to spread the workload across many -cores, or even many physical computers. The cluster deployment scenario for -Bro is the current solution to build these larger systems. The tools and -scripts that accompany Bro provide the structure to easily manage many Bro -processes examining packets and doing correlation activities but acting as -a singular, cohesive entity. This document describes the Bro cluster -architecture. For information on how to configure a Bro cluster, -see the documentation for -:doc:`BroControl <../components/broctl/README>`. - -Architecture ---------------- - -The figure below illustrates the main components of a Bro cluster. - -.. image:: /images/deployment.png - -Tap -*** -The tap is a mechanism that splits the packet stream in order to make a copy -available for inspection. Examples include the monitoring port on a switch -and an optical splitter on fiber networks. - -Frontend -******** -The frontend is a discrete hardware device or on-host technique that splits -traffic into many streams or flows. The Bro binary does not do this job. -There are numerous ways to accomplish this task, some of which are described -below in `Frontend Options`_. - -Manager -******* -The manager is a Bro process that has two primary jobs. It receives log -messages and notices from the rest of the nodes in the cluster using the Bro -communications protocol (note that if you are using a logger, then the -logger receives all logs instead of the manager). The result -is a single log instead of many discrete logs that you have to -combine in some manner with post-processing. The manager also takes -the opportunity to de-duplicate notices, and it has the -ability to do so since it's acting as the choke point for notices and how -notices might be processed into actions (e.g., emailing, paging, or blocking). - -The manager process is started first by BroControl and it only opens its -designated port and waits for connections, it doesn't initiate any -connections to the rest of the cluster. Once the workers are started and -connect to the manager, logs and notices will start arriving to the manager -process from the workers. - -Logger -****** -The logger is an optional Bro process that receives log messages from the -rest of the nodes in the cluster using the Bro communications protocol. -The purpose of having a logger receive logs instead of the manager is -to reduce the load on the manager. If no logger is needed, then the -manager will receive logs instead. - -The logger process is started first by BroControl and it only opens its -designated port and waits for connections, it doesn't initiate any -connections to the rest of the cluster. Once the rest of the cluster is -started and connect to the logger, logs will start arriving to the logger -process. - -Proxy -***** -The proxy is a Bro process that manages synchronized state. Variables can -be synchronized across connected Bro processes automatically. Proxies help -the workers by alleviating the need for all of the workers to connect -directly to each other. - -Examples of synchronized state from the scripts that ship with Bro include -the full list of "known" hosts and services (which are hosts or services -identified as performing full TCP handshakes) or an analyzed protocol has been -found on the connection. If worker A detects host 1.2.3.4 as an active host, -it would be beneficial for worker B to know that as well. So worker A shares -that information as an insertion to a set which travels to the cluster's -proxy and the proxy sends that same set insertion to worker B. The result -is that worker A and worker B have shared knowledge about host and services -that are active on the network being monitored. - -The proxy model extends to having multiple proxies when necessary for -performance reasons. It only adds one additional step for the Bro processes. -Each proxy connects to another proxy in a ring and the workers are shared -between them as evenly as possible. When a proxy receives some new bit of -state it will share that with its proxy, which is then shared around the -ring of proxies, and down to all of the workers. From a practical standpoint, -there are no rules of thumb established for the number of proxies -necessary for the number of workers they are serving. It is best to start -with a single proxy and add more if communication performance problems are -found. - -Bro processes acting as proxies don't tend to be extremely hard on CPU -or memory and users frequently run proxy processes on the same physical -host as the manager. - -Worker -****** -The worker is the Bro process that sniffs network traffic and does protocol -analysis on the reassembled traffic streams. Most of the work of an active -cluster takes place on the workers and as such, the workers typically -represent the bulk of the Bro processes that are running in a cluster. -The fastest memory and CPU core speed you can afford is recommended -since all of the protocol parsing and most analysis will take place here. -There are no particular requirements for the disks in workers since almost all -logging is done remotely to the manager, and normally very little is written -to disk. - -The rule of thumb we have followed recently is to allocate approximately 1 -core for every 250Mbps of traffic that is being analyzed. However, this -estimate could be extremely traffic mix-specific. It has generally worked -for mixed traffic with many users and servers. For example, if your traffic -peaks around 2Gbps (combined) and you want to handle traffic at peak load, -you may want to have 8 cores available (2048 / 250 == 8.2). If the 250Mbps -estimate works for your traffic, this could be handled by 2 physical hosts -dedicated to being workers with each one containing a quad-core processor. - -Once a flow-based load balancer is put into place this model is extremely -easy to scale. It is recommended that you estimate the amount of -hardware you will need to fully analyze your traffic. If more is needed it's -relatively easy to increase the size of the cluster in most cases. - -Frontend Options ----------------- - -There are many options for setting up a frontend flow distributor. In many -cases it is beneficial to do multiple stages of flow distribution -on the network and on the host. - -Discrete hardware flow balancers -******************************** - -cPacket -^^^^^^^ - -If you are monitoring one or more 10G physical interfaces, the recommended -solution is to use either a cFlow or cVu device from cPacket because they -are used successfully at a number of sites. These devices will perform -layer-2 load balancing by rewriting the destination Ethernet MAC address -to cause each packet associated with a particular flow to have the same -destination MAC. The packets can then be passed directly to a monitoring -host where each worker has a BPF filter to limit its visibility to only that -stream of flows, or onward to a commodity switch to split the traffic out to -multiple 1G interfaces for the workers. This greatly reduces -costs since workers can use relatively inexpensive 1G interfaces. - -OpenFlow Switches -^^^^^^^^^^^^^^^^^ - -We are currently exploring the use of OpenFlow based switches to do flow-based -load balancing directly on the switch, which greatly reduces frontend -costs for many users. This document will be updated when we have more -information. - -On host flow balancing -********************** - -PF_RING -^^^^^^^ - -The PF_RING software for Linux has a "clustering" feature which will do -flow-based load balancing across a number of processes that are sniffing the -same interface. This allows you to easily take advantage of multiple -cores in a single physical host because Bro's main event loop is single -threaded and can't natively utilize all of the cores. If you want to use -PF_RING, see the documentation on `how to configure Bro with PF_RING -`_. - -Netmap -^^^^^^ - -FreeBSD has an in-progress project named Netmap which will enable flow-based -load balancing as well. When it becomes viable for real world use, this -document will be updated. - -Click! Software Router -^^^^^^^^^^^^^^^^^^^^^^ - -Click! can be used for flow based load balancing with a simple configuration. -This solution is not recommended on -Linux due to Bro's PF_RING support and only as a last resort on other -operating systems since it causes a lot of overhead due to context switching -back and forth between kernel and userland several times per packet. diff --git a/doc/components/binpac/README.rst b/doc/components/binpac/README.rst deleted file mode 120000 index 4eb90ef658..0000000000 --- a/doc/components/binpac/README.rst +++ /dev/null @@ -1 +0,0 @@ -../../../aux/binpac/README \ No newline at end of file diff --git a/doc/components/bro-aux/README.rst b/doc/components/bro-aux/README.rst deleted file mode 120000 index 628879525d..0000000000 --- a/doc/components/bro-aux/README.rst +++ /dev/null @@ -1 +0,0 @@ -../../../aux/bro-aux/README \ No newline at end of file diff --git a/doc/components/broctl/README.rst b/doc/components/broctl/README.rst deleted file mode 120000 index cba305f48a..0000000000 --- a/doc/components/broctl/README.rst +++ /dev/null @@ -1 +0,0 @@ -../../../aux/broctl/doc/broctl.rst \ No newline at end of file diff --git a/doc/components/broker/README.rst b/doc/components/broker/README.rst deleted file mode 120000 index eafa3b8e77..0000000000 --- a/doc/components/broker/README.rst +++ /dev/null @@ -1 +0,0 @@ -../../../aux/broker/README \ No newline at end of file diff --git a/doc/components/btest/README.rst b/doc/components/btest/README.rst deleted file mode 120000 index 0da2935df1..0000000000 --- a/doc/components/btest/README.rst +++ /dev/null @@ -1 +0,0 @@ -../../../aux/btest/README \ No newline at end of file diff --git a/doc/components/capstats/README.rst b/doc/components/capstats/README.rst deleted file mode 120000 index cb2380145d..0000000000 --- a/doc/components/capstats/README.rst +++ /dev/null @@ -1 +0,0 @@ -../../../aux/broctl/aux/capstats/README \ No newline at end of file diff --git a/doc/components/index.rst b/doc/components/index.rst deleted file mode 100644 index ca269cd17b..0000000000 --- a/doc/components/index.rst +++ /dev/null @@ -1,22 +0,0 @@ - -============= -Subcomponents -============= - -The following are snapshots of documentation for components that come -with this version of Bro (|version|). Since they can also be used -independently, see the `download page -`_ for documentation of any -current, independent component releases. - -.. toctree:: - :maxdepth: 1 - - BinPAC - A protocol parser generator - Broker - Bro's (New) Messaging Library - BroControl - Interactive Bro management shell - Bro-Aux - Small auxiliary tools for Bro - BTest - A unit testing framework - Capstats - Command-line packet statistic tool - PySubnetTree - Python module for CIDR lookups - trace-summary - Script for generating break-downs of network traffic diff --git a/doc/components/pysubnettree/README.rst b/doc/components/pysubnettree/README.rst deleted file mode 120000 index 42ce17d303..0000000000 --- a/doc/components/pysubnettree/README.rst +++ /dev/null @@ -1 +0,0 @@ -../../../aux/broctl/aux/pysubnettree/README \ No newline at end of file diff --git a/doc/components/trace-summary/README.rst b/doc/components/trace-summary/README.rst deleted file mode 120000 index 78778364bd..0000000000 --- a/doc/components/trace-summary/README.rst +++ /dev/null @@ -1 +0,0 @@ -../../../aux/broctl/aux/trace-summary/README \ No newline at end of file diff --git a/doc/conf.py b/doc/conf.py deleted file mode 100644 index adff691f71..0000000000 --- a/doc/conf.py +++ /dev/null @@ -1,235 +0,0 @@ -# -*- coding: utf-8 -*- -# -# Zeek documentation build configuration file, created by sphinx-quickstart -# -# This file is execfile()d with the current directory set to its containing dir. -# -# Note that not all possible configuration values are present in this -# autogenerated file. -# -# All configuration values have a default; values that are commented out -# serve to show the default. - -import sys, os - -extensions = [] - -# If extensions (or modules to document with autodoc) are in another directory, -# add these directories to sys.path here. If the directory is relative to the -# documentation root, use os.path.abspath to make it absolute, like shown here. -sys.path.insert(0, os.path.abspath('ext')) - -# -- General configuration ----------------------------------------------------- - -# If your documentation needs a minimal Sphinx version, state it here. -#needs_sphinx = '1.0' - -# Add any Sphinx extension module names here, as strings. They can be extensions -# coming with Sphinx (named 'sphinx.ext.*') or your custom ones. -extensions += ['bro', 'sphinx.ext.todo'] - -# Add any paths that contain templates here, relative to this directory. -templates_path = ['_templates'] - -# The suffix of source filenames. -source_suffix = '.rst' - -# The encoding of source files. -#source_encoding = 'utf-8-sig' - -# The master toctree document. -master_doc = 'index' - -# General information about the project. -project = u'Zeek' -copyright = u'2018, The Zeek Project' - -# The version info for the project you're documenting, acts as replacement for -# |version| and |release|, also used in various other places throughout the -# built documents. -# -# The short X.Y version. -with open('../VERSION', 'r') as f: - version = f.readline().strip() - -# The full version, including alpha/beta/rc tags. -release = version - -# The language for content autogenerated by Sphinx. Refer to documentation -# for a list of supported languages. -#language = None - -# There are two options for replacing |today|: either, you set today to some -# non-false value, then it is used: -#today = '' -# Else, today_fmt is used as the format for a strftime call. -today_fmt = '%B %d, %Y' - -# List of patterns, relative to source directory, that match files and -# directories to ignore when looking for source files. -exclude_patterns = [".#*"] - -# The reST default role (used for this markup: `text`) to use for all documents. -#default_role = None - -# If true, '()' will be appended to :func: etc. cross-reference text. -#add_function_parentheses = True - -# If true, the current module name will be prepended to all description -# unit titles (such as .. function::). -#add_module_names = True - -# If true, sectionauthor and moduleauthor directives will be shown in the -# output. They are ignored by default. -show_authors = True - -# The name of the Pygments (syntax highlighting) style to use. -pygments_style = 'sphinx' - -highlight_language = 'none' - -# A list of ignored prefixes for module index sorting. -#modindex_common_prefix = [] - - -# -- Options for HTML output --------------------------------------------------- - -# The theme to use for HTML and HTML Help pages. See the documentation for -# a list of builtin themes. -on_rtd = os.environ.get('READTHEDOCS', None) == 'True' - -if not on_rtd: - # only import and set the theme if we're building docs locally - import sphinx_rtd_theme - html_theme = 'sphinx_rtd_theme' - html_theme_path = [sphinx_rtd_theme.get_html_theme_path()] - -html_last_updated_fmt = '%B %d, %Y' - -# Theme options are theme-specific and customize the look and feel of a theme -# further. For a list of options available for each theme, see the -# documentation. -html_theme_options = { - 'collapse_navigation': False, - 'display_version': True, -} - -# Add any paths that contain custom themes here, relative to this directory. -#html_theme_path = [] - -# The name for this set of Sphinx documents. If None, it defaults to -# " v Documentation". -html_title = u'Zeek User Manual v' + release - -# A shorter title for the navigation bar. Default is the same as html_title. -#html_short_title = None - -# The name of an image file (relative to this directory) to place at the top -# of the sidebar. -#html_logo = None - -# The name of an image file (within the static path) to use as favicon of the -# docs. This file should be a Windows icon file (.ico) being 16x16 or 32x32 -# pixels large. -#html_favicon = None - -# Add any paths that contain custom static files (such as style sheets) here, -# relative to this directory. They are copied after the builtin static files, -# so a file named "default.css" will overwrite the builtin "default.css". -#html_static_path = ['_static'] - -# If not '', a 'Last updated on:' timestamp is inserted at every page bottom, -# using the given strftime format. -#html_last_updated_fmt = '%b %d, %Y' - -# If true, SmartyPants will be used to convert quotes and dashes to -# typographically correct entities. -#html_use_smartypants = True - -# Custom sidebar templates, maps document names to template names. -#html_sidebars = { -#'**': ['localtoc.html', 'sourcelink.html', 'searchbox.html'], -#} - -# Additional templates that should be rendered to pages, maps page names to -# template names. -#html_additional_pages = {} - -# If false, no module index is generated. -#html_domain_indices = True - -# If false, no index is generated. -#html_use_index = True - -# If true, the index is split into individual pages for each letter. -#html_split_index = False - -# If true, links to the reST sources are added to the pages. -#html_show_sourcelink = True - -# If true, "Created using Sphinx" is shown in the HTML footer. Default is True. -#html_show_sphinx = True - -# If true, "(C) Copyright ..." is shown in the HTML footer. Default is True. -#html_show_copyright = True - -# If true, an OpenSearch description file will be output, and all pages will -# contain a tag referring to it. The value of this option must be the -# base URL from which the finished HTML is served. -#html_use_opensearch = '' - -# This is the file name suffix for HTML files (e.g. ".xhtml"). -#html_file_suffix = None - -# Output file base name for HTML help builder. -htmlhelp_basename = 'zeek-docs' - -# -- Options for LaTeX output -------------------------------------------------- - -# The paper size ('letter' or 'a4'). -#latex_paper_size = 'letter' - -# The font size ('10pt', '11pt' or '12pt'). -#latex_font_size = '10pt' - -# Grouping the document tree into LaTeX files. List of tuples -# (source start file, target name, title, author, documentclass [howto/manual]). -latex_documents = [ - ('index', 'Zeek.tex', u'Zeek Documentation', - u'The Zeek Project', 'manual'), -] - -# The name of an image file (relative to this directory) to place at the top of -# the title page. -#latex_logo = None - -# For "manual" documents, if this is true, then toplevel headings are parts, -# not chapters. -#latex_use_parts = False - -# If true, show page references after internal links. -#latex_show_pagerefs = False - -# If true, show URL addresses after external links. -#latex_show_urls = False - -# Additional stuff for the LaTeX preamble. -#latex_preamble = '' - -# Documents to append as an appendix to all manuals. -#latex_appendices = [] - -# If false, no module index is generated. -#latex_domain_indices = True - -# -- Options for manual page output -------------------------------------------- - -# One entry per manual page. List of tuples -# (source start file, name, description, authors, manual section). -man_pages = [ - ('index', 'bro', u'Zeek Documentation', - [u'The Zeek Project'], 1) -] - -# -- Options for todo plugin -------------------------------------------- -todo_include_todos=True diff --git a/doc/configuration/index.rst b/doc/configuration/index.rst deleted file mode 100644 index c418ec636c..0000000000 --- a/doc/configuration/index.rst +++ /dev/null @@ -1,253 +0,0 @@ - -.. _configuration: - -===================== -Cluster Configuration -===================== - -A *Bro Cluster* is a set of systems jointly analyzing the traffic of -a network link in a coordinated fashion. You can operate such a setup from -a central manager system easily using BroControl because BroControl -hides much of the complexity of the multi-machine installation. - -This section gives examples of how to setup common cluster configurations -using BroControl. For a full reference on BroControl, see the -:doc:`BroControl <../components/broctl/README>` documentation. - - -Preparing to Setup a Cluster -============================ - -In this document we refer to the user account used to set up the cluster -as the "Bro user". When setting up a cluster the Bro user must be set up -on all hosts, and this user must have ssh access from the manager to all -machines in the cluster, and it must work without being prompted for a -password/passphrase (for example, using ssh public key authentication). -Also, on the worker nodes this user must have access to the target -network interface in promiscuous mode. - -Additional storage must be available on all hosts under the same path, -which we will call the cluster's prefix path. We refer to this directory -as ````. If you build Bro from source, then ```` is -the directory specified with the ``--prefix`` configure option, -or ``/usr/local/bro`` by default. The Bro user must be able to either -create this directory or, where it already exists, must have write -permission inside this directory on all hosts. - -When trying to decide how to configure the Bro nodes, keep in mind that -there can be multiple Bro instances running on the same host. For example, -it's possible to run a proxy and the manager on the same host. However, it is -recommended to run workers on a different machine than the manager because -workers can consume a lot of CPU resources. The maximum recommended -number of workers to run on a machine should be one or two less than -the number of CPU cores available on that machine. Using a load-balancing -method (such as PF_RING) along with CPU pinning can decrease the load on -the worker machines. Also, in order to reduce the load on the manager -process, it is recommended to have a logger in your configuration. If a -logger is defined in your cluster configuration, then it will receive logs -instead of the manager process. - - -Basic Cluster Configuration -=========================== - -With all prerequisites in place, perform the following steps to setup -a Bro cluster (do this as the Bro user on the manager host only): - -- Edit the BroControl configuration file, ``/etc/broctl.cfg``, - and change the value of any BroControl options to be more suitable for - your environment. You will most likely want to change the value of - the ``MailTo`` and ``LogRotationInterval`` options. A complete - reference of all BroControl options can be found in the - :doc:`BroControl <../components/broctl/README>` documentation. - -- Edit the BroControl node configuration file, ``/etc/node.cfg`` - to define where logger, manager, proxies, and workers are to run. For a - cluster configuration, you must comment-out (or remove) the standalone node - in that file, and either uncomment or add node entries for each node - in your cluster (logger, manager, proxy, and workers). For example, if you - wanted to run five Bro nodes (two workers, one proxy, a logger, and a - manager) on a cluster consisting of three machines, your cluster - configuration would look like this:: - - [logger] - type=logger - host=10.0.0.10 - - [manager] - type=manager - host=10.0.0.10 - - [proxy-1] - type=proxy - host=10.0.0.10 - - [worker-1] - type=worker - host=10.0.0.11 - interface=eth0 - - [worker-2] - type=worker - host=10.0.0.12 - interface=eth0 - - For a complete reference of all options that are allowed in the ``node.cfg`` - file, see the :doc:`BroControl <../components/broctl/README>` documentation. - -- Edit the network configuration file ``/etc/networks.cfg``. This - file lists all of the networks which the cluster should consider as local - to the monitored environment. - -- Install Bro on all machines in the cluster using BroControl:: - - > broctl install - -- See the :doc:`BroControl <../components/broctl/README>` documentation - for information on setting up a cron job on the manager host that can - monitor the cluster. - - -PF_RING Cluster Configuration -============================= - -`PF_RING `_ allows speeding up the -packet capture process by installing a new type of socket in Linux systems. -It supports 10Gbit hardware packet filtering using standard network adapters, -and user-space DNA (Direct NIC Access) for fast packet capture/transmission. - -Installing PF_RING -^^^^^^^^^^^^^^^^^^ - -1. Download and install PF_RING for your system following the instructions - `here `_. The following - commands will install the PF_RING libraries and kernel module (replace - the version number 5.6.2 in this example with the version that you - downloaded):: - - cd /usr/src - tar xvzf PF_RING-5.6.2.tar.gz - cd PF_RING-5.6.2/userland/lib - ./configure --prefix=/opt/pfring - make install - - cd ../libpcap - ./configure --prefix=/opt/pfring - make install - - cd ../tcpdump-4.1.1 - ./configure --prefix=/opt/pfring - make install - - cd ../../kernel - make install - - modprobe pf_ring enable_tx_capture=0 min_num_slots=32768 - - Refer to the documentation for your Linux distribution on how to load the - pf_ring module at boot time. You will need to install the PF_RING - library files and kernel module on all of the workers in your cluster. - -2. Download the Bro source code. - -3. Configure and install Bro using the following commands:: - - ./configure --with-pcap=/opt/pfring - make - make install - -4. Make sure Bro is correctly linked to the PF_RING libpcap libraries:: - - ldd /usr/local/bro/bin/bro | grep pcap - libpcap.so.1 => /opt/pfring/lib/libpcap.so.1 (0x00007fa6d7d24000) - -5. Configure BroControl to use PF_RING (explained below). - -6. Run "broctl install" on the manager. This command will install Bro and - required scripts to all machines in your cluster. - -Using PF_RING -^^^^^^^^^^^^^ - -In order to use PF_RING, you need to specify the correct configuration -options for your worker nodes in BroControl's node configuration file. -Edit the ``node.cfg`` file and specify ``lb_method=pf_ring`` for each of -your worker nodes. Next, use the ``lb_procs`` node option to specify how -many Bro processes you'd like that worker node to run, and optionally pin -those processes to certain CPU cores with the ``pin_cpus`` option (CPU -numbering starts at zero). The correct ``pin_cpus`` setting to use is -dependent on your CPU architecture (Intel and AMD systems enumerate -processors in different ways). Using the wrong ``pin_cpus`` setting -can cause poor performance. Here is what a worker node entry should -look like when using PF_RING and CPU pinning:: - - [worker-1] - type=worker - host=10.0.0.50 - interface=eth0 - lb_method=pf_ring - lb_procs=10 - pin_cpus=2,3,4,5,6,7,8,9,10,11 - - -Using PF_RING+DNA with symmetric RSS -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -You must have a PF_RING+DNA license in order to do this. You can sniff -each packet only once. - -1. Load the DNA NIC driver (i.e. ixgbe) on each worker host. - -2. Run "ethtool -L dna0 combined 10" (this will establish 10 RSS queues - on your NIC) on each worker host. You must make sure that you set the - number of RSS queues to the same as the number you specify for the - lb_procs option in the node.cfg file. - -3. On the manager, configure your worker(s) in node.cfg:: - - [worker-1] - type=worker - host=10.0.0.50 - interface=dna0 - lb_method=pf_ring - lb_procs=10 - - -Using PF_RING+DNA with pfdnacluster_master -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -You must have a PF_RING+DNA license and a libzero license in order to do -this. You can load balance between multiple applications and sniff the -same packets multiple times with different tools. - -1. Load the DNA NIC driver (i.e. ixgbe) on each worker host. - -2. Run "ethtool -L dna0 1" (this will establish 1 RSS queues on your NIC) - on each worker host. - -3. Run the pfdnacluster_master command on each worker host. For example:: - - pfdnacluster_master -c 21 -i dna0 -n 10 - - Make sure that your cluster ID (21 in this example) matches the interface - name you specify in the node.cfg file. Also make sure that the number - of processes you're balancing across (10 in this example) matches - the lb_procs option in the node.cfg file. - -4. If you are load balancing to other processes, you can use the - pfringfirstappinstance variable in broctl.cfg to set the first - application instance that Bro should use. For example, if you are running - pfdnacluster_master with "-n 10,4" you would set - pfringfirstappinstance=4. Unfortunately that's still a global setting - in broctl.cfg at the moment but we may change that to something you can - set in node.cfg eventually. - -5. On the manager, configure your worker(s) in node.cfg:: - - [worker-1] - type=worker - host=10.0.0.50 - interface=dnacluster:21 - lb_method=pf_ring - lb_procs=10 - diff --git a/doc/devel/plugins.rst b/doc/devel/plugins.rst deleted file mode 100644 index 3cdb59cd65..0000000000 --- a/doc/devel/plugins.rst +++ /dev/null @@ -1,488 +0,0 @@ - -=============== -Writing Plugins -=============== - -Bro internally provides a plugin API that enables extending -the system dynamically, without modifying the core code base. That way -custom code remains self-contained and can be maintained, compiled, -and installed independently. Currently, plugins can add the following -functionality to Bro: - - - Bro scripts. - - - Builtin functions/events/types for the scripting language. - - - Protocol analyzers. - - - File analyzers. - - - Packet sources and packet dumpers. - - - Logging framework backends. - - - Input framework readers. - -A plugin's functionality is available to the user just as if Bro had -the corresponding code built-in. Indeed, internally many of Bro's -pieces are structured as plugins as well, they are just statically -compiled into the binary rather than loaded dynamically at runtime. - -Quick Start -=========== - -Writing a basic plugin is quite straight-forward as long as one -follows a few conventions. In the following we create a simple example -plugin that adds a new built-in function (bif) to Bro: we'll add -``rot13(s: string) : string``, a function that rotates every character -in a string by 13 places. - -Generally, a plugin comes in the form of a directory following a -certain structure. To get started, Bro's distribution provides a -helper script ``aux/bro-aux/plugin-support/init-plugin`` that creates -a skeleton plugin that can then be customized. Let's use that:: - - # init-plugin ./rot13-plugin Demo Rot13 - -As you can see, the script takes three arguments. The first is a -directory inside which the plugin skeleton will be created. The second -is the namespace the plugin will live in, and the third is a descriptive -name for the plugin itself relative to the namespace. Bro uses the -combination of namespace and name to identify a plugin. The namespace -serves to avoid naming conflicts between plugins written by independent -developers; pick, e.g., the name of your organisation. The namespace -``Bro`` is reserved for functionality distributed by the Bro Project. In -our example, the plugin will be called ``Demo::Rot13``. - -The ``init-plugin`` script puts a number of files in place. The full -layout is described later. For now, all we need is -``src/rot13.bif``. It's initially empty, but we'll add our new bif -there as follows:: - - # cat src/rot13.bif - module Demo; - - function rot13%(s: string%) : string - %{ - char* rot13 = copy_string(s->CheckString()); - - for ( char* p = rot13; *p; p++ ) - { - char b = islower(*p) ? 'a' : 'A'; - *p = (*p - b + 13) % 26 + b; - } - - BroString* bs = new BroString(1, reinterpret_cast(rot13), - strlen(rot13)); - return new StringVal(bs); - %} - -The syntax of this file is just like any other ``*.bif`` file; we -won't go into it here. - -Now we can already compile our plugin, we just need to tell the -configure script (that ``init-plugin`` created) where the Bro -source tree is located (Bro needs to have been built there first):: - - # cd rot13-plugin - # ./configure --bro-dist=/path/to/bro/dist && make - [... cmake output ...] - -This builds the plugin in a subdirectory ``build/``. In fact, that -subdirectory *becomes* the plugin: when ``make`` finishes, ``build/`` -has everything it needs for Bro to recognize it as a dynamic plugin. - -Let's try that. Once we point Bro to the ``build/`` directory, it will -pull in our new plugin automatically, as we can check with the ``-N`` -option:: - - # export BRO_PLUGIN_PATH=/path/to/rot13-plugin/build - # bro -N - [...] - Demo::Rot13 - (dynamic, version 0.1.0) - [...] - -That looks quite good, except for the dummy description that we should -replace with something nicer so that users will know what our plugin -is about. We do this by editing the ``config.description`` line in -``src/Plugin.cc``, like this:: - - [...] - plugin::Configuration Plugin::Configure() - { - plugin::Configuration config; - config.name = "Demo::Rot13"; - config.description = "Caesar cipher rotating a string's characters by 13 places."; - config.version.major = 0; - config.version.minor = 1; - config.version.patch = 0; - return config; - } - [...] - -Now rebuild and verify that the description is visible:: - - # make - [...] - # bro -N | grep Rot13 - Demo::Rot13 - Caesar cipher rotating a string's characters by 13 places. (dynamic, version 0.1.0) - -Bro can also show us what exactly the plugin provides with the -more verbose option ``-NN``:: - - # bro -NN - [...] - Demo::Rot13 - Caesar cipher rotating a string's characters by 13 places. (dynamic, version 0.1.0) - [Function] Demo::rot13 - [...] - -There's our function. Now let's use it:: - - # bro -e 'print Demo::rot13("Hello")' - Uryyb - -It works. We next install the plugin along with Bro itself, so that it -will find it directly without needing the ``BRO_PLUGIN_PATH`` -environment variable. If we first unset the variable, the function -will no longer be available:: - - # unset BRO_PLUGIN_PATH - # bro -e 'print Demo::rot13("Hello")' - error in , line 1: unknown identifier Demo::rot13, at or near "Demo::rot13" - -Once we install it, it works again:: - - # make install - # bro -e 'print Demo::rot13("Hello")' - Uryyb - -The installed version went into -``/lib/bro/plugins/Demo_Rot13``. - -One can distribute the plugin independently of Bro for others to use. -To distribute in source form, just remove the ``build/`` directory -(``make distclean`` does that) and then tar up the whole ``rot13-plugin/`` -directory. Others then follow the same process as above after -unpacking. - -To distribute the plugin in binary form, the build process -conveniently creates a corresponding tarball in ``build/dist/``. In -this case, it's called ``Demo_Rot13-0.1.0.tar.gz``, with the version -number coming out of the ``VERSION`` file that ``init-plugin`` put -into place. The binary tarball has everything needed to run the -plugin, but no further source files. Optionally, one can include -further files by specifying them in the plugin's ``CMakeLists.txt`` -through the ``bro_plugin_dist_files`` macro; the skeleton does that -for ``README``, ``VERSION``, ``CHANGES``, and ``COPYING``. To use the -plugin through the binary tarball, just unpack it into -``/lib/bro/plugins/``. Alternatively, if you unpack -it in another location, then you need to point ``BRO_PLUGIN_PATH`` there. - -Before distributing your plugin, you should edit some of the meta -files that ``init-plugin`` puts in place. Edit ``README`` and -``VERSION``, and update ``CHANGES`` when you make changes. Also put a -license file in place as ``COPYING``; if BSD is fine, you will find a -template in ``COPYING.edit-me``. - -Plugin Directory Layout -======================= - -A plugin's directory needs to follow a set of conventions so that Bro -(1) recognizes it as a plugin, and (2) knows what to load. While -``init-plugin`` takes care of most of this, the following is the full -story. We'll use ```` to represent a plugin's top-level -directory. With the skeleton, ```` corresponds to ``build/``. - -``/__bro_plugin__`` - A file that marks a directory as containing a Bro plugin. The file - must exist, and its content must consist of a single line with the - qualified name of the plugin (e.g., "Demo::Rot13"). - -``/lib/.-.so`` - The shared library containing the plugin's compiled code. Bro will - load this in dynamically at run-time if OS and architecture match - the current platform. - -``scripts/`` - A directory with the plugin's custom Bro scripts. When the plugin - gets activated, this directory will be automatically added to - ``BROPATH``, so that any scripts/modules inside can be - "@load"ed. - -``scripts``/__load__.bro - A Bro script that will be loaded when the plugin gets activated. - When this script executes, any BiF elements that the plugin - defines will already be available. See below for more information - on activating plugins. - -``scripts``/__preload__.bro - A Bro script that will be loaded when the plugin gets activated, - but before any BiF elements become available. See below for more - information on activating plugins. - -``lib/bif/`` - Directory with auto-generated Bro scripts that declare the plugin's - bif elements. The files here are produced by ``bifcl``. - -Any other files in ```` are ignored by Bro. - -By convention, a plugin should put its custom scripts into sub folders -of ``scripts/``, i.e., ``scripts///