FILEFORMAT 12 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304305306307308309310311312313314315316317318319320321322323324
  1. The test suite's file format is very simple and extensible, closely
  2. resembling XML. All data for a single test case resides in a single
  3. ASCII file. Labels mark the beginning and the end of all sections, and each
  4. label must be written in its own line. Comments are either XML-style
  5. (enclosed with <!-- and -->) or C-style (beginning with #) and must appear
  6. on their own lines and not alongside actual test data. Most test data files
  7. are syntactically valid XML, although a few files are not (lack of
  8. support for character entities and the preservation of CR/LF characters at
  9. the end of lines are the biggest differences).
  10. The file begins with a 'testcase' tag, which encompasses the remainder of
  11. the file.
  12. <testcase>
  13. Each file is split up in three main sections: reply, client and verify. The
  14. reply section is used for the server to know what to send as a reply for the
  15. requests curl sends, the client section defines how the client should behave
  16. while the verify section defines how to verify that the data stored after a
  17. command has been run ended up correctly.
  18. Each main section has a number of available subsections that can be
  19. specified, that will be checked/used if specified. This document includes all
  20. the subsections currently supported.
  21. Main sections are 'info', 'reply', 'client' and 'verify'.
  22. <info>
  23. <keywords>
  24. A newline-separated list of keywords describing what this test case uses and
  25. tests. Try to use an already used keyword. These keywords will be used for
  26. statistical/informational purposes and for choosing or skipping classes
  27. of tests. "Keywords" must begin with an alphabetic character, "-", "["
  28. or "{" and may actually consist of multiple words separated by spaces
  29. which are treated together as a single identifier.
  30. </keywords>
  31. </info>
  32. <reply>
  33. <data [nocheck="1"] [sendzero="yes"] [base64="yes"]>
  34. data to be sent to the client on its request and later verified that it arrived
  35. safely. Set nocheck="1" to prevent the test script from verifying the arrival
  36. of this data.
  37. If the data contains 'swsclose' anywhere within the start and end tag, and
  38. this is a HTTP test, then the connection will be closed by the server after
  39. this response is sent. If not, the connection will be kept persistent.
  40. If the data contains 'swsbounce' anywhere within the start and end tag, the
  41. HTTP server will detect if this is a second request using the same test and
  42. part number and will then increase the part number with one. This is useful
  43. for auth tests and similar.
  44. 'sendzero' set to yes means that the (FTP) server will "send" the data even if
  45. the size is zero bytes. Used to verify curl's behaviour on zero bytes
  46. transfers.
  47. 'base64' set to yes means that the data provided in the test-file is a chunk
  48. of data encoded with base64. It is the only way a test case can contain binary
  49. data. (This attribute can in fact be used on any section, but it doesn't make
  50. much sense for other sections than "data").
  51. </data>
  52. <dataNUM>
  53. Send back this contents instead of the <data> one. The num is set by:
  54. A) The test number in the request line is >10000 and this is the remainder
  55. of [test case number]%10000.
  56. B) The request was HTTP and included digest details, which adds 1000 to NUM
  57. C) If a HTTP request is NTLM type-1, it adds 1001 to num
  58. D) If a HTTP request is NTLM type-3, it adds 1002 to num
  59. </dataNUM>
  60. <datacheck [nonewline="yes"]>
  61. if the data is sent but this is what should be checked afterwards. If
  62. 'nonewline' is set, we will cut off the trailing newline of this given data
  63. before comparing with the one actually received by the client
  64. </datacheck>
  65. <size>
  66. number to return on a ftp SIZE command (set to -1 to make this command fail)
  67. </size>
  68. <mdtm>
  69. what to send back if the client sends a (FTP) MDTM command, set to -1 to
  70. have it return that the file doesn't exist
  71. </mdtm>
  72. <postcmd>
  73. special purpose server-command to control its behavior *after* the
  74. reply is sent
  75. For HTTP/HTTPS, these are supported:
  76. wait [secs]
  77. - Pause for the given time
  78. </postcmd>
  79. <servercmd>
  80. Special-commands for the server.
  81. For FTP, these are supported:
  82. REPLY [command] [return value] [response string]
  83. - Changes how the server responds to the [command]. [response string] is
  84. evaluated as a perl string, so it can contain embedded \r\n, for example.
  85. COUNT [command] [num]
  86. - Do the REPLY change for [command] only [num] times and then go back to the
  87. built-in approach
  88. DELAY [command] [secs]
  89. - Delay responding to this command for the given time
  90. RETRWEIRDO
  91. - Enable the "weirdo" RETR case when multiple response lines appear at once
  92. when a file is transfered
  93. RETRNOSIZE
  94. - Make sure the RETR response doesn't contain the size of the file
  95. NOSAVE
  96. - Don't actually save what is received
  97. SLOWDOWN
  98. - Send FTP responses with 0.1 sec delay between each byte
  99. PASVBADIP
  100. - makes PASV send back an illegal IP in its 227 response
  101. For HTTP/HTTPS:
  102. auth_required if this is set and a POST/PUT is made without auth, the
  103. server will NOT wait for the full request body to get sent
  104. idle do nothing after receiving the request, just "sit idle"
  105. stream continuously send data to the client, never-ending
  106. pipe: [num] tell the server to expect this many HTTP requests before
  107. sending back anything, to allow pipelining tests
  108. skip: [num] instructs the server to ignore reading this many bytes from a PUT
  109. or POST request
  110. </servercmd>
  111. </reply>
  112. <client>
  113. <server>
  114. What server(s) this test case requires/uses:
  115. file
  116. ftp
  117. ftp-ipv6
  118. ftps
  119. http
  120. http-ipv6
  121. https
  122. none
  123. scp
  124. sftp
  125. socks4
  126. socks5
  127. Give only one per line. This subsection is mandatory.
  128. </server>
  129. <features>
  130. A list of features that MUST be present in the client/library for this test to
  131. be able to run (if these features are not present, the test will be
  132. SKIPPED). Features testable here are:
  133. crypto
  134. getrlimit
  135. GnuTLS
  136. idn
  137. ipv6
  138. large_file
  139. libz
  140. netrc_debug
  141. NSS
  142. OpenSSL
  143. SSL
  144. as well as each protocol that curl supports. A protocol only needs to be
  145. specified if it is different from the server (useful when the server
  146. is 'none').
  147. </features>
  148. <killserver>
  149. Using the same syntax as in <server> but when mentioned here these servers
  150. are explicitly KILLED when this test case is completed. Only use this if there
  151. is no other alternatives. Using this of course requires subsequent tests to
  152. restart servers.
  153. </killserver>
  154. <precheck>
  155. A command line that if set gets run by the test script before the test. If an
  156. output is displayed by the command, the test will be skipped and the
  157. (single-line) output will be displayed as reason for not running the test.
  158. Variables are substituted as in the <command> section.
  159. </precheck>
  160. <postcheck>
  161. A command line that if set gets run by the test script after the test. If
  162. the command exists with a non-zero status code, the test will be considered
  163. to have failed. Variables are substituted as in the <command> section.
  164. </postcheck>
  165. <tool>
  166. Name of tool to use instead of "curl". This tool must be built and exist
  167. in the libtest/ directory.
  168. </tool>
  169. <name>
  170. test case description
  171. </name>
  172. <setenv>
  173. variable1=contents1
  174. variable2=contents2
  175. Set the given environment variables to the specified value before the actual
  176. command is run. They are cleared again after the command has been run.
  177. Variables are first substituted as in the <command> section.
  178. </setenv>
  179. <command [option="no-output"] [timeout="secs"] [delay="secs"]>
  180. command line to run, there's a bunch of %variables that get replaced
  181. accordingly.
  182. Note that the URL that gets passed to the server actually controls what data
  183. that is returned. The last slash in the URL must be followed by a number. That
  184. number (N) will be used by the test-server to load test case N and return the
  185. data that is defined within the <reply><data></data></reply> section.
  186. If a CONNECT is used to the server (to emulate HTTPS etc over proxy), the port
  187. number given in the CONNECT request will be used to identify which test that
  188. is being run, if the proxy host name is said to start with 'test'.
  189. Set option="no-output" to prevent the test script to slap on the --output
  190. argument that directs the output to a file. The --output is also not added if
  191. the verify/stdout section is used.
  192. Set timeout="secs" to override default server logs advisor read lock timeout.
  193. This timeout is used by the test harness, once that the command has completed
  194. execution, to wait for the test server to write out server side log files and
  195. remove the lock that advised not to read them. The "secs" parameter is the not
  196. negative integer number of seconds for the timeout. This 'timeout' attribute
  197. is documented for completeness sake, but is deep test harness stuff and only
  198. needed for very singular and specific test cases. Avoid using it.
  199. Set delay="secs" to introduce a time delay once that the command has completed
  200. execution and before the <postcheck> section runs. The "secs" parameter is the
  201. not negative integer number of seconds for the delay. This 'delay' attribute
  202. is intended for very specific test cases, and normally not needed.
  203. Available substitute variables include:
  204. %CLIENTIP - IPv4 address of the client running curl
  205. %CLIENT6IP - IPv6 address of the client running curl
  206. %HOSTIP - IPv4 address of the host running this test
  207. %HOSTPORT - Port number of the HTTP server
  208. %HOST6IP - IPv6 address of the host running this test
  209. %HOST6PORT - IPv6 port number of the HTTP server
  210. %HTTPSPORT - Port number of the HTTPS server
  211. %FTPPORT - Port number of the FTP server
  212. %FTP6PORT - IPv6 port number of the FTP server
  213. %FTPSPORT - Port number of the FTPS server
  214. %FTP2PORT - Port number of the FTP server 2
  215. %FTPTIME2 - Timeout in seconds that should be just sufficient to receive
  216. a response from the test FTP server
  217. %TFTPPORT - Port number of the TFTP server
  218. %TFTP6PORT - IPv6 port number of the TFTP server
  219. %SSHPORT - Port number of the SCP/SFTP server
  220. %SOCKSPORT - Port number of the SOCKS4/5 server
  221. %SRCDIR - Full path to the source dir
  222. %PWD - Current directory
  223. %CURL - Path to the curl executable
  224. %USER - Login ID of the user running the test
  225. </command>
  226. <file name="log/filename">
  227. This creates the named file with this content before the test case is run,
  228. which is useful if the test case needs a file to act on.
  229. Variables are substituted on the contents of the file as in the <command>
  230. section.
  231. </file>
  232. <stdin>
  233. Pass this given data on stdin to the tool.
  234. </stdin>
  235. </client>
  236. <verify>
  237. <errorcode>
  238. numerical error code curl is supposed to return. Specify a list of accepted
  239. error codes by separating multiple numbers with comma. See test 237 for an
  240. example.
  241. </errorcode>
  242. <strip>
  243. One regex per line that is removed from the protocol dumps before the
  244. comparison is made. This is very useful to remove dependencies on dynamically
  245. changing protocol data such as port numbers or user-agent strings.
  246. </strip>
  247. <strippart>
  248. One perl op per line that operates on the protocol dump. This is pretty
  249. advanced. Example: "s/^EPRT .*/EPRT stripped/"
  250. </strippart>
  251. <protocol [nonewline="yes"]>
  252. the protocol dump curl should transmit, if 'nonewline' is set, we will cut
  253. off the trailing newline of this given data before comparing with the one
  254. actually sent by the client
  255. Variables are substituted as in the <command> section.
  256. </protocol>
  257. <stdout [mode="text"] [nonewline="yes"]>
  258. This verifies that this data was passed to stdout. Variables are
  259. substituted as in the <command> section.
  260. Use the mode="text" attribute if the output is in text mode on platforms that
  261. have a text/binary difference.
  262. If 'nonewline' is set, we will cut off the trailing newline of this given data
  263. before comparing with the one actually received by the client
  264. </stdout>
  265. <file name="log/filename" [mode="text"]>
  266. The file's contents must be identical to this after the test is complete.
  267. Use the mode="text" attribute if the output is in text mode on platforms that
  268. have a text/binary difference.
  269. Variables are substituted as in the <command> section.
  270. </file>
  271. <stripfile>
  272. One perl op per line that operates on the file before being compared. This is
  273. pretty advanced. Example: "s/^EPRT .*/EPRT stripped/"
  274. </stripfile>
  275. <upload>
  276. the contents of the upload data curl should have sent
  277. </upload>
  278. <valgrind>
  279. disable - disables the valgrind log check for this test
  280. </valgrind>
  281. </verify>
  282. </testcase>