Cheyenne's Units Tests requirements

    Nenad Rakocevic
    start date: 21/11/2009


===Goals and needs

The goal is to build a test suite for regression testing. So, the tests should cover only features supported by Cheyenne (not all RFC features).


* short & simple testing engine
* simple tests description structure
* named groups of tests (with ability to make group of groups)
* ability to run tests for one given group only
* one file = one group of tests (if it makes things simplier only)
* all test files should be in the same folder

---Testing engine

I see 2 options :

* use an existing unit test engine written in REBOL.
* write one that will be adequate for the job.

The second one has my preference but only if it doesn't take more than 2-3 days to build. Ideally, it should be built incrementally as testing support needs grow.

The engine should define some local functions dedicated to HTTP requests sending and analyzing.

Exemple :

define-tests [					;-- 'define-tests will load this test group in engine
								;-- tests definition is a custom simple dialect

	group "HTTP low-level tests"	;-- group naming

	"GET method" [				;-- string! for test name, block! for test specs
								;-- spec block should accept REBOL code
		res: send server [
			method: "GET"		
			URL: "basic.html"
			version: 1.0		;-- HTTP protocol version
			keep-alive: no		;-- close the connection once response retrieved
		]						;-- keep-alive: yes will be required to test persistent
								;-- connections.
								
		check res/code 200				;-- 'check: func [value1 value2] : compare values 
										;-- and if == then test's OK, else test KO.
		
		check 							;-- check can be called several times in the same test
			res/content/length			;-- 'req is an object! storing pre-parsed response.
			res/headers/Content-length	
			
		check
			123456789
			checksum res/content/data
	]
	...
	
Possible output in console :

Group: HTTP low-level tests
	#1 : GET method => passed OK
or 
	#1 : GET method => *** Error on check #2




===HTTP protocol (priority 1)

Tests should be written using latest HTTP RFC (should be 2616) and other related RFCs as reference documentation.

* request line parsing
* client headers parsing and semantics
* server headers correctness
* adequate response codes
* correct HTTP1.0 answers (when client asks for 1.0)
* correct HTTP1.1 specific features support (like pipelining)

===Cheyenne (priority 2)

The following tests also include testing configuration keywords declared in each mod.

* mod-static specific features testing
* mod-ssi specific features testing
* mod-fastcgi & mod-extapp specific features testing (using PHP as backend)
* OS specific tests (low priority ~3)

Others mods will be implicitly tested in other groups of tests.

===RSP (priority 2)

* POST data handling (with cheyenne's specific features tests)
* GET & POST data decoding (including multipart encoding)
* RSP API complete cover (http://cheyenne-server.org/docs/rsp-api.html)
* Webapp's events
* Webapps isolation
* Sessions life cycle

===CGI (priority 3)

* REBOL CGI's compliant interface (simulation of REBOL -cgi startup mode)
* General CGI compliance (using simple Perl CGI scripts from Perl's units tests)


===Notes

---HTTP testing requirements

labels for status of individual tests groups.
[x] TO DO / Complete / Confirm
[o] DONE!
[|] In progress (there are already some tests which deal with this.)
[.] Ready to start (framework is known to have some specific support to test this, it now needs actual tests to be added.)



[ ] "OPTIONS"                ; Section 9.2
[|] "GET"                    ; Section 9.3
[|] "HEAD"                   ; Section 9.4
[.] "POST"                   ; Section 9.5
[ ] "PUT"                    ; Section 9.6
[ ] "DELETE"                 ; Section 9.7
[ ] "TRACE"                  ; Section 9.8
[ ] "CONNECT"                ; Section 9.9


GENERAL REQUEST FIELDS (request and response)
----
[ ] Cache-Control            ; Section 14.9
[x] Connection               ; Section 14.10		; supported: close, keep-alive
[|] Date                     ; Section 14.18
[ ] Pragma                   ; Section 14.32
[ ] Trailer                  ; Section 14.40
[x] Transfer-Encoding        ; Section 14.41		; chunk-encoding only in response
[x] Upgrade                  ; Section 14.42		; supported for web-sockets only
[ ] Via                      ; Section 14.45
[ ] Warning                  ; Section 14.46

REQUEST FIELDS
----
[ ] Accept                   ; Section 14.1
[ ] Accept-Charset           ; Section 14.2
[ ] Accept-Encoding          ; Section 14.3
[|] Accept-Language          ; Section 14.4
[ ] Authorization            ; Section 14.8
[ ] Expect                   ; Section 14.20
[ ] From                     ; Section 14.22
[.] Host                     ; Section 14.23
[ ] If-Match                 ; Section 14.24
[.] If-Modified-Since        ; Section 14.25
[ ] If-None-Match            ; Section 14.26
[ ] If-Range                 ; Section 14.27
[ ] If-Unmodified-Since      ; Section 14.28
[ ] Max-Forwards             ; Section 14.31
[ ] Proxy-Authorization      ; Section 14.34
[ ] Range                    ; Section 14.35
[ ] Referer                  ; Section 14.36
[ ] TE                       ; Section 14.39
[ ] User-Agent               ; Section 14.43

RESPONSE FIELDS				; any valid response header can be returned by a RSP script
----
[ ] Accept-Ranges           ; Section 14.5
[ ] Age                     ; Section 14.6
[ ] ETag                    ; Section 14.19
[x] Location                ; Section 14.30
[ ] Proxy-Authenticate      ; Section 14.33
[ ] Retry-After             ; Section 14.37
[x] Server                  ; Section 14.38
[ ] Vary                    ; Section 14.44
[ ] WWW-Authenticate        ; Section 14.47


ENTITY FIELDS
---- 
[ ] Allow                    ; Section 14.7
[x] Content-Encoding         ; Section 14.11
[ ] Content-Language         ; Section 14.12
[|] Content-Length           ; Section 14.13
[ ] Content-Location         ; Section 14.14
[ ] Content-MD5              ; Section 14.15
[ ] Content-Range            ; Section 14.16
[|] Content-Type             ; Section 14.17
[x] Expires                  ; Section 14.21
[x] Last-Modified            ; Section 14.29