WebTst Manual

Table of contents

Features at a glance


Back to top

Installing WebTst

To be included soon, instructions below are now seriously out of date, mod_perl is no longer required, no apache recompiling required!

***************** Old info *****************************

WebTst is a combination of perl code running with Apache. Installing WebTst is therefore mostly a question of installing Apache with mod_perl, installing a series of required perl packages and hooking up via Apache config files. To install follow the steps:
  1. Compile apache (tested on 1.3.27) with mod_perl and with -DSECURITY_HOLE_PASS_AUTHORIZATION (to allow httpd authentication).
    • mod_perl can be obtained online from http://perl.apache.org/
    • Uncompress apache
    • Uncompress mod_perl
    • In apache dir: ./configure --prefix=
    • In mod_perl dir: /bin/perl Makefile.PL EVERYTHING=1 Accept location of apache to where you decompressed apache.
    • The SECURITY_HOLE_PASS_AUTHORIZATION flag can be set by editing the apache src/main/Makefile and by adding the -D... definition to the EXTRA_CFLAGS definition. Example: 
           	       EXTRA_CFLAGS=-fno-strict-aliasing -I/usr/lib/perl5/5.6.0/i386-linux/CORE -I. -I../.. -DUSE_PERL_SSI  -fno-strict-aliasing -DMOD_PERL -DSECURITY_HOLE_PASS_AUTHORIZATION
    • From mod_perl dir: make
    • From apache dir: make
    • From apache dir: make install
  2. Edit ./apache/httpd.conf to reflect the location of the WebTst installation
  3. Include ./apache/httpd.conf into your web server's httpd.conf file. Example, add the following apache directive to the main httpd.conf file: 
              Include /apache/httpd.conf
  4. Make sure your main httpd.conf reflects the appropriate port location for test tool. Change Port directive to have the correct port.
  5. (Re)Start your main apache web server.
  6. To make sure it is working point a browser to http://:/webtst/dashboard.cgi
  7. Edit ./bash.env to define WEBTST_HOME (location of WebTst install) and WEBTST_GROUP_ID (WebTst running group id as taken from /etc/group) environment variables. Example: 
           export WEBTST_HOME=/home/rosa/works/perl/webtst
       export WEBTST_GROUP_ID=507
Note: You might be required to install a couple of extra perl packages. That depends on whatever you already have installed. Know dependencies include: These are the versions against wich this tool has been developed and tested. More recent versions may work without any problems but there are no guarantees that it does work with these more recent versions. ***************** Old info *****************************


Back to top

The Web Interface


Back to top

The Command Line Interface

WebTst comes "out of the box" with two command line interfaces for running tests. The first one (cronjob Runner) allows the user to run specific tests. It can be used to add test running to your cronjobs. The second one (suite Runner) allows the user to define suites of tests and to run these suites via command line. Suite runner can also be added to crontab to allow automatic running of test suites.

Cronjob Runner

What does it do ?

This command line script will allow the users to run specific tests. It can be used from the user's crontab to run automatically specific tests.

Where can it be found ?

The cronjob runner can be found at <WebTstInstallDir>/bin/cronjob.pl.

What are it's call parameters ?

cronjob.pl takes the following switches/parameters:

Running Examples


Back to top

Suite Runner

What does it do ?

The suite runner will allow users to maintain and run suites of tests. By being a command line tool, allows users to add executions to crontabs thus allowing for automated execution of tests.

Where can it be found ?

The suite runner can be found at <WebTstInstallDir>/bin/suiteRunner.pl.

What are it's call parameters ?

suiteRunner.pl takes the following switches/parameters:

Running Examples

Defining Test Suite Files

A Test suite file is a simple tab separated file with the following fields in each row: The following is an example of a test suite file: 
    test    smoke   rosa  /osoTests/critter/dev/  getCookiesInLoginLogout
    suite   smoke   rosa  /home/webtst/suites/acs.suite  .*

Back to top

The Recording File Format

Directory Structure

Recordings are stored in the file system for now. The entries in the file system obey the following rules:

recording.xml Format

The format for the recording file is a valid XML format ready to be handled by the XML::Dumper perl module (available from http://www.cpan.org/). This is a very loose format ideal for translation of XML files into perl structures. This means that although you have to follow a syntax, you can pretty much stuff whatever you want into it. The format does not enforce semantic restrictions and therefore you should probably know what is needed if you decide to handcode one of these recording files. The web interface should deal nicely with the creation of these files.

recording.xml DTD

The format for these files follows the DTD: 
	<?xml version="1.0" encoding="ISO-8859-1"?>

	<!DOCTYPE perldata [
		<!ELEMENT perldata (hash)>
		<!ELEMENT hash (item*)>
		<!ELEMENT array (item*)>
		<!ELEMENT item (#PCDATA|hash|array)*>

		<!ATTLIST item key CDATA #REQUIRED>
	]>


The file will map to a hash from which fields are extracted. These are:

A sample recording.xml

The following link leads to an example of a recording file.
Back to top

The Test Types

The following are the currently available test types.
Test Id Description TEST_PARAM1 TEST_PARAM2
ACCESS Check if the page can be accessed. Unused Unused
TITLE_SUCCESS Success if the returned page title matches a specified pattern. A perl regular expression for title matching (e.g. Francisco.*Rosa) Unused
TITLE_FAIL Failure if the returned page title matches a specified pattern. A perl regular expression for title matching (e.g. Francisco.*Rosa) Unused
TIME Success if the time to process the requests falls within the specified range. The lower limit for the time range (e.g. 2, as in 2 seconds). The higher limit for the time range (e.g. 30, as in 30 seconds).
PAGE_DIFF Success if the retrieved content matches bit by bit the stored content at recording session time. Unused Unused
CUSTOM_TEST Allow running a user-built custom test. The perl module, accessible from the running environment where the run method can be found. The parameters to the test execution.
CONTENT_PATTERN_SUCCESS Success if the retrieved content matches the specified regular expression. A perl regular expression to match the content against (e.g. Francisco.*Rosa) Unused
CONTENT_PATTERN_FAIL Failure if the retrieved content matches the specified regular expression. A perl regular expression to match the content against (e.g. Francisco.*Rosa) Unused
STATUS_SUCCESS Success if the status of the request execution matches given status id. Numeric status for success of test (e.g. 401, 200) Unused
STATUS_FAIL Failure if the status of the request execution matches given status id. Numeric status for success of test (e.g. 401, 200) Unused
COOKIE_SET_PATTERN_SUCCESS Success if the execution of the request results in the specified cookie being set with the specified value. Name of the cookie being set. A perl regular expression to match the cookie value against.
TEST_SEQUENCE Allows running a sequence of tests against returned request. Successfull if all tests are successfull, failure if any one of the tests fails. A sequence of tests following the syntax: TestID[TEST_PARAM1|TEST_PARAM2](;TestID[TEST_PARAM1|TEST_PARAM2])*. Example: COOKIE_SET_PATTERN_SUCCESS[critter_javascript|.*?Ingenta TEST.*?];ACCESS[];TITLE_SUCESS[Francisco]). Unused
SUBTEST_CALL Run a full subtest when hitting this step. Effectively allows reuse of defined tests. The subtest folder. The subtest name.

Back to top

Custom Tests

What are Custom Tests

Custom tests can be called upon selecting test type CUSTOM_TEST for the test step. They represent hooks to any kind of tests a user may wish to perform upon requests, tests which can not be provided "out of the box" in no way possible.

Custom Tests Requirements

Custom tests are nothing more than perl modules (.pm files) that meet the following requirements:

The Data Passed Along by WebTst's Playback Engine

WebTst's playback engine passes along a single testParams perl hash containing the following fields:

The TestResult Class

Custom tests need to return one object of the class TestResult. Upon completion. This should actually be pretty simple to do. The typical syntax to do that is: 
    return new TestResult(<itemResult>,<testType>,<testStepDuration>,<resultMessage>);
Where:

A Custom Test Example

The following is an example of a custom test. This is one of the "out of the box" available custom test that can be used. It picks up and stores a value to a variable in the variable store based on a specified pattern to match against returned content. 
package varSet_FirstMatch;
use strict;

# Custom test run method
sub run {
	my ($testParams) = @_;
	my $recordingName = $testParams->{"RECORDING_NAME"};
	my $testItem = $testParams->{"TEST_ITEM"};
	my $testDuration = $testParams->{"TEST_DURATION"};
	my $request = $testParams->{"TEST_REQUEST"};
	my $response = $testParams->{"TEST_RESPONSE"};
	my $feedbackLineMethod = $testParams->{"FEEDBACK_METHOD"};
	my $testType = $testItem->{"TEST_TYPE"};
	my $paramList = &Utils::unescapeXML($testItem->{"TEST_PARAM2"});
	my $variableStore = $testParams->{"VARIABLE_STORE"};

	# get content from step response
	my $newContentText =  ${$response->content_ref()};

	# split parameters to get variable name and pattern to match
	my @paramArray = split(",",$paramList);
	if ( @paramArray != 2 ) {
		return new TestResult("FAIL",$testType,$testDuration,"varSet_FirstMatch requires exactly 2 parameters, got: >>$paramList<<");
	}

	# get variable name
	my $varName = $paramArray[0];
	# get pattern expression
	my $matchExpression = $paramArray[1];

	# try matching
	my $firstMatch = "";
	if ( (($firstMatch) = ($newContentText =~ m@$matchExpression@is)) ) {
		
		# Success, store and return
		$variableStore->{$varName} = $firstMatch;
		return new TestResult("OK",$testType,$testDuration,"varSet_FirstMatch assigned $firstMatch to var: $varName");
	}
	else {
		# Failure, could not match
		return new TestResult("FAIL",$testType,$testDuration,"varSet_FirstMatch failed setting var: >>$varName<<, when matching: >>$matchExpression<< in content.");
	}
}
1;

Back to top

Tests Analysers

Out-of-the-box Results Analysers

Upon completion of execution of a test, the user gets the option of analysing the produced results (dashboard feature only). At this point WebTst comes out of the box with three analysers:

Custom Results Analysers

In addition to out-of-the-box results analysers, WebTst was built to support pluging-in of customer specific results analysers. The logic is, as always, that it is impossible to predict the kinds of analysis one might require and to allow a way for people to extend the basic functionality. To add a result analyser you need to: For an example, you can look at the included results analysers (ProcessStepTimeBar.pm, ProcessStepTimeLine.pm, ResultsInXML.pm).
Back to top
Main Menu Close Window