|
USER’S GUIDE
version=1.012 04 Mar 2002
Contents
1. Introduction
Welcome to the SiteTester User’s Guide. This guide explains how to use SiteTester to
load-test web sites and web applications.
About SiteTester
SiteTester is a utility application, designed to load-test web and application servers. SiteTester
simulates concurrent access of various users from different hosts to a web/application server. Each virtual user
follows a predefined procedure to access the server. By increasing the number of virtual users you can test the
capacity of your servers and determine the extra hardware and bandwidth needs, if any.
Many web applications fail to operate on simultaneous uses, despite every effort to test and debug the application
on the development environment. This is mainly due to the nature of multi-threaded programming, which makes it difficult
to simulate the actual process. Application codes, which are not multi-thread-safe, which leak memory, which may produce
deadlocks, have great affect on the performance and stability of the web sites. SiteTester allows web application
developers to test their codes amongst many users, and take precaution before the actual deployment occurs.
Features include:
- Allows definition of requests, jobs, procedures and tests.
- Allows HTTP1.0/1.1 compatible requests.
- Allows POST/GET methods.
- Allows user-defined pauses between jobs, for each virtual user.
- Stores and handles cookies.
- Allows applying the same test(s) on different occasions.
- Displays test progress visually.
- Allows running each job, in multi-threaded or single-threaded mode.
- Allows repeating the procedure any number of times, for each virtual user.
- Generates various reports in HTML format.
- May gather and analyze data, collected on different hosts.
- Keeps and reads XML formatted files for test definitions and test logs.
2. Installation
Installing the SiteTester utility is a simple process of extracting the zip archive to some directory on your machine,
with execute and write rights.
SiteTester is a Java application, compliant with the JDK1.2 or later, so a java virtual machine (JVM) is required to run
this software. If not already installed, you can download and install the JVM from
Sun Microsystems.
3. Running SiteTester
Before running SiteTester you have to edit the SiteTester.bat file (or SiteTester.sh on UNIX). Point the JAVA_HOME
parameter to the root of the JVM (e.g. C:\jdk1.3.1_01\jre).
In order to launch the application on windows, double-click sitetester.bat. On UNIX, execute "sh ./SiteTester.sh"
command from the shell.
The application frame should open when the scripts(s) succeed to run.
4. Demo mode limitations
If not registered, SiteTester runs in demo mode, which is intended for evaluation use only. There are some limitations
in the demo mode, however the demo mode operation does not prevent you from evaluating any features of the utility.
Here is where the limitations take place:
- Only 3 concurrent users are allowed per test instance.
- Only 3 jobs can be run per virtual user
- Only 2 requests can be run per job.
There are no limitations in the definition of tests, jobs, users etc; the system just discards the values that are out of limits.
5. Defining the procedures
Before running tests, you have to define the actions to be performed by the virtual users. The definitions tree includes the elements
site, job, job-list, request, test, session and run. Here are the meanings of these elements:
Site: A workspace involving tests and test definitions related to one site. (Such as www.pilotltd.com)
Request: An http request.
Job: A simple task such as loging on to a system, or loading a page. Every job has a number of http requests.
Job-list: A list of jobs describing a procedure to be followed.
Session: A virtual user. Every virtual user has an associated job-list.
Test: Describes a test procedure for this site. Every test has a number of sessions.
Run: Indicates the results of a test. Tests can be run many times and results of every instance are kept until you delete.
In order to define the test procedures you have to add these elements on the tree and thereby compose the whole test definition(s).
Here is a description of how to use the tree to add/delete elements.
- If you select an element with the left mouse button, the details pane on the right of the window, displays the details of the
selected element. You can edit/change the content of the elements from here.
- The selected elements content is updated when you select any other element on the tree.
- If you press the right mouse button on some tree element, the private menu of the selected element will appear.
- In order to delete an element, select the element on the tree, open its menu and click "Delete".
- In order to add new elements to a list click the list node, open its menu and click "Add".
- In order to save changes to file, select the site of concern, open its menu and click "Save".
You can alternatively choose "Save All" from "File" menu to save all the sites.
6. Running tests
Having defined all the test procedures and virtual users, running tests is a simple process. Just open the site of concern,
select the test, open its private menu by pressing the right mouse button, and select "Run". The progress indicator
window will open after you follow the directions. There will be a new element under the "Run" directory of the test,
which will summarize the results of the run, when selected. This node will be used to generate and view the reports.
You can run the same test on different hosts simultaneously (they may also be distributed on different domains). In order to do
this, you have to install SiteTester to every client host. You don’t need to create the test definitions for every host,
it is possible to create the test definitions once and copy them to other hosts. All the definitions related to a site
are under the directory "\sites\sitename" within the SiteTester install directory. You only need to copy this directory
under the sites directory of the other hosts.
When multiple hosts do the same test on this site, you have to gather the results on one of the hosts in order to use the
whole data for analysis. SiteTester stores the results in an XML file located at:
SiteTesterRoot\sites\sitename\testname\runname\log.runname.hostname.xml
The hostname is required every time you call the run method. You must give different names for each host, so that the file names
will not mix-up.
7. Generating and viewing reports
You can generate statistical reports on a test instance after the test is completed. Just click on the run-instance under the
tests "Run" directory, right-click for the menu to open and select "Generate Reports".
The default browser should open the first page of the reports, which contains links to other pages, after the generation is completed.
Report generation may take some time if the resulting data is large.
The browser may not open due to problems in finding the default browser; in this case you may try to open
the reports yourself. The HTML files are under the directory:
SiteTesterRoot\sites\sitename\testname\runname\reports
Open the file "runname_index.html" on this directory, to view the reports.
If multiple hosts are involved in the test process, copy the data files described in section (6) to one of the hosts,
after all the tests are completed. For example if 4 hosts (namely host1, host2, host3, host4) are loaded with test1, the host
with the complete data should have the following files before generating reports.
SiteTesterRoot\sites\site1\test1\run1\log.run1.host1.XML
SiteTesterRoot\sites\site1\test1\run1\log.run1.host2.XML
SiteTesterRoot\sites\site1\test1\run1\log.run1.host3.XML
SiteTesterRoot\sites\site1\test1\run1\log.run1.host4.XML
The generated reports will contain the analysis of all the hosts, from now on.
In order the view or review the test results, just click on the run instance, open the menu and click "View Reports".
8. Reports description
SiteTester generates various analysis on the tests carried out. Some of the analysis contains cumulative (average or sum)
values of the individual records. Here is a description of some fields found in the reports:
- Connect time: the time, in milliseconds, passed to establish a valid HTTP connection to the defined host, on the defined port.
- Send time: the time, in milliseconds, passed to send the HTTP request and post-data, if applicable,
to the server, over the established connection.
- Receive time: the time, in milliseconds, passed to receive the HTTP headers and content body of the response.
- Total time: the total time, in milliseconds, passed for the request. This is equal to the sum of connect, send and receive times.
- Return code: the HTTP return codes are:
2xx: The document follows
3xx: The document is moved
4xx: No such document/you can’t access the document
5xx: Internal server error
- Errors codes:
10: The bytes received is less than the content length header value, if any.
20: There is response from the server, but the return code is not 2xx or 3xx.
30: The URL is not valid. Either the protocol cannot be found or the URL string cannot be parsed.
40: I/O Error. The connection is broken or some protocol error occured.
51: Bind error. The socked connection cannot be established due to a binding error.
52: Connection error. The socket connection is refused. There might be no process on the remote address/port.
61: The specified remote address cannot be reached. This may be due to local routers and firewalls.
62: The IP address cannot be determined.
63: Unknown service error.
70: Unknown error.
- Bytes/s: the total bytes receivedX1000/receive-time.
- Hits: Shows how many times this URL is called.
- Requests: Shows how many requests are involved in this analysis.
- Session:Job: Shows the belonging virtual user id, previous repeat count, and the job name in progress, in a short notation.
|