Site Map | FAQ
FREE Articles and Resources
Articles and Book Reviews
Tell a Friend
Link to Us!
How to Develop Test Cases and Test Scripts for Web Testing
Test scripting has been a popular way to build testware since the advent of interactive software. With Graphical User Interfaces and web-based interfaces, the applicability of test scripts should be questioned. In some cases, test scripts may apply very well to object and component-based interfaces, while in other cases they may not apply well at all. This article examines the nature of test cases and how to determine the best vehicle for designing a test in the web-based environment.
Understanding the Components of a Web Test Case
Test terminology has reached the point where people use a variety of terms innon-standard ways. One of the most inconsistently used test terms is that of a "test case." There are several definitions I could use for test cases, but the one I like the best is used in the Quality Assurance Institute's Effective Methods of Systems Testing training course. In this definition, a test case is comprised of a test condition, an expected result, and a procedure for performing the test case.
I like this definition because it implies that test cases can be performed either in combination with other test cases or in isolation. This is an important distinction when testing web interfaces, which allow a user to perform actions in a random and unpredictable manner. Modular test cases are like objects which can be performed in isolation but often have the requirement of interfacing with other test cases to achieve a particular objective. The important concept to remember is that the sequence of performing test cases may be unpredictable from the test designer's perspective, but can be critical in manifesting a defect.
How to Determine When to Use Test Cases vs. Test Scripts
When deciding to use test cases vs. test scripts, its much like deciding to take a guided tour of a museum or browse around on your own. Much depends on how much time you have, your prior knowledge of what you're viewing and the freedom you would like to have during the experience.
Likewise, in the decision of using test scripts vs. test cases, much depends on the context of the application you’re testing. This could include the tester’s level of knowledge of the application, the amount of freedom the user is intended to have in using the web application and the amount of time available for testing.
The decision to use test cases versus test scripts depends on:
The main benefit of a test script is that it predefines a procedure to follow in performing a test. This can also be its greatest curse. Sometimes you want the randomness of user actions. At the same time, you want to know in advance the conditions to be tested and how they should behave. This is the classic tradeoff between test scripts and test cases.
Just because you choose to use a test script in testing one area of your application does not imply you must use test scripts for testing everything in an application. You may choose to only create test scripts for the definable processes that are critical and that apply well to scripting. For everything else you could decide to use test cases.
The Sources of Web Test Cases
Many web testers describe the process of testing web applications as trying to hit a moving target. Just when you think you have things in place, they change. In fluid development environments like the web, software development happens quickly and functionality can change just a quickly. Sometimes the testers know what the web application requirements are, but many times the requirements tend to take shape over time as developers and the customer/users interact.
Before we can define a test case, we need to know how the web application is intended to behave. There are many sources of test cases in a typical web application and I will examine the ones that give you the best results with the least amount of effort in test case design and execution.
Web Test Case Components
The structure of test cases is one of the things that stays remarkably the same regardless of the technology being tested. The conditions to be tested may differ greatly from one technology to the next, but you still need to know three basic things about what you plan to test:
ID #: This is a unique identifier for the test case. The identifier does not imply a sequential order of test execution in most cases. The test case ID can also be intelligent. For example, the test case ID of ORD001 could indicate a test case for the ordering process on the first web page.
Condition: This is an event that should produce an observable result. For example, in an e-commerce application, if the user selects an overnight shipping option, the correct charge should be added to the total of the transaction. A test designer would want to test all shipping options, with each option giving a different amount added to the transaction total.
Expected Result: This is the observable result from invoking a test condition. If you can’t observe a result, you can’t determine if a test passes or fails. In the previous example of an e-commerce shipping option, the expected results would be specifically defined according to the type of shipping the user selects.
Procedure: This is the process a tester needs to perform to invoke the condition and observe the results. A test case procedure should be limited to the steps needed to perform a single test case.
Pass/Fail: This is where the tester indicates the outcome of the test case. For the purpose of space, I typically use the same column to indicate both "pass" (P) and "fail" (F). In some situations, such as the regulated environment, simply indicating pass or fail is not enough information about the outcome of a test case to provide adequate documentation. For this reason, some people choose to also add a column for "Observed Results."
Defect Number Cross-reference: If you identify a defect in the execution of a test case, this component of the test case gives you a way to link the test case to a specific defect report.
Sample Business Rule
A customer may select one of the following options for shipping when ordering products. The shipping cost will be based on product price before sales tax and the method of shipment according to the table below.
If no shipping method is selected, the customer receives an error message, "Please select a shipping option." The ordering process cannot continue until the shipping option has been selected and confirmed.
The table below shows how the shipping options of an e-commerce application could be independently tested.
Table 1 - Test Case Examples
Things to Notice in This Example
At this point you might be wondering where the test data reside and how to know the expected results. There are a variety of ways to create and maintain test data, but it could be as simple as creating a PC-based database or spreadsheet. The test data can then be obtained manually through a cross-reference, can be imported into an automated test tool, or can be entered into the application using a home-grown scripting procedure (This will be the topic of future articles.).
2. There is no sequence implied or necessary for the test cases. The important thing is that all of the conditions are tested, not necessarily the sequence. Consider the scenario where a customer selects a certain form of shipping and then decides to select a less expensive option. While it is possible to script such a scenario, the possible combinations of such scenarios can be overwhelming and not very profitable when it comes to a return on your testing investment.
Test Case Design Strategies for Web Applications
In the search for the best way to test your web application, you will likely have several possible approaches. It's difficult sometimes to decide which test case development approach will eventually work best, but here are some common test case strategies. The intent of this discussion is to focus on how to develop testware using a variety of approaches, many of which have been effective in other technologies and are still useful in testing web-based applications.
To arrive at the approach that works best for you, you will need to consider:
Each of the following approaches to design web-based test cases has its strengths and weaknesses.
Test scripts became popular when mainframe applications went interactive. These procedural "green screen" types of interfaces are typically very predicable in nature and the user is not given much latitude in how they can interact with the application. These procedural user interfaces gave rise to written, or manual, test scripts that told the user what to enter in each screen field, then to go to the next field, etc.
Then came the Graphical User Interface (GUI). With the advent of the GUI came the promise, and sometimes the ability, of the user to work according to their own preferences and not necessarily to a rigid procedure. The impact on test planning is that it may impossible to predict all the ways a user may perform a process while using a GUI.
Web pages tend to be visually-oriented and strive to deliver the same type of user flexibility as a client/server GUI. In addition to the web browser GUI, a typical web page may contain:
You might be asking, "But what about GUI automated test tools? Don't they use recorded test scripts as the primary means for testing?"
Yes, they do. Each script follows one path through a process, or on a smaller level, each script describes a sub-process in a modular manner. Modularity does a lot to help reduce the large numbers of test scripts required for some GUIs, but you can still only describe so many combinations of possible actions before you run out of time, people or money.
The issue is whether or not you need to predefine a sequential order of actions for testing a particular process. Since a test script describes sequence, actions and expected results, the level of interface "randomness" affects the effectiveness of test scripting. Before we decide to eliminate test scripts as a tool for testing web applications, there are some important considerations:
Test scripting can involve a variety of forms and methods. I've seen about as many test script standards as I have seen testers. To illustrate, here is a sample manual test script format. This test script format is designed to handle tests that are isolated to one page in the web application, or that span multiple pages of the application as well as other system processes over a given period of time.
Moving Into Automation
The reason I have laid the foundation in manual test cases and test scripts is because much test analysis requires getting the manual process documented first. The main point in automating tests is the ability to repeat them later. The best thing you can do to facilitate repeatable tests is to plan them to be repeatable. This implies thinking about modularity and keeping scripts free of embedded data except where absolutely essential. In other words, let's take time at the beginning and define the problem before we immediately start solving it.
Once you have the manual concepts mastered and once you have some idea of the processes and cases you want to automate, then you are ready to start thinking about automating the test. There are also other things to consider, such as naming conventions, test script management, etc.
Actually, this is where we arrive at where we started this discussion: using test cases to design a modular test. Once we can define the triggering events (test conditions) and have a way to observe the outcome (expected results), then we can design and perform a test, whether it be manual or automated.
An exciting advancement in test automation is that some of the tools are able to generate test cases automatically from the web interface. That's a step in the right direction, but there is still a need to define tests that span interfaces, such as in the case of ordering a product. It's the integration of functions that give test designers their greatest challenge.
All materials on this site copyright 1996 - 2009, Rice Consulting Services, Inc.
Consulting Services, Inc.
"Leaders are made, they are not born. They are made by hard effort, which is the price which all of us must pay to achieve any goal that is worthwhile." -- Vince Lombardi
This site best
viewed with the Mozilla Firefox