Buy it Today!

Randy's book, Surviving the Top Ten Challenges of Software Testing, will help you solve some of your toughest testing problems: people problems!

Click on the image to buy it from bn.com.

by Randall W. Rice

Abstract

Traditionally, testers have been behind the development curve in defining and adopting new test strategies. This article is a strategy-level overview which describes the background and future of wireless technology, the major risks and critical success factors, an overview of the technologies used to deliver wireless applications, and a strategy for testing wireless applications.

The Hype vs. The Reality

There is a great deal of promise in the future of wireless applications. However, in many ways, wireless technology is a plane that has not yet taken off. The forecasts of wireless technology adoption vary greatly, depending on the source. Major problems, such as inconsistent service levels, bandwidth constraints and usability factors have yet to be solved.

Consider the results of the Forrester Group survey recently published in the April 2001 issue of M-business magazine. In this survey of North American PDA and Web Phone users, over 85% of current users of these devices indicated they were either uncomfortable or very uncomfortable about using the wireless features of those devices for M-commerce. Is it any wonder? How many times do you get disconnected from a wireless phone call, or find yourself in a "dead zone"?

If you examine where the M-commerce users are, A study by M-business Research finds that North America lags behind Europe and Eastern Asia in the percentage of the global population of M-commerce users, with 21% of the world's M-commerce users. If projections hold true, the M-commerce user base may become even more prevalent in Europe and Eastern Asia.

When it comes to projections of future wireless and M-business activity, the outlook depends on which forecast you read. The forecast from ETFORCASTS predict that the number of U.S. wireless web users will skyrocket to 90 million people by 2005. However, one of the more significant findings was that the number of European wireless web users is predicted to outdistance those in the U.S. by a 2 to 1 margin by 2005!

What do these trends and forecasts mean in terms of testing? For the first time, in my memory at least, the U.S. will be a follower instead of a leader in IT applications. This means that many of the lessons learned will come from Europe and Eastern Asia instead of here. This also means that people in the U.S. will have to seek out expertise from other countries in finding out what works and what doesn't in wireless web and M-commerce applications.

If you look carefully at the issues behind wireless web applications, you will start to see flawed business logic. Is this "deja vu all over again"? For example, one might think that it would be helpful to check airline departure times from your web-enabled phone or PDA while you are in the taxi on the way to the airport, stuck in traffic. Consider this, though. How often are the flight times posted on the monitors in the airport correct?

It seems that most of the activity, benefit and confidence of the wireless web is coming from wireless web vendors, not actual users! See Figure 1.

If you want to read an interesting article on this topic, go to http://www.mbizcentral.com/m-business_story/new/05_top10 to read the article, "The Top 10 Lies Wireless Marketers Tell."

Wireless Technology is More Than the Wireless Web

Even in this time of uncertainty, people are working on pilot projects using wireless technology and we are seeing a variety of types of wireless applications. Wireless technology is more than the wireless Internet. People are using wireless technologies to build long-range wireless networks and perform short-range personal wireless functions. Some of these wireless applications are extremely critical in terms of risk. Paging systems allow medical professionals to view an EKG chart and other diagnostic data using pagers. Wireless scanners help stores keep track of inventory levels.

Regardless of the application and specific technology, wireless applications require a unique testing strategy that can be identified and documented. In addition, although the future of the wireless web is hard to predict in terms of when and how it will actually take off, it is time to start thinking about the implications of testing wireless applications. Even if the wireless web is very slow to gain stride, many businesses will be building and deploying other types of wireless initiatives to serve customers better and more efficiently.

Some examples of other types of wireless applications beside the wireless web and M-commerce include:

• Handheld devices for applications such as inventory control and shipping
• Pagers for instant messaging and data delivery - even for applications such as medical diagnostics
• PDA applications to sync with notebooks computers and to perform functions such as automated hotel check-in and check-out while in the hotel
• Peer-to-peer applications for workgroup collaboration and gaming
• Wireless networks to allow maximum flexibility in workspace configuration

How Wireless Applications are Developed

 To develop a test strategy for wireless applications, we need to know how they are developed in terms of tools and technologies. We don't have time in this article to cover all technologies, but some of the major ones include:

Wireless Access Protocol (WAP) - An initiative that was started by Unwired Planet, Motorola, Nokia, and Ericsson to define a secure specification that allows users to access information instantly via handheld wireless devices such as mobile phones, pagers, two-way radios, smartphones and communicators.

Bluetooth - A short-range (up to 10m) 2.4 GHz radio technology aimed at simplifying communications among Net devices and between devices and the Internet. It also aims to simplify data synchronization between Net devices and other computers. Products with Bluetooth technology must be qualified and pass interoperability testing by the Bluetooth Special Interest Group prior to release. The Bluetooth 1.0 specification consists of two documents: the Foundation Core, which provides design specifications, and the Foundation Profile, which provides interoperability guidelines.

XML - Extensible Markup Language - allows designers to create their own customized tags, enabling the definition, transmission, validation, and interpretation of data between applications and between organizations.

WML - Wireless Markup Language is an XML language used to specify content and user interface for WAP devices. WML is supported by almost every mobile phone browser around the world. WML pages are requested and served in the same way as HDML pages.

WMLScript - Scripting language used with WML to allow interactivity on WML applications.

It is important to understand that just like web applications are often built using a combination of technologies such as XML, Java, and Java Script, so are wireless applications. The mix depends on the application objectives, project objectives and business strategies.

Wireless application developers use development tool kits provided by the wireless device vendors, such as Nokia, Ericcson and Motorolla. These development kits also contain simulators that allow developers and testers to test wireless applications in a controlled environment.

The Big Picture - How M-Commerce Applications are Delivered

The above figure shows how the M-commerce user conducts a transaction. One of the major differences between this model and the E-commerce model of recent years is the prevalent use of a mobile wallet to facilitate fast and secure payments. Like the E-commerce model, integration is the key to making the business side of M-commerce work.

Primary Concerns and Risks of Wireless Applications

A very useful starting point for a test strategy is to examine the risks. For wireless applications, the risks are:

Correctness - Like any other application, if the information is not correct, people will abandon the application. Here are two interesting findings from Price Waterhouse Coopers survey of 600 mid-sized and dot-com companies in the U.S., U.K. and Australia, as reported in The Industry Standard, April 18, 2001:

1. "Only 1 in 3 traditional companies and only half of the e-businesses, which should have state-of-the-art technology, reported being "very confident" about the quality of the data they collect."

 2. "A third of e-businesses and a quarter of traditional companies surveyed said they had lost sales because of poor information about what inventory they had in stock."

These are shocking findings from both the customer and business perspective!

Usability - Wireless applications shrink the user interface to the extent that two concerns emerge greatly when dealing with wireless usability.

1. Interface - The display size of wireless devices is often very small. This affects readability and feature sets in a big way.

2. Ergonomics - Wireless devices are compact and have smaller keys for input. If you have small fingers, the issue is lessened, but if you have fingers that resemble small sausages, you need another way to input data beside your fingers.

What does the research show on wireless usability factors?

1. Most WAP users rate usability very low.

2. Over 50% of wireless users in the U.S, U.K and Finland surveyed by Accenture saw "no compelling reason" to use wireless web services. They cited factors such as poor interface design and slow performance as key reasons for their lack of interest.

3. Even small usability flaws loom large in wireless apps.

Security

Even though WAP is designed to be a secure technology when used correctly, there is a big difference between knowing and doing when it comes to security. Hackers have become adept at "cell phone cloning" and can intercept phone transmissions out of the air. This happened to me personally while on a trip in New York City a year or so back. Hackers intercepted one of my phone calls and proceeded to charge over $4,000 of phone calls to my account. Fortunately, the people at my phone service provider were helpful and understanding, so I didn't have to pay for the fraudulent calls. However, it makes me think twice about what I would do via wireless technology.

In addition to the street threats, there are also the security issues of what happens to the transaction at the gateways, which are controlled by nameless, faceless third parties. Peer-to-peer computing also holds risks for the privacy of data sent between devices.

Performance

Performance issues can result from a variety of sources including:

Slow infrastructure

One of the most pressing challenges in wireless applications is the growing demand on bandwidth. This demand grows daily as new users are coming onto the web and as businesses increase the quantity and quality of media content, such as streaming media.

There is considerable discussion as to whether or not third generation (3G) networks will ever become a reality in the U.S. In addition to slow networks, the reliability of wireless connections leave much to be desired. It's one thing to be able to decipher a garbled voice message, but another to transmit a stream of data across the same faulty connection.

Client bottlenecks

Performance problems can also be traced back to client bottlenecks in WAP applications as well as hardware limitations with the wireless device.

Integration

As in any robust application, wireless or otherwise, integration often plays a major role in the ability of the application to deliver correct and efficient results. Wireless integration concerns include:

• Online wallets for fast and secure online payments
• Business systems which must handle the transactions generated by wireless devices
• Other devices which receive data from wireless devices:
  • Notebooks, which may sync with PDAs
  • Vending machines, which my provide inventory control data to an central business system
  • Handheld units, which may transmit inventory or shipping data to central systems
  • Web-enabled phones, which my send data to other cell phones such as with instant messaging

Flexibility

People want to make the most of their investment in wireless technology and are often cautious about hardware and applications that may become obsolete in a year or less. Therefore, flexibility is a very important issue to wireless users. Flexibility in this context can include:

• Ease of upgrades to software and hardware
• Application flexibility with various hardware and wireless operating systems.

Profitability

This is an opportunity to apply some of the lessons learned from the recent dot-com boom and bust experience. Wireless providers must be able to prove that the envisioned users will actually buy the applications and services under development. In addition, the wireless development companies must be able to survive until the wireless market can support the new technology and pray that a new technology does not catch on and render their product or service obsolete.

Some questions that should be asked about any project include:

• Does the application solve a problem effectively and easily?
• Is there a customer base?
• Can development costs be recouped quickly?

A Testing Strategy for Testing Wireless Applications

Now that we have examined the basis of wireless applications and the risks of wireless technology, let's look at a strategy for testing wireless applications, regardless of the method of application.

The major items to consider are:

• Type of Wireless Application
• Audience Demographics
• Technologies Used
• Tools
• Test Execution Strategies

Type of Wireless Application

Some possibilities include:

• WAP
• M-Commerce
• Workgroup
• Personal productivity
• Peer-to-peer
• Business system integration
• Entertainment

The type of application will determine the type of tests you will need to perform.

Audience Demographics

This will dictate the types of testers you will need to get a representative assessment of usability.

• Age (The most likely acceptance group for wireless apps are teens!)
• Geography - This applies to issues of localization as mobile applications are deployed in a variety of countries.
• Web comfort level - This will determine the important of usability factors.

Technologies Used

This will determine the type of test environment and tools you will need.

• WAP
• Bluetooth, etc.
• XML, WML
• Security protocols

Tools

• WAP Simulators by hardware vendor: Nokia, Ericsson, Motorola
• Capture/Playback - such as TestPartner by Compuware
• Performance Testing - such as QA Load Compuware

Test Execution Strategies

Pilot tests

• Other countries - With early deployment in other countries, it is possible to introduce a product to a smaller and more remote audience. You can practice in a small room!
• Limited user base - This type of test would include usability testing with a small number of users to get an assessment of initial usability.

What Does This Mean for Testing Wireless Apps?

 It's a small world after all!

User interfaces tend to be much smaller than other applications, which translates into smaller test cases.

There's a large part of testing wireless apps that is like other testing.

This includes tests for validating the integration of transactions, platform portability and integration testing, and having a basis for testing such as use cases and/or requirements.

Wireless test strategies can take many forms

Just a few of the many flavors include M-commerce, data collection and delivery, entertainment and workgroup collaboration.

Tools can help

This is especially true in regression and load testing.

The test lab is greatly different than the real world.

Simulators are great, but you need to test also with real people in real-world conditions.

Summary - Dos and Don'ts of Testing Applications

The world of testing wireless applications is growing and changing daily. I'm sure that within a few weeks, I will learn even more considerations for this rapidly changing technology. In the meantime, here are my closing thoughts.

Do:

• Ask questions early about application feasibility
• Remember that wireless technology is still very young
• Perform early usability testing
• Be prepared for an onslaught of new development when people and the technology are ready.

Don't:

• Forget that you still need something to base tests upon, such as use cases and requirements.
• Forget that much of M-commerce is based on integration between organizations.
• Forget your audience.

I hope this article helps in your search for information on testing wireless applications. I would like to hear about your challenges and success stories. You can e-mail me from the contact page.

Latest News

New Online Course - Gathering, Documenting and Testing User Requirements!

New Course Series - IEEE Standards

Basic Training in Software Testing Course Now Online!

Join Our Free Newsletter

RCS e-Learning

Quote
"Leaders are made, they are not born. They are made by hard effort, which is the price which all of us must pay to achieve any goal that is worthwhile." -- Vince Lombardi

Home   About Us    Site Map     Events    Training   Consulting   Resources   Community

Newsletter    Contact Us    Careers   Certification    The QA Zone

All materials on this site copyright 1996 - 2005, Rice Consulting Services, Inc.

Rice Consulting Services, Inc.
P.O. Box 6127
Moore, OK  73153
405-691-8075