|
By Randall W.
Rice
Every
software application in use today relies on some level of middleware
and firmware to deliver the functionality the user eventually sees.
This places middleware and firmware in a very important, yet often
overlooked, layer of architecture. When approaching the test of
middleware and firmware, people sometimes struggle to find effective
strategies and techniques. The purpose of this article is to outline a
testing strategy and to point you to further resources.
Definitions
Middleware
Middleware
is the layer that resides between the hardware layer and the
application layer to provide services such as database management. From
the CNET glossary, middleware is defined as:
"This software manages the
communication between a client program and a database. For example, a
Web server connected to a database can be considered middleware--the
Web server sits between the client program (a Web browser) and a
database. The middleware allows the database to be changed without
necessarily affecting the client, and vice versa."
The
Software Engineering Institute (www.sei.cmu.edu)
defines middleware as:
"Middleware
is connectivity software that consists of a set of enabling services
that allow multiple processes running on one or more machines to
interact across a network. Middleware is essential to migrating
mainframe applications to client/server applications and to providing
for communication across heterogeneous platforms."
Middleware
can handle transaction processing (TP) for distributed applications,
Remote Procedure Calls (RPCs) for extending application logic across a
network, Object Request Brokers (ORBs) which allow for the distribution
of application objects across heterogeneous networks, and
Message-Oriented Middleware, which allows the passing of messages
between objects and applications across a network.
Figure
1 - Middleware
Firmware
Firmware
can take on many forms, but is basically the embedded software that is
stored in hardware memory to control hardware functions. Firmware
controls PCs, printers, and every other electronic device (remember
Y2K?).
The
Issues
As you
can see, middleware carries quite a responsibility in delivering
quality information. In some recent projects, I have had the
opportunity to delve into the testing issues surrounding middleware.
There are significant testing differences required for middleware and
firmware, however, most testing literature assumes the ability to test
through a user interface. While this article might not have a broad
audience, I often tell people that the information they think is not
applicable today may become very important tomorrow.
Some of
the issues surrounding both middleware and firmware are:
- Complex
processing due to extremely high numbers of interface combinations
- High
criticality in the operations performed
- The
difficulty in testing from an external perspective
- The
importance of developer testing at the unit level
- The
uniqueness of the types of test tools needed
- The
difficulty in building a test environment.
- An
emphasis on structural testing
- An
object-oriented focus
All of
the above concerns shape an overall test strategy that is:
- Performed
early
- Performed
by developers first
- Leveraged
by structural test tools
- Realized
by using test harnesses and stubs
- More
object-oriented than procedural
- Involves
testing large numbers of interfaces
We will
deal with each of these points in the test strategy for middleware and
firmware.
A Test Strategy for Middleware and Firmware
We have
defined middleware and firmware and understand that they are different,
yet have many characteristics in common when it comes to testing. The
discussion of test strategy for these types of software will include
both middleware and firmware, and can be extended to test any software
which is not accessed by a user interface.
Early
Testing
Early
testing will multiply the testing effectiveness of any software
application, regardless of technology. However, in the world of
middleware and firmware early testing is most critical because finding
defects at later stages carries a higher penalty of rework. This is due
to the extent of integration with hardware and other software.
The
problem with early testing in this environment is that with so many
integration dependencies, how does someone create test harnesses and
stubs that allow for an accurate test? Manually, the job is possible,
but can be overwhelming when there are many interfaces involved. If you
are developing in a language that has tool support for structural test
case design and testing, you may find that the job can be very easy.
Specifically, for C++ and Java, Parasoft (www.parasoft.com)
has a great toolset to design and perform structural tests, with a
feature to automatically create a test harness and test stubs. Similar
tools are available from International Software Automation (ISA) www.softwareautomation.com.
Developer
Testing
Developer
testing is essential to avoid high rework costs. To the testers, the
software is a black box. Only the developers have the view and access
to the code to test all conditions. In addition, not only are
functional cases at stake, but also the structural tests for memory
boundary violations and memory leaks.
My
experience is that developers can test software if the have a good
process to follow, standards to show what is expected of them in terms
of testing, and a way to hold developers accountable for the quality of
their work. Management must also be making the message loud and clear
that testing is part of the job and that quality is a shared
responsibility between developers, testers, QA, and management.
An
Object-oriented View of Testing
In the
object-oriented view of testing, tests are isolated at a smaller scope,
yet can have high complexity due to the interfaces with other objects.
The object-oriented view of testing must be able to deal with classes,
methods, and attributes and to validate those at a high level of
coverage.
In Shel
Siegel's book, "Object-Oriented Software Testing," he describes the
Hierarchical approach to O-O testing.
"The hierarchical approach is at
the heart of the object-oriented testing system. This test approach
uses and builds upon several well-understood testing techniques, tying
them together into a comprehensive testing system. The hierarchical
approach leverages the fact that "everything is a system." It defines
and applies testing standards for several levels of software component:
objects, classes, foundation components, and systems. The hierarchical
approach designates as SAFE those components that meet the testing
standards for that kind of component. Once you designate a component as
SAFE, you can integrate it with other SAFE components to produce the
next-level component. In turn, you test this component to the level of
safety associated with the component level it represents. SAFE is
always a relative state. It depends entirely on the standards you
choose to enforce, your application, your attitude toward risk, and the
specific risks and risk management practices you adopt in your project.
The hierarchical approach provides guidelines for minimum safety; you
decide what is right for you."
Figure
2 - The Hierarchical Test Approach
Much
more could be said about testing middleware and firmware. This article
is intended to provide a strategy for testing and a starting point to
start developing detailed techniques. My best advice to you is to read
the books on Object-oriented testing by Shel Siegel and Robert Binder,
investigate the available test tools (new ones are emerging all the
time) and work at the developer level to reinforce the criticality of
testing.
All materials on this site
copyright 1996 - 2008, Rice Consulting Services, Inc.
Rice
Consulting Services, Inc.
P.O. Box 892003
Oklahoma City, OK 73189
405-691-8075
"Leaders
are made, they are not born. They are made by hard effort,
which is the price which all of us must pay to achieve any goal that is
worthwhile." -- Vince Lombardi
This site best
viewed with the Mozilla Firefox
browser!
|
