|
There are two methodologies that organizations are using to declare their
commitment to quality. These methodologies are ISO 9000 and SEI CMM. While
these methodologies may not be thought of as tools, they each possess the
characteristics of tools. That is, they are used to construct, maintain, and refine the
essential ele ments of quality for products produced in an environment. Both the ISO
9000 Standards and the SEI CMM provide separate and distinct sets of guidelines for
attaining and measuring quality. As such, each of them in their own way has caused
the redefinition of quality.
No longer is quality considered an illusive term in the software community, promised
by many, delivered by few. Quality is no longer just about testing software. Both the
ISO 9000 Standards and the SEI CMM set forth guidelines that force the
establishment of processes and procedures that extend well beyond testing and even
beyond the MIS, IT, IS, and data processing organizations.
Each of these sets of guidelines, while developed independently of each other, is
designed to resolve specific proble ms for the developers and customers. Both sets of
guidelines are based on the premise that if an organization has good business
procedures that can be reviewed, assessed, and graded, then the organization can
be determined to produce quality products. The question it raises is: Do process and
procedures, even when adhered to, really ensure quality? The answer is, unrefutably,
no.
Managing quality goes beyond the institutionalization of processes. While a good
quality management program will have defined and repeatable processes, what must
really be defined are the quality goals as they pertain to a company’s specific
business. That means that before quality processes are defined, the company will
have performed the analysis necessary to determine what the characteristics and
attributes of quality are for their products, their customers, and their environment.
The success of the quality program will lie in the clear definition of productivity and
quality goals, a solid explanation of the value in achieving the goals, and both formal
and informal communication about all the aspects of the goals. The approach for any
quality program must be commitment, consistency, and willingness to continuously
improve.
To really have an active and successful quality management program, the culture of
the company must be aware of the investment that will be required and the benefits
of the program to them. This is accomplished by defining and measuring quality in
products and people as well as processes. All of this has to be done considering the
environment in which the company operates and the nature of the competition. The
inherent danger in using ISO Standards and the CMM lies in producing the
procedures and the paperwork that allow certification or rating to be granted without
providing the education and integrating the processes into the corporate culture to
ensure real quality.
Unfortunately, as consumers demand more proof of quality, organizations feel
pressure to achieve the ISO 9000 certification or SEI CMM rating in order to make
the company look good. More and more organizations are trying to determine how to
make either or both of these quality management structures work for them.
Questions are being raised, such as: What CMM level does one need to achieve in
order to become ISO certified? Why does one have to choose between them?
There are even people dedicated to drawing the parallels between these two sets of
guidelines. There are, however, some inherent dangers in traveling down that path.
The danger is not because the two sets of guidelines are incompatible. Quite the
contrary; there are parallel points between them. The problem is that they are two
separate and distinct things. One is a standard that requires compliance and
provides for certified proof of quality. The other is a model that can be evaluated and
validated to show capability to produce quality.
Thus, it may be assumed to be correct to use CMM to achieve ISO 9000 certification!
Wrong! It is not that this is an impossible task, but there are two problems that
surface when this approach is attempted. First, both the ISO Standards and the CMM
have unique and distinct vocabularies that are used in the program and the
certification/evaluation processes. Second, the certification/evaluation processes are
conducted differently. Thus, when on the CMM train, one does not automatically end
up at the ISO 9000 station.
Since the costs associated with implementing either of these methodologies can be
formidable, it is important to understand both methodologies and to determine, in
advance, which structure best fits the needs of the organization. Then, one can build
or improve on the internal quality management program from there.
ISO 9000 STANDARDS
The International Standards Organization (ISO) was created as an economic
undertaking to ensure that agreements between countries have a solid value base.
The primary objective of ISO, as stated in its statutes, is to promote the
development of standards and related materials to facilitate the exchange of goods
and services between countries and to develop cooperation within the intellectual,
scientific, and economic communities. To this end, the ISO structure supports
technical advisory groups and technical committees for the standardization of goods
and services in 172 areas ranging from steel, tractors and machinery for agriculture
and forestry, to cinematography, air quality, and biological evaluation of medical
devices. The technical committees are structured into subcommittees to ensure focus
on specific areas within their major field. Work is performed by working groups
defined within the subcommittees and approved by the ISO general assembly.
Included within the family of technical committees, their designated subcommittees,
and approved working groups are two committees more pertinent to software
development and system integration than others. These technical committees are:
Technical Committee 176 (TC176), and Quality Management and Quality Assurance
and Joint Technical Committee 1 (JTC1), Information Technology. The implications of
work performed by TC176 has had a steadily increasing impact in the software world
during the past 5 to 7 years.
Within the JTC1, Subcommittee 7 (SC7) was established to address standardization
of software engineering. It was in 1982 that A. Neuman, from the National Institute
of Standards and Technology (NIST), petitioned ISO to change the United States
membership status from observer to principal member. As a result, the number of
U.S. member companies and individual technical experts participating in ISO work
grew substantially. With that growth came an increase in the scope of influence on
newly developed standards and the revisions of existing ISO Information Technology
(IT) standards. This influence has been greatest in JTC1/SC7, which has undertaken
the development of Software Engineering and System Documentation standards
worldwide.
As a principal member, the United States has become a major player in JTC1/SC7.
Many of the U.S. Department of Defense Standards and Military Specifications have
been introduced into global working groups as a starting point for revamping old
standards and developing new standards. Software development and system
documentation standards approved by the Institute of Electronic and Electrical
Engineers (IEEE) have also been introduced.
Similarly, the Canadians, the British, the Germans, the Australians, and other
member countries have brought their country’s existing standards to the table.
These existing standards are discussed, revised, and rewritten at the working group
level until the international membership reaches a consensus. Only then is the work
submitted to the entire subcommittee for a vote. An affirmative vote places the
standard on the ISO calendar for action.
Because JTC1/SC7 and TC176 are working together to ensure that the standards for
quality management adequately address software quality needs, software developers
and system integrators need to take the ISO Software Engineering Standards into
consideration when electing to focus their energies on achieving ISO 9000
certification.
The set of guidelines that have become known as ISO 9000 were established through
the International Standards Organization. ISO 9000 is actually a series of standards.
The ISO 9000 series comes complete with a certification process that conveys
recognition of quality achievement for a specific ISO 9000 standard, as determined
by a registered external auditing team. For instance, a company may be ISO 9001
(Model for Quality Assurance in Design/Development, Production, Installation, and
Servicing) certified, ISO 9002 (Model for Quality Assurance in Production and
Installation) certified, or ISO 9003 (Model for Quality Assurance in Final Inspection
and Test) certified, depending on the type of product being produced. There is no
such thing as a blanket ISO 9000 certification.
At first, the ISO 9000 series appeared to focus only on manufactured goods and
services and many people felt this series of quality management standards would
never impact the software community. Software companies have tried to convince
themselves of the insignificance of ISO 9000 in the software development
community. Some of the arguments that have been heard included statements that
this set of standards were too loose and too vague to be able to ensure quality of
developed software. This, of course, was a matter of interpretation that may have
initially had some degree of truth to it. Taking this under advisement, Technical
Committee (TC) 176, which was initially chartered to standardize quality
management by the International Standards Organization headquarters in Geneva,
Switzerland, undertook the tremendous effort of updating the ISO 9000 Standards.
Some of the issues were successfully resolved in the revisions; others still beg to be
addressed. Nevertheless, the argument that ISO 9000 standards are not useful in
software development companies has faded away.
Another argument, used primarily in the United States, was that this standard was
not going to have an effect on U.S. companies. Its popularity and usefulness in
Europe and Pacific Rim companies made sense, but U.S. companies felt that they
were beyond compliance. Wrong! It was not long before companies whose tentacles
reach out beyond the shores of the United States began to seek ISO 9000
certification in order to maintain their competitive option in their overseas operations.
The ripple effect of this led to the creation of ISO certified companies within the
United States from whom quality systems, services, and products could be bought.
National companies now had to reassess their own positions based on the
implications of these standards on their market.
SEI CMM
The development of the Software Capability Maturity Model (CMM) was undertaken
at Carnegie Mellon’s Software Engineering Institute (SEI) beginning in 1986 under
the sponsorship of the U.S. Department of Defense. Work on the CMM continues
today; it is a living document that espouses the principles of continuous process
improvement for users and applies them in maintaining the model. The goal in
undertaking the development of this model was to help organizations improve their
software development process.
The CMM was initially created as a tool that could be used by the Department of
Defense to evaluate and measure the quality of contractors bidding to develop
complex software-based systems for them. The CMM carries with it an evaluation
process that defines the corporate qualification boundaries in the following five
prescribed levels of software process maturity:
1. Initial. The software process is characterized as ad hoc and occasionally even
chaotic.
2. Repeatable. Basic project management processes are established to track
cost, schedule, and functional capabilities. The necessary process discipline is
in place to repeat earlier successes on projects with similar applications.
3. Defined. The software process for both management and engineering
activities is documented, standardized, and integrated into a corporatewide
software process. All projects use a documented and approved version of the
organization’s process for developing and maintaining software. This level
includes all characteristics defined for level 2.
4. Managed. Detailed measures of the software process and product quality are
collected. Both the software process and products are quantitatively
understood and controlled using detailed measures. This level includes all
characteristics defined for level 3.
5. Optimizing. Continuous process improvement is enabled by quantitative
feedback from the process and from testing innovative ideas and technologies.
This level includes all characteristics defined for level 4.
These levels provide guidance for measuring the degree of quality of processes used
within an organization for software development efforts. The entire premise of SEI
CMM is directed under the principles of total quality management and continuous
process improvement. As such, the model itself and related evaluation activities are
under constant improvement status at the SEI.
Organizations demonstrate that they meet the goals for each level by producing
evidence of work processes performed within key process areas (KPAs) of the
individual projects and within the company. KPAs can be thought of as functional
areas or offices, such as quality assurance, configuration management, or the office
of system design and development. It is within the KPAs that specific guidelines, in
the form of questions, are provided. When questions within each KPA at a given level
can be answered in the positive, the answers validated with some form of physical
output, and the personnel who produced the output can explain how the output is
produced, how it is used, and what happens to it after it is produced, ratings are
awarded.
The formal CMM evaluation process is conducted by auditors from outside the
organization who want confirmed levels of capability in order to conduct business
with the federal government. The audits are performed by people trained in
assessing software development efforts that are based on the criteria spelled out in
the model. Specific pieces of information, referred to as evidence, are validated for
all functional areas of a project. The assessment training is provided by the SEI,
which is associated with Carnegie Mellon University, in Pittsburgh, Pennsylvania.
Representatives fro m the SEI are actively promoting the concepts and methods
presented in the CMM, both nationally and internationally. What was originally
developed as a tool for the Department of Defense is now being used by other
federal government agencies and is beginning to reach into the commercial
marketplace as well.
PARALLEL POINTS BETWEEN ISO 9000 AND SEI CMM
The strongest areas in which a parallel effort may be drawn between ISO 9000-3,
9001, and CMM appear to be: peer reviews, software product engineering, software
configuration management, software quality assurance, and requirements
management. Practices that are more strongly addressed by the ISO quality
standards than by CMM include: process change management, technology change
management, defect prevention, quantitative process management, integrated
software management, organization process definition, and organization process
focus. It is important to note that both the ISO standards and the CMM address
many additional areas wherein the relationship may be moderate to weak.
An international organization dedicated to the quality assessment process has
undertaken an initiative called Software Process Improvement and Capability
dEtermination (SPICE). This international organization is committed to the
development of a standard for software process assessment or through the
implementation of some other means in order to support companies doing business
across borders.
DOES ISO 9000 CERTIFICATION OR SEI CMM RATING
CONSTITUTE QUALITY MANAGEMENT?
An important element to keep in mind is that both ISO standards and SEI CMM are
tools that an organization can use to achieve a true quality program. If either of the
methodologies has been institutionalized and developed until a formal certification or
rating has been achieved, the organization has been recognized by external sources
as having a viable quality management program at the time of the audit. However,
as previously stated, maintaining a quality management program goes beyond the
institutionalization of processes. Because a good quality management program will
have defined and repeatable processes, what must really be defined are the quality
goals as they pertain to a company’s specific business.
Since this is not a static environment, a company must continue to perform
necessary analyses to determine what characteristics and attributes of quality are
right for its products and customers as the business environment continues to evolve.
This means that the quality management program must sustain activity in all areas
affected by ongoing and new development projects. Personnel at all levels should be
encouraged to contribute and participate in the analysis and evaluation. The culture
of the company must continue to be aware of the investment and the benefits of the
program to them. The quality management program should undergo continuous
improvement by updating the goals as well as the processes used to achieve the
goals as the environment changes. In this way, a company is assured of having a
successful, ongoing quality management program.
The inherent danger in relying on tools to accomplish this, rather than culture and
commitment, lies in producing the procedures and the paperwork that allow ISO
certification or CMM rating to be granted without providing the education and
integrating the processes into the corporate culture to ensure real quality. It is
possible that neither of these methodologies is the right tool for a particular
organization to follow to develop a quality management program, especially if
industry standard practices and the customer base does not require the formal
certification of the organization’s quality by an external agency. | 
Disclaimer
