IEE Brief No. 9, October 1993
Benchmarking, a relatively new tool developed by manufacturing companies to achieve dramatic and continuous improvement in the way they do things, is available to achieve quality improvements in healthcare and education. Hospitals–the most costly segment of the healthcare system–are, in fact, beginning to use benchmarking to discover how other hospitals and other businesses have developed cost-cutting and time-saving ways of doing things. The purpose of this Brief is to show how a field comparable to education is using benchmarking.
For leaders in healthcare, the most powerful pro-benchmarking force is intangible: the professional commitment of clinicians and healthcare workers to share knowledge that can benefit patients. Similarly, educational leaders will find that their commitment to students, their networks of professional affiliations, and their analytic skills can help them tailor the valuable technique of benchmarking to the needs of schools and school systems. Further, as healthcare organizations have discovered that benchmarking can improve the quality of clinical care as well as administrative processes, schools will discover that benchmarking is a tool that can improve teaching and learning.
In organizations as varied as Xerox, AT&T, Eli Lilly, and Southwest Airlines, managers seeking a surge in quality have used benchmarking to make dramatic improvements in controlling their costs and speeding up and improving their service. They have located other organizations with superior performance in specific areas, learned what the others did to excel, and instituted similar–or better–improvements.
The potential rewards from benchmarking are high because the gaps in performance between the best companies and others can be vast: manufacturing-industry leaders generate new products up to two and a half times faster than the industry average, and at half the cost. The director of quality at Motorola, which has been employing benchmarking for several years, goes even further: "Best-in-class companies," he says, "have error rates 500 to 1,000 times lower than average" (First find your bench, 1991).
Benchmarking is not just a new name for networking or comparative data analysis. It is a sophisticated tool, a multi-faceted procedure, usually conducted by teams, for identifying and adapting best-in-class practices. Benchmarking involves adapting and implementing "the best of the best," wherever that best is. It definitely need not be in one's own industry or type of service. Indeed, often the most useful comparisons come from outside one's own industry or technology; one's own field is so familiar that it is sometimes difficult to see it with a fresh perspective. "The purpose of benchmarking is to expose managers to new ways of doing things in order to spark creativity, not to create efficient copy cats" (Walleck, 1991, p. 4.). In fact, benchmarking does not necessarily provide a best system to copy and adopt. Instead, it reveals and suggests an amalgam of different approaches and ideas to improve a process. A benchmarking partner is never going to have the exact same problem or resources. You look at their system, see what works and why it works, and apply pieces of it to your own situation. The creativity and openness fostered by benchmarking frequently yield ideas and approaches that were not even being done by the benchmarking partner.
Perhaps more than in the industrial world, healthcare organizations and educational institutions can tap into a lode of benchmarking partners, organizations that are similar but not really competitors. Indeed, healthcare systems and alliances, whose members are not competitors, are conducting group benchmarking: reaching consensus on project selection and developing common measures and workable benchmarking strategies. For example, the 18-member Voluntary Hospitals of America (VHA) Tri-State Region are using their organization to share information and ideas and the wisdom of the 860 hospitals in the national system. Members are required to participate in two of VHA's national database projects and in regional data sharing. This foundation of information-sharing led them to develop a comprehensive system for conducting benchmarking studies.
Even competing hospitals are finding group benchmarking useful. For example, in San Diego, 20 civilian and military hospitals formed a consortium, the Southern California Coalition for Improving Healthcare Quality, that meets once a month to conduct educational sessions and share information about quality initiatives at their hospitals.
In the following examples illustrating the variety of ways that groups of hospitals have employed benchmarking, educators will recognize processes and situations that are comparable to the world of education.
Twenty Wisconsin hospitals in the Rural Wisconsin Hospital Cooperative, a cooperative owned by 19 rural hospitals and one urban hospital, are conducting a joint effort to solve administrative and clinical problems. Benchmarking was their first step, and the goal of the benchmarking effort was to improve the quality of the hospitals' administrative and financial services.
The benchmarking team took six steps:
Six hospitals formed a dietary department team; three formed a respiratory department team; three formed a physical therapy department team; and five formed a business office team.
The business office team is attempting to improve the cash flow at its hospitals. The time lag between patient discharge and payment was averaging 56 days. As a result of benchmarking, the team decided to start patient paperwork earlier through pre-admission programs.
A hospital acting on its own, 300-bed Cleveland Memorial Hospital in Shelby, NC, has used benchmarking to improve its administrative processes and has already documented measurable improvements. Before the benchmarking project, the insurance verification clerk and the financial counselor at the hospital each had to work an average of two hours extra on Mondays to catch up on the paperwork (on patient admissions) that had piled up on the weekend, verifying insurance information by phone, one patient at a time. Through benchmarking, they learned that most records could be grouped into batches according to payer, and verified by using computer terminals in the hospital that were linked to computers for the three major payers–Blue Cross/Blue Shield, Medicare, and Medicaid. Since these three constituted roughly 60 percent of their business, large volumes could be dealt with faster, leaving clerks more time to deal with the commercial carriers that had to be dealt with individually by phone. They thus eliminated 200 overtime hours logged by the two clerks each year.
In addition, Cleveland Memorial has made a variety of improvements in its check-in process for preoperative testing patients. For example, they have dedicated a clerk to check in these patients so that they do not have to wait with other inpatients and outpatients; installed a fax machine so that physicians can send test orders from their offices in advance rather than having patients bring them in on the day of admission; and streamlined the system for ordering tests, entering them into the system the day before the patient arrives for testing. As a result of these changes, patients coming to the hospital for preoperative testing spend an average of 23 minutes less at the hospital, and the staff have reclaimed their lunch hour, which they had often sacrificed to the chaos of the admissions process. Another benefit, not measured in minutes or hours, is that hospital staff who have had time for lunch are probably more pleasant toward the patients. Thus, the customer service improves in more than one dimension.
The Catholic Healthcare West (CHW) system, which includes San Diego Mercy and 13 other hospitals in California, Nevada, and Arizona, targeted two critical processes: medication delivery, and the entire system of medical records management, including the processes of moving, filing, preparing, and retrieving records.
Once the group agreed on these two processes, they had to go beyond their traditional ways of measuring things. The first big barrier to overcome was that they were not all measuring processes in the same way. With 14 very different hospitals, in terms of size and structure, the development of a consistent set of metrics also proved a knotty undertaking. For the medical records project, they decided to collect data on, among other things, the number of full-time equivalencies who work on medical records, their functions, how many employees serve across functions, and the time it takes to turn around a record.
The largest group benchmarking effort now underway in the healthcare field is a national effort coordinated by the Healthcare Forum and the American Productivity and Quality Center's International Benchmarking Clearinghouse (IBC). A group of 32 hospitals in the Healthcare Forum's Quality Improvement Network are conducting a functional benchmarking study of the admissions process. They chose admissions because it presented complex problems and was important to customers. This rationale reflects IBC's criteria for selecting a process for benchmarking: The process selected should be of strategic importance, a competitive area, a critical success factor, a problem area, or otherwise significant in terms of quality, cost, or cycle time. Admissions was selected also because it is a process found not only in all hospitals, but in slightly different forms in other industries and fields as well (school registration, for example). The Healthcare Forum/IBC group is examining a hotel, a car rental company, and an airline with superior check-in or registration procedures.
A process improvement team at Crawford-Long Hospital (Emory University, Atlanta) decided to benchmark the hospital's admissions, discharge, and transfer systems. The team chose admissions because the extent of the problems in that area dictated the need for fundamental changes, not just incremental improvements. The benchmarking team wanted improvement over current performance, but didn't want "stopgap measures." They wanted to have the best practice.
They brainstormed, made detailed flow charts of their admissions process, drew cause-and-effect diagrams, and compiled lists of problems. They found that patients were being held in the emergency department too long, and physicians couldn't get their patients admitted. As a result, physicians and staff circumvented the system.
The team collected data on numerous quantitative and qualitative indicators, which formed the basis for a two-page questionnaire.
The team identified 28 hospitals to survey, some chosen simply by word of mouth, by people's experiences with them; others chosen because of their professional affiliation with Crawford-Long Hospital. The analysis of the survey results included a matrix ranking the strong and weak points for each hospital, and rankings that pinpointed the hospitals to benchmark. The analysis included hospital type and demographics. The group identified three best-in-class hospitals in the areas of admissions, discharge, and transfer. The team asked each hospital for permission to interview staff from admissions, information systems, nursing, housekeeping, and marketing.
The Crawford-Long team uncovered many attributes that contributed to superior admissions, discharge, and transfer systems in the benchmark hospitals. Reporting its findings, the benchmarking team made three levels of recommendations: things that can be implemented immediately, those that could be implemented later, and those that they might be able to implement "down the road."
Only nine months elapsed between the inception of the benchmark committee and the implementation of the first process improvements. They had definite target dates for every step of the process.
When 15 members of SunHealth Alliance, a voluntary affiliation of 240 hospitals and other healthcare organizations in the southern US, conducted two pilot benchmarking studies, they discovered that each hospital contributed, and learned, certain superior practices or techniques. The key to that result was choosing a process that is crucial to all participants. They selected emergency services and admissions because they are the first point of contact for many patients and physicians and, as such, are critical to the hospital from the customer's perspective.
The team studying emergency services, consisting of representatives from nine hospitals ranging in size from 250 to 750 beds, examined patient flow. They decided that the time it took a patient to go through the Emergency Services department was the critical metric. The team identified six key steps in the patient's movement through the department:
After measuring the average amount of time it took for patients to proceed from one step to another, the benchmarking team focused on performance differences and analyzed what caused the differences. Some of the key items they found that determined more efficient and effective performance were:
Benchmarking does not have to be limited to the improvement of administrative and support processes. In fact, some believe that the ultimate test of benchmarking's utility in healthcare will be in the arena of clinical processes–the actual delivery of medical care. At Sutter Health, a 12-hospital integrated healthcare system in Sacramento, California, physicians and managers have begun the formidable task of clinical benchmarking. Sutter Memorial Hospital has teamed up with three similar but noncompeting hospitals in Los Angeles, Portland, Oregon, and Seattle to benchmark their procedures for coronary artery bypass grafts and coronary angioplasty. The goal is more cost-effective, demonstrably high-quality care. The group selected cardiac care because the hospitals in the group have a high volume of these patients, and because it is an area of high risk and great variation.
Other hospitals and healthcare groups are planning similar efforts. SunHealth will soon be conducting four or five benchmarking projects focusing on circulatory disorders. Member hospitals have also expressed interest in benchmarking procedures for orthopedic surgery, treatment for cardiovascular accidents and stroke, and pneumonia.
Insurers and health maintenance organizations (HMOs) have begun to join in benchmarking efforts with hospitals. The National Committee for Quality Assurance and 30 major HMOs and insurers are developing a common set of benchmark indicators and standards. Under consideration are breast and cervical cancer screening tests, prenatal care, and frequency of hospitalization for children with asthma, a potential indicator of the quality of primary care.
The Maryland Quality Indicator Project (600 hospital participants in 46 states) is a national effort to track the quality of the delivery of medical care. The participants are trying to find a way for hospitals to measure objectively the quality of the clinical care that they provide. Although many hospitals already collect the kind of data that are requested, participation in the Maryland project allows for benchmarking of an individual hospital's performance with others. "The combined focus on outcomes and process allows the provider to look at both ends of the spectrum–what care is delivered and how it's delivered," says Sandy Metzler, of the American Hospital Association, Chicago (Sabatino, 1992).
Benchmarking is critical for the success of this project. In the absence of normative data, hospitals must rely on the phrase, "we're unique" to explain findings like a high mortality rate. It's hard to embrace any process to improve care without baseline information on what needs to be improved, and benchmarking provides that data. The following is a sampling of the kind of indicators on which the project is building baseline data:
Streamlining production at IBM, boosting the efficiency of warehousing operations at Xerox, cutting overtime in a hospital admissions department, and improving the clinical treatment of patients–these very different success stories have resulted from benchmarking. But just as there is no "one right way" to order supplies, register students, or conduct a classroom session, there is no one right way to benchmark. In fact, each healthcare organization approaches benchmarking a little differently as it applies the technique to administrative, clinical support, and patient care processes. Educators can look at how these healthcare organizations benchmarked, and adapt the process to their own needs in their schools and school systems.
Benchmarking is not a mechanical process; it is "a skill, an attitude, and a practice that ensures the organization always has its sights set on excellence, not merely on improvement" (Walleck, O'Halloran, & Leader, 1991, p.13). Through participation in benchmarking, line managers can be brought face-to-face with superior practices. Instead of being exhorted to do better, they can see for themselves how much better something can be done. If hospitals can learn from airlines, hotels, and other hospitals, schools can learn from businesses and other schools, too.
Benchmarking is not a short cut to quality and success. It is hard work. But it is not extra work; it becomes the way an organization does its work. The continuous focus on excellence transforms an organization's culture and stimulates creativity; this is the feature that gives benchmarking such power.
Portions of this paper have been distilled from the September, 1992 issue of The Quality Letter for Healthcare Leaders, volume 4, number 7. Other works consulted are:
Burda, D. (1990, September 3). Rural cooperative consolidates expertise. Modern Healthcare, p. 37.
First find your bench. (1991, May 11). The Economist, p. 72.
Patrick, M. S. (1992, July/August). Benchmarking: Targeting "best practices." Healthcare Forum Journal, pp. 71–72.
Sabatino, F. (1992, June). Quality review: Why Maryland's clinical indicator project is building steam. Trustee, pp. 16–17.
Walleck, A. S. (1991). A backstage view of world-class performers. The McKinsey Quarterly Reprint Series, pp. 2–5.
Walleck, A. S., O'Halloran, J. D., & Leader, C.A. (1991). Benchmarking world-class performance. The McKinsey Quarterly Reprint Series, pp. 6–27.
Winslow, R. (1993, March 9). Report card on quality and efficiency of HMOs may provide a model for others. The Wall Street Journal, pp. B1, B6.
This publication was developed with support from the Ford Foundation and the National Center for Research in Vocational Education (NCRVE), funded by the Office of Vocational and Adult Education, U.S. Department of Education, authorized by the Carl D. Perkins Vocational Education Act.
Institute on Education and the Economy, Teachers College, Columbia University
525 West 120th Street, Box 174, New York, NY 10027
Phone: (212) 678-3091 | Fax: (212) 678-3699 | firstname.lastname@example.org