 |
|
Assessing Quality of Care via HEDIS 3.0
Is There a Better Way?
Arch G. Mainous III, PhD;
Jeffery Talbert, PhD
Arch Fam Med. 1998;7:410-413.
ABSTRACT
Patients, employers, and third-party payers are all calling for improved measures of health care quality. This has led to the development of "report cards," assessments that are many times applied not just to health plans but also to providers. One attempt at creating a standardized set of quality and effectiveness measures is the Health Plan Employer Data and Information Set (HEDIS). The HEDIS measures are based primarily on analyses of administrative data sets. Problems with HEDIS measures, including the probability that plans will use different data collection methods and a lack of risk adjustment, may result in incorrect conclusions about the quality of care delivered by various providers. An alternative method of standardized surveys is proposed that will overcome many of the limitations of the current HEDIS measures, provide outcome rather than process data, and provide data for developing interventions to improve quality.
INTRODUCTION
As managed care becomes more important as a health care financing mechanism and cost limits are set for health care, quality of care is becoming a major concern to payers, patients, and physicians providing care for covered populations. Unfortunately, this task of searching for and assessing quality performance is one of the most demanding tasks for everyone involved in the changing health care system.
Leading the efforts to evaluate health plan performance has been the development of health plan "report cards."1 The purpose of report cards is to collect and present appropriate information to consumers and purchasers of health care. One of the most widely used standardized approaches for assessing the quality or effectiveness of a health plan's performance is the Health Plan Employer Data and Information Set (HEDIS). These measures make the health plans accountable for quality in services, such as clinical preventive services.2
In addition to evaluating health plans generally, measures of quality are also applied directly to physician performance. Practice profiling is an important and increasingly common way of evaluating physician behavior.3-4 A variety of measures in HEDIS (eg, screening for diabetic retinopathy, use of mammography, and Papanicolaou smears) can easily be applied to physician performance. However, the limitations of quality measures in assessing health plan performance are still present and may be accentuated if applied to physician performance.5
The purpose of this discussion is to examine several HEDIS quality and effectiveness measures in relation to their ease of use and informational value and propose that, for some concepts, an alternative method may be a better choice. We will first review the development of quality performance measures and discuss their use to providers, consumers, and health plans. Next, we will identify several examples in which HEDIS specifications using administrative data are particularly problematic. Finally, we will offer an alternative to HEDIS measures for these areas that build on current survey methodology, thereby supplying a standardized alternative to standard HEDIS measures with several distinct advantages.
BACKGROUND ON HEDIS QUALITY AND EFFECTIVENESS MEASURES
Originally titled the HMO Employer Data and Information Set, HEDIS developed from an effort of the HMO Group to establish measures to evaluate the value of health plans.6 First published in 1991, HEDIS 1.0 covered data on quality, access, satisfaction, utilization, and financial information. In late 1993 the original HEDIS measures were revised with the assistance of the National Committee for Quality Assurance and published as HEDIS 2.0. These measures continued to focus on the core areas of quality, access, satisfaction, membership, utilization, and finance. The National Committee for Quality Assurance reports that in 1996 more than 330 health plans were reporting HEDIS measures and more than 50% of large corporations use HEDIS measures when making health plan purchasing decisions. Plans must implement the computation of these measures at their own cost and responsibility. The current version of HEDIS, version 3.0, was released in 1997. Version 3.0 integrates measures for Medicaid (previously a separate set of measures) with those for commercial plans, thus offering a single set of measures for both public and private plans. Version 3.0 continues to refine measures and adds an additional focus on what HEDIS terms "health outcomes." These outcome measures tend to be process measures of the delivery of appropriate preventive care and care for selected disease states (eg, rates of immunization of infants and adolescents, flu shots for at-risk adults, cervical cancer screening, treatment of otitis media, asthma treatment for children and adults, smoking cessation efforts, and the assessment of functional status for elderly patients).
PROBLEMS WITH HEDIS
Most private health plans and public systems like Medicaid rely heavily on claims or administrative data to evaluate their systems. The core of most HEDIS measures is derived from administrative information. Therefore, the use of current administrative data specifications are one of the benefits of the HEDIS measures, because they are based on currently collected and readily available information. However, while administrative data provide a rich source of information on a variety of measures focusing on process, they have proved inadequate for addressing health outcomes. However, although HEDIS includes these new outcome-oriented measures, they offer little insight into the plan's ability to treat serious illnesses.7 Moreover, these measures will not allow for an examination of why performance is the way it is. In other words, these measures provide no insight to either explain why quality would be low and neither do they provide information for interventions to try and improve quality.
In addition, the move to more outcome-based measures has brought new problems for the use of HEDIS specifications. Several of the stated operational definitions of quality and effectiveness measures are difficult to implement or require multiple types of data. Thus, it is unfeasible to compute the measures using standard administrative data. Consequently, health plans' desire to compute some HEDIS measures may lead them to produce measures that are similar in intent but do not follow the HEDIS definitions. If health plans use different specifications for either of these measures, comparisons will be made of measures computed according to different operational definitions, a situation that HEDIS was directly designed to avoid. Evidence indicates that health plans vary widely in how they originate, store, and collect data.8 In fact, the National Committee on Quality Assurance, in their audit of HEDIS 2.0, found that plans frequently departed from the specifications for computing the measures.9
As HEDIS moves beyond administrative performance measures to more quality performance measures, it is less dependent on administrative data and more likely to use supplemental data sources. Many of the measures suggest using chart audits as a data supplement for some measures (eg, prenatal care in the first trimester). Although claims are the suggested measure in assessing cervical cancer screening, an alternative measure specified by HEDIS is to use documentation in the medical record that includes an author-identified note that contains the date the test was performed and the result or finding. Unfortunately, doing chart audits to evaluate a health plan or even a physician's performance is an extremely labor-intensive and time-consuming task and relies on sophisticated sampling techniques to provide useful data.
An alternative method to claims analysis is to survey members through standard items. A measure that acknowledges that claims may not capture the full range of member services is flu shots for older adults, which uses a survey of a random sample of health plan members. This method is used to assess patient satisfaction as well as an assessment of the functional health status of elderly patients by using the MOS 36-Item Short Form Health Status Survey.10 This method may hold promise as a cost-effective alternative to several HEDIS quality and effectiveness measures and may provide more reliable and consistent findings.
EXAMPLES OF PROBLEMATIC HEDIS MEASURES
Childhood Immunizations
Childhood immunizations are one of the most cost-effective health interventions. Unfortunately, low immunization rates can lead to outbreaks of vaccine-preventable disease.11 Thus, the goal of this effectiveness measure is to determine if children are up-to-date with their immunizations. The HEDIS 3.0 system requires that this measure be based on data for individuals continuously eligible for 1 year, thus necessitating 2 years of administrative data. This data requirement allows enough time to determine if the children received the full course of immunizations, particularly because several (eg, hepatitis B vaccine) require multiple immunizations within the time frame to be up to date. Two key issues complicate the HEDIS recommended specification for assessing childhood immunizations.
First, many children are referred to health departments for childhood immunizations,12 especially in rural areas.13-16 It may be possible for some health plans to merge data from their state's health department database to try and provide a more complete picture of the immunization history. However, immunization delivery rates within a health plan based only on paid claims will not provide a reasonable understanding of the provision or receipt of childhood immunizations.
Second, problems arise due to the multiyear requirement for using administrative data. As the volume of claims increases from single to multiple years, the size of the data files and computing time required increases substantially. This requirement may have the effect of eliminating some health plans for computing this measure or computing measures that are not comparable.
Survey data on childhood immunization may provide a more usable alternative for most health plans. While surveys of members for childhood immunization must rely on proxy reports (ie, parents or caretakers) of immunizations, which may suffer from recall error and self-selection bias,17 proxy respondents for child health care issues have been shown to be adequate sources of data on these issues.18-19 This memory recall error can be minimized by purposely limiting the length of time for recall of events in order by surveying only parents of children younger than 3 years old.20
Low Birth Weight
The delivery of low-birth-weight infants has important implications for utilization of resources, as well as for morbidity and mortality.21 Low-birth-weight infants are 40 times more likely to die than normal-birth-weight infants.22 Moreover, there are several modifiable risk factors of low birth weight (eg, maternal smoking) that could be affected by physician intervention.23-24 Consequently, HEDIS assesses the proportion of low-birth-weight deliveries out of the population of live births as an outcome-based indicator of quality. Because HEDIS 3.0 computations are based not on claims but on hospital discharge abstracts and/or birth certificate data, many health plans may not have that data readily available in computerized form. Although the International Classification of Diseases, Ninth Revision (ICD-9) codes used in the discharge abstract contain a fifth digit that can be used to specify the weight of an infant, the data in this field is not uniformly reliable. Noting this, HEDIS suggests that when focusing on this data using the discharge abstract, health plans need to verify the accuracy of the data.
A possible alternative strategy to an analysis of discharge abstracts or birth certificates that a health plan may undertake would be to use claims data and diagnosis codes to develop a proxy measure for low-birth-weight infants. Because low birth weight and preterm birth are seen as covarying to a great degree, one could compute the number of infants who are diagnosed with low birth weight by using the diagnosis "Other Preterm Infants," ICD-9 code 765.1. Some manuals suggest that this code usually implies a birth weight of 1000 to 2499 g and/or a gestation of 28 to 37 completed weeks.25 Similarly, the diagnosis of "Extreme Immaturity," ICD-9 code 765.0, is suggested to imply a birth weight of less than 1000 g and/or a gestation of less than 28 completed weeks. There is no specific ICD-9 code for either very low birth weight, normally defined as less than 1500 g, or low birth weight, normally defined as less than 2500 g.
This particular strategy could be misleading, primarily because preterm delivery and low birth weight are not synonymous conditions. Although they tend to covary they are not collinear. In fact, only 54% of low-birth-weight infants are preterm.26 Thus, using a preterm delivery diagnosis to represent low birth weight incorporates a substantial misclassification bias into the measure.
Survey data offers an effective alternative to the HEDIS specification. As with recall issues of childhood immunizations from proxies, reports of low birth weight have a certain degree of expected error. However, surveys like the National Health Interview Survey have been used successfully to gather this information.23-24
ADVANTAGES TO SURVEYS AS AN ALTERNATIVE MEASURE FOR ASSESSING QUALITY AND EFFECTIVENESS
There are several advantages to a method of assessing quality and effectiveness through surveys rather than through an analysis of administrative data. First, an advantage of member surveys for delivery of care measures is the additional information that can be obtained for planning and decision making. Although purchasers can use the HEDIS report cards to compare health plans, these summary data can do little beyond pointing to problem areas. A survey can ask why these are problem areas. For example, a measure of breast cancer screening is useful to tell if women are receiving appropriate mammograms, but additional information needs to be gathered to determine why they are not getting services. Reasons could range from a high cost for the service to limited access to lack of physician recommendations. This knowledge is particularly important if a health plan or physician is to make changes to improve services.
Second, analysis of claims data may be focusing on a soon to be outdated model of health plan data. As Medicaid moves to managed care, for instance, providers may no longer be required to submit claims like they did in a fee-for-service environment. If providers no longer submit claims then health plans will not be able to compute measures that are defined by claims analysis. However, health plans will have access to their members for surveys.
Third, a disadvantage of the current HEDIS effectiveness measures is that the population being assessed is not adjusted for severity of illness or risk. Current report cards do not account for differences in plan populations. It has been shown that an inadequate strategy for risk adjustment can lead to the wrong conclusions about quality of care.27 By not accounting for differences in populations while comparing report cards of quality performance, a misleading picture may be presented.
Surveys of the patient population could assess not just the current HEDIS measures but a dimension like functional health status, which could be used not only as an outcome but also to stratify the population.10, 28-30 The patient's reported information on how healthy or sick he or she is can be invaluable as an outcome.
Fourth, a final problem with effectiveness measures based solely on administrative data that could be attenuated with survey methodology is the time lag necessary for analyses based on claims. For example, Medicaid programs are using HEDIS to evaluate their performance but because providers have 1 year from the date of service to submit the claims, an analysis of a complete set of 1 year's worth of claims is at least 1 year old for the most recent claim.31 Survey data can be more timely in planning and evaluation, a situation that is particularly important if feedback is to be given to physicians on their performance and have an effect on helping consumers make knowledgeable choices.32
Finally, although great strides are being made in the creation of electronic medical records,33 an improved information system still may not overcome the limitations of analyses based on administrative data. As plans move to integrated measurement management information system, they will still not answer some of the questions that surveys can. However, the future applications of the electronic medical record suggests that neither surveys nor administrative data should be used exclusively, but rather used together in a coordinated effort.
The demand for data on quality performance in health care must be balanced by evaluations that assure valid conclusions. All methods have advantages and disadvantages and inherent limitations. Surveys are not a panacea to problems in HEDIS and in fact may cost more as an evaluation tool than analysis of administrative data. Moreover, survey data is not necessarily more accurate than claims data for every measure. However, for some measures, like the examples highlighted above, they may be a viable alternative. Unless supplemented with information from additional data like medical records and enrollee surveys, administrative data sets are unlikely to provide a complete picture of the care received by a health plan's enrollees.6 Surveys will overcome many of the problems that HEDIS 3.0 has for some outcome measures and will allow health plans to better assess outcomes, control for health status, and focus on possible areas for intervention. Health plans and physicians whose performance is being assessed in this standardized fashion can be more confident in the results contained in their report card.
AUTHOR INFORMATION
Accepted for publication December 24, 1997.
Corresponding author: Arch G. Mainous III, PhD, Department of Family Medicine, Medical University of South Carolina, Charleston, SC 29425 (e-mail: mainouag{at}musc.edu).
| Editor's Note:Some of my experiences with health maintenance organizations' (HMO) attempts to measure quality have really been quite funny. On its first try, one HMO said we were not doing Papanicolaou smears at a high enough rate; on further inspection, several of those for whom we had not completed "indicated" smears were men. In another situation, an HMO informed our group that our pay rate would be reduced because we spent far too much money on prescription drugs—our numbers were unbelievable compared with other family physicians. "Unbelievable" was right. When we finally got them to give us a breakdown, our number one category of drugs was chemotherapeutic drugs; somehow all prescriptions in the tertiary care hospital oncology office were being attributed to us! Drs Mainous and Talbert remind us that even potentially objective measures and the use of computer databases can be fraught with problems. The HMOs have improved their abilities in quality measurement over the years, but further improvement is needed for the health of the plans and for those of us family physicians dependent on their ability to do so.—Marjorie A. Bowman, MD, MPA
|
|
From the Department of Family Medicine, Medical University of South Carolina, Charleston (Dr Mainous) and the Martin School for Public Policy, University of Kentucky, Lexington (Dr Talbert).
REFERENCES
1. Cronin CA, Kunkle CM. Coalition activities addressing public accountability for quality of health care delivery. In: Markson LE, Nash DB, eds. Accountability and Quality in Health Care: The New Responsibility. Terrace, Ill: Joint Commission on Accreditation Healthcare Organizations; 1995:84-99.
2. Cooper JK. Accountability for clinical preventive services. Mil Med. 1995;160:297-299.
WEB OF SCIENCE
| PUBMED
3. Braham RL, Ruchlin HS. Physician practice profiles: a case study of the use of audit and feedback in an ambulatory care group practice. Health Care Manage Rev. 1987;12:11-16.
PUBMED
4. Kerr EA, Mittman BS, Hays RD, Siu AL, Leake B, Brook RH. Managed care and capitation in California: how do physicians at financial risk control their own utilization? Ann Intern Med. 1995;123:500-504.
FREE FULL TEXT
5. Spoeri RK, Ullman R. Measuring and reporting managed care performance: lessons learned and new initiatives. Ann Intern Med. 1997;127:726-732.
FREE FULL TEXT
6. Corrigan J, Rogers L. Comparative performance measurement for health plans. In: Kazandjian VA, ed. The Epidemiology of Quality. Gaithersburg, Md: Aspen; 1995:84-106.
7. Iglehart JK. The National Committee for Quality Assurance. N Engl J Med. 1996;335:995-999.
FULL TEXT
|
WEB OF SCIENCE
| PUBMED
8. Morrissey J. HEDIS needs uniform provider data. Modern Healthcare. 1994;24:78-79.
9. National Committee for Quality Assurance. The Report Card Pilot Project Technical Report. Washington, DC: National Committee for Quality Assurance; 1995.
10. Ware JE Jr, Sherbourne CD. The MOS 36-Item Short Form Health Survey (SF-36). Med Care. 1992;30:473-483.
WEB OF SCIENCE
| PUBMED
11. National Vaccine Advisory Committee. The measles epidemic: the problems, barriers, and recommendations. JAMA. 1991;266:1547-1552.
FREE FULL TEXT
12. Zimmerman RK, Medsger AR, Ricci EM, Raymund M, Mieczkowski TA, Grufferman S. Impact of free vaccine and insurance status on physician referral of children to public vaccine clinics. JAMA. 1997;278:996-1000.
FREE FULL TEXT
13. Hueston WJ, Meade RL, Mainous III AG. Childhood immunization practices of primary care physicians. Arch Fam Med. 1992;1:225-228.
FREE FULL TEXT
14. Hueston WJ, Mainous III AG, Palmer C. Delays in childhood immunizations in public and private settings. Arch Pediatr Adolesc Med. 1994;148:470-473.
FREE FULL TEXT
15. Hueston WJ, Mainous III AG, Farrell IB. Childhood immunization availability in primary care practices: effects of programs providing free immunizations to physicians. Arch Fam Med. 1994;3:605-609.
FREE FULL TEXT
16. Mainous III AG, Hueston WJ. Medicaid free distribution programs and the availability of childhood immunizations in rural practices. Fam Med. 1995;27:166-169.
PUBMED
17. Sudman S, Bradburn NM. Effects of time and memory factors on response in surveys. J Am Stat Assoc. 1973;68:805-815.
FULL TEXT
|
WEB OF SCIENCE
18. Miller MA, Sutter RW, Strebel PM, Hadler SC. Cost-effectiveness of incorporating inactivated poliovirus vaccine into the routine childhood immunization schedule. JAMA. 1996;276:967-971.
FREE FULL TEXT
19. Smith KR, McElwee NE. Recalling pediatric poison events: situational and temporal determinants of accuracy. In: Fowler FJ Jr, ed. Health Survey Research Methods. Washington, DC: National Center for Health Services Research and Health Care Technology Assessment; 1989.
20. Sudman S, Bradburn NM, Schwarz N. Thinking About Answers: The Application of Cognitive Processes to Survey Methodology. San Francisco, Calif: Jossey-Bass; 1996:227-243.
21. Overpeck MS, Hoffman HJ, Prager K. The lowest birth-weight infants and the US infant mortality rate: NCHS 1983 linked birth/infant death data. Am J Public Health. 1992;82:441-444.
WEB OF SCIENCE
| PUBMED
22. McLaughlin F, Altemeier W, Christensen M, et al. Randomized trial of comprehensive care for low-income women: effect on infant birth weight. Pediatrics. 1992;89:128-132.
FREE FULL TEXT
23. Mainous III AG, Hueston WJ. The effect of smoking cessation during pregnancy on preterm delivery and low birthweight. J Fam Pract. 1994;38:262-266.
WEB OF SCIENCE
| PUBMED
24. Hueston WJ, Mainous III AG, Farrell JB. A cost-benefit analysis of smoking cessation programs during the first trimester of pregnancy for the prevention of low birthweight. J Fam Pract. 1994;39:353-357.
WEB OF SCIENCE
| PUBMED
25. St. Anthony's Color-Coded ICD-9-CM Book for Physician Payment. Reston, Va: St Anthony Publishing Inc; 1995.
26. National Center for Health Statistics. Vital Statistics of the United States 1987. Vol 1. Hyattsville, Md: US Dept of Health and Human Services; 1990.
27. Localio AR, Hamory BH, Sharp TJ, Weaver SL, Ten Have TR, Landis JR. Comparing hospital mortality in adult patients with pneumonia: a case study of statistical methods in a managed care program. Ann Intern Med. 1995;122:125-132.
FREE FULL TEXT
28. McHorney CA, Ware JE Jr, Rogers W, Raczek AE, Lu JFR. The validity and relative precision of MOS short- and long-form health status scales and Dartmouth COOP charts. Med Care. 1992;30:MS253-MS265.
29. Nelson EC, Wasson JH, Johnson DJ, Hays RD. Dartmouth COOP functional health assessment charts: brief measures for clinical practice. In: Spilker B, ed. Quality of Life and Pharmacoeconomics in Clinical Trials. Philadelphia, Pa: Lippincott-Raven Publishers; 1996:161-168.
30. Hornbrook MC, Goodman MJ. Assessing relative health plan risk with the RAND-36 Health Survey. Inquiry. 1995;32:56-74.
WEB OF SCIENCE
| PUBMED
31. Mainous III AG, Talbert J, Walker MK, Hagen MD. Baseline Evaluation of the Kentucky Medicaid Program for the 1115 Waiver. Frankfort, Ky: Cabinet for Human Resources, Commonwealth of Kentucky; 1997.
32. Palmer RH. Quality health care. JAMA. 1996;275:1851-1852.
FREE FULL TEXT
33. Ornstein SM. Electronic medical records in family practice: the time is now. J Fam Pract. 1997;44:45-48.
WEB OF SCIENCE
| PUBMED
RELATED ARTICLE
Measuring the Health of Seniors
James K. Cooper
Arch Fam Med. 1998;7(5):415-416.
FULL TEXT
THIS ARTICLE HAS BEEN CITED BY OTHER ARTICLES
|
Are Claims Data Accurate Enough to Identify Patients for Performance Measures or Quality Improvement? The Case of Diabetes, Heart Disease, and Depression
Solberg et al.
American Journal of Medical Quality 2006;21:238-245.
ABSTRACT
Interactive Voice Response Reminder Effects on Preventive Service Utilization
Crawford et al.
American Journal of Medical Quality 2005;20:329-336.
ABSTRACT
Is Risk-Adjustor Selection More Important Than Statistical Approach for Provider Profiling? Asthma as an Example
Huang et al.
Med Decis Making 2005;25:20-34.
ABSTRACT
Can We Be Certain of PRO Accuracy and Accountability? Inaccurate "Report Cards" Generated by Medicare Claims Data as a Marker for Laboratory Testing in Diabetic Patients
Gruber and Rudnitsky
American Journal of Medical Quality 2002;17:171-174.
ABSTRACT
Using Artificial Neural Networks and the Gutenberg-Richter Power Law to "Rightsize" a Behavioral Health Care System
Davis and Lowell
American Journal of Medical Quality 1999;14:216-228.
ABSTRACT
Measuring the Health of Seniors
Cooper
Arch Fam Med 1998;7:415-416.
FULL TEXT
|
