Quality Measures - Validation Resources

OBJECTIVE: To assess the validity of obstetric complications, including the Joint Commission on Accreditation of Healthcare Organizations (JCAHO) Core Measure on perineal lacerations, in the California Patient Discharge Data Set.
METHODS: We randomly sampled 1,611 deliveries from 52 of the 267 hospitals that performed more than 678 eligible deliveries in California in 1992-1993. We compared hospital-reported complications against our recoding of the same records.
RESULTS: Third- and fourth-degree perineal lacerations were reported accurately, with estimated sensitivities exceeding 90% and positive predictive values exceeding 65% (weighted to account for the stratified sampling design) or 85% (unweighted). Based on in-depth review of discrepant cases, we estimate the actual positive predictive value at over 90%. Most coding discrepancies were between no injury and first degree, or between first and second degree. Most postpartum complications, including urinary tract and wound infections, endometritis, anesthesia complications, and postpartum hemorrhage were reported with less than 70% sensitivity, but at least 80% positive predictive value. Composite measures from HealthGrades and Solucient, which include these complication codes, also suffer from high false-negative rates.
CONCLUSION: Third- and fourth-degree perineal lacerations are accurately reported on hospital discharge abstracts, confirming the validity of related quality indicators sponsored by the Agency for Healthcare Research and Quality and JCAHO. Administrative data seem less useful for monitoring other in-hospital postpartum complications.

OBJECTIVE: The objective of the study was to estimate the validity of obstetric procedures and diagnoses in California patient discharge data.
STUDY DESIGN: We randomly sampled 1611 deliveries from 52 of 267 California hospitals that performed more than 678 eligible deliveries in 1992 to 1993. We compared hospital-reported procedures and diagnoses against our recoding of the same records.
RESULTS: Cesarean, forceps, and vacuum delivery were accurately reported, with sensitivities and positive predictive values exceeding 90%. Episiotomy was underreported (70% sensitivity). Cesarean indications were reported with at least 60% sensitivity, except uterine inertia, herpes, and long labor. Among comorbidities, sensitivity exceeded 60% for chorioamnionitis, diabetes, premature labor, preeclampsia, and intrauterine death. Sensitivity was poor (less than 60%) for anemia, asthma, thyroid disorders, mental disorders, drug abuse, genitourinary infections, obesity, fibroids, excessive fetal growth, hypertension, premature rupture, polyhydramnios, and postdates.
CONCLUSION: The validity of hospital-reported obstetric procedures and diagnoses varies, with moderate to high accuracy for some codes but poor accuracy for others.

Medical professionals and educators recognize that Simulation Based Medical Education (SBME) can contribute considerably to improving medical care by boosting medical professionals' performance and enhancing patient safety. A central characteristic of SBME is its unique approach to making (and learning from) mistakes, which is regarded as a powerful educational experience and as an opportunity for professional improvement. The basic assumption underlying SBME is that increased practice in learning from mistakes and in error management in a simulated environment will reduce occurrences of errors in real life and will provide professionals with the correct attitude and skills to cope competently with those mistakes that could not be prevented. The main message of the present paper is that this assumption, which serves as the driving force of SBME, should also serve as a starting point for critical thinking and questioning regarding the multiple aspects and components of SBME. These questions, in turn, should lead to empirical research that will provide feedback concerning changes that may be necessary in order to attain the goal of improving medical professionals' performance. Based on such research, SBME will be held accountable for its outcomes, i.e. whether its educational techniques indeed result in decreased occurrence of errors or not, and whether the ability to cope with the errors that do occur is significantly improved. The first of three issues that were addressed concerns individuals' experience of performing mistakes. It is suggested that in order to benefit fully from the experience of performing mistakes in a simulated context, medical educators should create a balance between the emotional load associated with the experience and the professional lessons that can be learned. Furthermore, research should focus on the long-term effects of the experience in changing professionals' attitudes and behaviour. The second question concerned the contribution of the different components of the educational experience to creating the desired changes in professionals' performance. Analysis of the teaching and learning involved in each stage of the educational event should serve as the basis for research that aims at identifying the unique contribution and efficiency of each element, and defining the essential core activities of a simulated experience. Finally, the need to define a newly emerging profession-SBME educator-was addressed. The professional qualifications are, clearly, multidisciplinary and should be based on the growing experience of medical educators in training students and professionals. Defining the profession is essential in order to create academic environments in which professionals will be trained to develop and implement new programmes, accompanied by research and assessment.