HiQuiPs: QI and Patient Reported Measures

Authors: Samuel Vaillancourt, Ahmed Taher

You are about to start a Quality Improvement (QI) project in your ED. You have noted that clinicians are being interrupted frequently by patient questions about the ED process and non patient care related inquiries. You have read previously that emergency physicians were being interrupted an average of 9.7 times per hour and may result in a break in tasks, which may lead to negative patient safety implications.​(1-3)​ You have set up a QI team and created an aim statement. You have also engaged the different stakeholders in the project, and completed a root cause analysis. Based on your results, you have decided to use sequential PDSA cycles with targeted interventions. These include:

1. Utilizing a patient volunteer in front of the physician desk to answer questions

2. A brief educational intervention for clinicians on how to respond to patients appropriately and answer their question

3. Pamphlets to answer common non patient related care inquiries

4. A media solution with a video that explains the ED process

At your next meeting, the group is excited to start implementing, and you decide to collect process data that is relevant to the intervention. You recall the Donabedian model categorizing outcome, process and structure measures, all of which are being used and discussed by the group.​(4) You start to wonder about all the different stakeholders in the room, and ask yourself which of these measures are most relevant to your patients?

Welcome to another HiQuiPs post, where we will be discussing patient reported outcomes!

Patient Outcomes

Outcome measures are inherently more difficult to measure than structure and process measures as they may take a long time to appear and are less directly linked to the intervention. For example, it is easy to link an intervention to the availability of levofloxacin in the ED (structure measure) or time to antibiotic administration (process measure), but pneumonia cure rate (outcome measure) is harder to link to an intervention from observational data as it is influenced by patient, environmental and treatment characteristics. QI projects usually focus on the former two due to pragmatism, yet they are not infrequently labelled as outcome measures as they are outcomes of a QI initiative (even if they may not be outcomes to patients). No matter the difficulties, careful consideration of outcome measurement or the relationship of the measures selected with outcomes valued by patients warrants consideration. Outcome denotes the effects of care on the health status of patients. Improvements in the patient’s knowledge or behaviour are also included broadly as health outcomes.​(5)​ Survival is an early example of an outcome measure, but is not a relevant measurement to much of contemporary medicine, where improvements in symptoms, functioning or quality of life are sought. Patient outcomes data can be categorized by using the nature of the information collected as seen in the figure below. Patient reported outcomes are a subset of patient outcomes and need to be derived directly from our patients. They cannot be obtained from other sources.

Patient Outcomes. (source: adapted from Acquadro et.al.​ (6))

Patient Reported Outcome Measures (PROMs)

To capture patients’ perspectives in a comparable, reliable and valid way, rigorous questionnaires have been developed referred to as patient-reported outcome measures (PROMs). Over the last decade, PROMs have been used as outcome measures in 27% of registered clinical trials. (7) PROM instruments are standardized and validated questionnaires about patient perceptions with regards to their health status, level of impairment, disability, and health-related quality of life.​(8) PROM instruments can be general (examples would be the EQ-5D and the SF-36) or specific to a disease (i.e. the Oxford Hip Score). The importance of these measures is highlighted by national uptake in the United Kingdom as part of the NHS, and the creation of the patient reported outcome information system (PROMIS) in the United States.​(8)(9)​

Patient-Reported Experience Measures (PREMs)

Another important element to consider is the patient experience in the ED. Patient-reported experience measures (PREMs) look at the experience in the process of receiving care, and thus can be seen as a measure of the patient’s perception of the process of receiving care.​(8)​ PREMs are more inline with understanding ED processes and thus can be thought of as process measures as they reflect the subjective experience of the patient during the ED visit. Increasing numbers of countries are using national PREM measurements as in Europe, Canada [The Canadian Patient Experience Survey-Inpatient Care (CPES-IC)] and the United States (CAHPS Survey).​(10)(11)​ A recent review outlines the available patient experience instruments, albeit for inpatients.​(12)

Table 1 below illustrates some examples of PROMs and PREMs.

Emergency Department PROMs and PREMs

There is currently no validated PROM specific to the ED setting, however, one has recently been developed and validated for patients seen and discharged in the ED, called PROM-ED, and should be published and available in the coming months.​(13) On the other hand, there are several validated ED PREM tools.​(14)(15) The process of generating these instruments is a rigorous one which involves patient consultation, psychometric testing, and validation. If there are no validated questionnaires available for the disease or setting you are in, it may be better to utilize elements from current validated instruments or if creating an instrument for your specific questions, it would be important to include patient consultations, question testing and validation.​(8)

The Patient Perspective

There are other ways of eliciting the patient perspective with their healthcare experience. The figure below presents previous methods that have been used to generate this information.

Patient Experience measurements. (source: adapted from Edwards et al.​ (12​))

Eliciting the patient perspective in project design and implementation can be done through stakeholder consultation. This can also be done in the form of inviting a patient representative to sit on your QI team, utilizing patient focus groups, or soliciting patient surveys to help inform design. When ED patients were asked about what they value after they leave the hospital and as a result of their care in the ED, they named four themes: understanding the cause and expected trajectory of their symptoms; reassurance; symptom relief; and having a plan to manage their symptoms, resolve their issues, or pursue further medical care.​(16)​ It is important to include the patient perspective as engaged stakeholders in any ED QI project.

The QI Project

Reflecting back to our QI issue of decreasing clinician interruptions in the ED – you decide to engage patients through your hospital’s patient partners office (or equivalent). You invite a patient who has been treated frequently in the ED to attend several of your group meetings for input. Moreover, to measure the patient experience during your QI interventions, you also decide to utilize several validated questions from the Emergency Department Patient Experience of Care Survey for your project which include:​ (15)

• During your emergency room visit, did someone let you know about how long you would wait before you got care for the first time?

• During this emergency room visit, how often did nurses treat you with courtesy and respect?

• During this emergency room visit, did doctors spend enough time with you?

• During this emergency room visit, how often did doctors explain things in a way you could understand?

You also decide on including some open-ended questions to gather qualitative information about their experiences.

Conclusion

Overall, patients do not come to the ED for the experience, but there is much we can do to reduce the suffering and anxiety associated with it. We have been measuring this through PREMs. Moreover, when patients present with concerns associated with chest pain, a fever, or bruised ankle, the outcomes of such visits often are in the understanding, the plans, reassurance and symptom relief that patients may benefit from in the days after their time in the ED. These are the focus of new work and may soon be measurable through ED relevant PROMs.

That is it for this month’s post. Join us next time as we delve into the concept of variation and as it relates to QI as a prelude to reporting QI findings. See you next time!

Senior editor: Shawn Mondoux

Copyedited by: Paula Sneath

References

1. Chisholm C, Dornfeld A, Nelson D, Cordell W. Work interrupted: a comparison of workplace interruptions in emergency departments and primary care offices. Ann Emerg Med. 2001;38(2):146-151. https://www.ncbi.nlm.nih.gov/pubmed/11468609.

2. Westbrook J, Coiera E, Dunsmuir W, et al. The impact of interruptions on clinical task completion. Qual Saf Health Care. 2010;19(4):284-289. https://www.ncbi.nlm.nih.gov/pubmed/20463369.

3. Chisholm C, Collison E, Nelson D, Cordell W. Emergency department workplace interruptions: are emergency physicians “interrupt-driven” and “multitasking”? Acad Emerg Med. 2000;7(11):1239-1243. https://www.ncbi.nlm.nih.gov/pubmed/11073472.

4. Donabedian A. The methods and findings of quality assessment and monitoring: an illustrated analysis. Journal for Healthcare Quality. 1985;7(3):15.

5. Donabedian A. The quality of care. How can it be assessed? JAMA. 1988;260(12):1743-1748. https://www.ncbi.nlm.nih.gov/pubmed/3045356.

6. Acquadro C, Berzon R, Dubois D, et al. Incorporating the patient’s perspective into drug development and communication: an ad hoc task force report of the Patient-Reported Outcomes (PRO) Harmonization Group meeting at the Food and Drug Administration, February 16, 2001. Value Health. 2003;6(5):522-531. https://www.ncbi.nlm.nih.gov/pubmed/14627058.

7. Vodicka E, Kim K, Devine E, Gnanasakthy A, Scoggins J, Patrick D. Inclusion of patient-reported outcome measures in registered clinical trials: Evidence from ClinicalTrials.gov (2007-2013). Contemp Clin Trials. 2015;43:1-9. https://www.ncbi.nlm.nih.gov/pubmed/25896116.

8. Kingsley C, Patel S. Patient-reported outcome measures and patient-reported experience measures. BJA Education. April 2017:137-144. doi:10.1093/bjaed/mkw060

9. Cella D, Riley W, Stone A, et al. The Patient-Reported Outcomes Measurement Information System (PROMIS) developed and tested its first wave of adult self-reported health outcome item banks: 2005-2008. J Clin Epidemiol. 2010;63(11):1179-1194. https://www.ncbi.nlm.nih.gov/pubmed/20685078.

10. Male L, Noble A, Atkinson J, Marson T. Measuring patient experience: a systematic review to evaluate psychometric properties of patient reported experience measures (PREMs) for emergency care service provision. Int J Qual Health Care. 2017;29(3):314-326. https://www.ncbi.nlm.nih.gov/pubmed/28339923.

11. Klein D, Elliott M, Haviland A, et al. Understanding nonresponse to the 2007 Medicare CAHPS survey. Gerontologist. 2011;51(6):843-855. https://www.ncbi.nlm.nih.gov/pubmed/21700769.

12. Edwards KJ, Walker K, Duff J. Instruments to measure the inpatient hospital experience: A literature review. Patient Experience Journal. November 2015:77-85. doi:10.35680/2372-0247.1088

13. Vaillancourt S, Cullen J, Linton D, et al. LO08: PROM-ED: the development and testing of a patient-reported outcome measure for use with emergency department patients who are discharged home. CJEM. May 2018:S9-S9. doi:10.1017/cem.2018.70

14. Bos N, Sizmur S, Graham C, van S. The accident and emergency department questionnaire: a measure for patients’ experiences in the accident and emergency department–reply. BMJ Qual Saf. 2013;22(9):787-788. https://www.ncbi.nlm.nih.gov/pubmed/23431218.

15. Weinick R, Becker K, Parast L, et al. Emergency Department Patient Experience of Care Survey: Development and Field Test. Rand Health Q. 2014;4(3):5. https://www.ncbi.nlm.nih.gov/pubmed/28560075.

16. Vaillancourt S, Seaton M, Schull M, et al. Patients’ Perspectives on Outcomes of Care After Discharge From the Emergency Department: A Qualitative Study. Ann Emerg Med. 2017;70(5):648-658.e2. https://www.ncbi.nlm.nih.gov/pubmed/28712607.