Quality of care in walk-in clinics, family practices and emergency departments: the Ontario Walk-In Clinic Study
Reviewed by: Andrew Affleck, MD;* Grant Innes, MD†
*Thunder Bay Regional Hospital, Thunder Bay, Ont.
†St. Paul's Hospital, Vancouver, BC
How do patient satisfaction and quality of care indicators compare between walk-in clinics, family practices and emergency departments?
Hutchison B, Østbye T, Barnsley J, Stewart M, Mathews M, Campbell MK, et al. Patient satisfaction and quality of care in walk-in clinics, family practices and emergency departments: the Ontario Walk-In Clinic Study. CMAJ 2003; 168(8):977-83.
To determine differences in patient satisfaction and quality of care for common acute care conditions in walk-in clinics, family practices and emergency departments (EDs).
Over the past decade, several factors, including physician remuneration models, public demands for convenient care, changing physician practice patterns, and public expectations, have led to a proliferation of walk-in clinics. This has been met with skepticism on the part of family and emergency physicians, who express concerns about quality, continuity and cost-effectiveness of walk-in clinic care. To date, however, there is little evidence to support this skepticism.
Between Feb. 15 and Dec. 21, 1998, patients who presented to a participating walk-in clinic, family practice or ED with 1 of 8 acute care tracer conditions -- pharyngitis, gastroenteritis, serous otitis media, acute otitis media, upper respiratory infection, acute bronchitis, urinary tract infection and low back pain -- were eligible for study. Family practices were defined as practices with 2 or more physicians where over 50% of visits involved "regular patients." Walk-in clinics were defined as clinics where less than 50% of visits were by "regular patients." In the family practice setting, patients who were not "regular patients" were excluded, and in the walk-in clinic setting, patients who were "regular patients" were excluded.
This study was a prospective cohort study conducted in large urban Canadian centres. Twenty randomly selected walk-in clinics, 35 family practices (32 randomly selected, 3 targeted) and 13 EDs in 11 geographic regions were invited to participate in the study. Of these, 12 walk-in clinics (60%), 17 family practices (49%, 14 random and 3 targeted) and all 13 EDs agreed to enter the study. An expert panel consisting of 2 family physicians, 1 family/walk-in clinic physician, 1 family/emergency physician and 1 emergency/walk-in pediatrician selected the tracer conditions and developed the quality-of-care criteria.
In each setting, data collectors screened patients while they were waiting to be seen. Eligible patients had one of the tracer conditions, were proficient in English, and met the defined criteria for a "family practice" or "walk-in clinic" patient.
Satisfaction was measured in 3 key domains: Perception of patient-centred communication, Satisfaction with physician's attitude and Satisfaction with delay in waiting room. Univariate comparisons were performed using analysis of variance (ANOVA), and a multivariate analysis was performed using multiple linear regression. The College of Family Physicians of Canada quality-of-care criteria were modified by consensus and used to measure quality of care. Intra- and inter-rater reliability were assessed, and mean values reported. Confounding factors were analyzed and used to derive adjusted quality scores for each setting.
All 600 recruited patients were included in the quality-of-care analysis, and 433 provided satisfaction data (174 walk-in clinic, 122 family practice, 137 ED). Distribution of the tracer conditions varied markedly in the 3 settings, as shown here in Table 1. Quality-of-care scores were 73.1% for EDs, 69.9% for walk-in clinics and 64.1% for family practices, with ED and walk-in clinic scores being significantly higher than family practice scores. After adjustment, ED patients were significantly less satisfied than other patients on all 3 satisfaction scales. The authors speculate that lower satisfaction scores may be because ED physicians lack family medicine training, or because they prefer fast-paced episodic care with less focus on interpersonal relationships. The authors acknowledge that family practice offices offer the comfort of familiarity, shorter waiting times and a non-institutional atmosphere.
Family practices and walk-in clinics scored higher than EDs in patient satisfaction dimensions, while walk-in clinics and EDs scored higher in quality-of-care.
Emergency physicians who view walk-in clinics as entrepreneurial enterprises designed to harvest health care's low-hanging fruit will appreciate this study, which attempts to measure some of the components of care quality in an objective manner. They will also like the fact that EDs achieved higher quality ratings than walk-in clinics or family practices. They may be disappointed by low ED satisfaction scores, and they should keep in mind that this study has several limitations.
It is important to note that the conditions studied are non-urgent problems, which are not the ED's focus or raison d'être. Had the authors chosen to study severe asthma, myocardial infarction or multiple trauma, EDs would probably have performed even better. Nevertheless, the relatively high ED quality scores suggest that EDs can deliver excellent care across a broad spectrum of illness.
While EDs had favourable quality scores, the measures and the methods used to derive these were problematic. First, the indicators chosen are process measures (performance of appropriate clinical actions and avoidance of inappropriate actions) that may not reflect any meaningful improvement in actual patient outcomes. Second, they are based on chart review and may merely indicate that ED physicians charted better rather than practised better. Third, the authors provided no indication that outcome assessment was blinded to study group, and if the assessors knew whether the patient was an ED, family practice or walk-in clinic patient at the time they ascertained quality outcomes, the study conclusions are compromised. Finally, the differences in quality scores reported may not be clinically important.
When evaluating the results of a study, it is important to ask whether the study participants (sites) are representative of the population they are drawn from. In this study, the ED sites are representative, because all of the EDs in the region participated in the study; however, it is not clear that participating walk-in clinic and family practice sites are representative, since only 60% of invited walk-in clinic and 49% of eligible family practice sites participated in the study, and it is likely that those that chose to subject themselves to scrutiny are different from those that declined (i.e., volunteer bias). The authors also indicate that some of the family practices were recruited on the basis of "personal relationships" with the research team -- a decidedly non-random sampling technique -- and that one of the participating family practices contributed no patients to the study. These facts show that family practice and walk-in clinic site selection was non-random and that patient enrollment within sites was also non-random. It is unclear how the (potentially selective) sampling methods might have influenced the study outcomes.
A significant concern is that in the family practice setting, eligibility was limited to "regular" patients, while in the walk-in clinic setting, it was limited to "nonregular" patients. It is likely that patients who are satisfied with their family physicians will become "regular" patients, while those who are dissatisfied might seek care elsewhere. Selective enrollment of "regular" patients who have specifically chosen to return to see their family physician introduces a bias in favour of family practice that has nothing to do with the actual quality of the patient care interaction studied. Other studies suggest that patient satisfaction is associated with health status and the ability to stay with primary care providers and keep doctor's appointments. Emergency patients as a group have poorer health status and perform more poorly on these behaviours. Consequently, the "satisfaction" components of this study may be biased against the walk-in clinic and ED settings.
In the Interpretation section, the authors note that they found a relationship between satisfaction and patient characteristics (i.e., sex, age, family size and socioeconomic status). Despite this, they did not present data to show that the study groups were similar at baseline on important predictor variables. In fact, it is likely that baseline predictors were very different at baseline, and this makes comparing outcomes questionable. The absence of a table comparing baseline characteristics is a glaring omission in this study.
The study data suggest that ED patients were less satisfied and that the differences in satisfaction were statistically significant. Unfortunately, the magnitude of the differences are difficult to interpret, and no data are presented to suggest that the differences are clinically important. For example, in "Perception of patient-centred communication," EDs were rated at 3.2 (p = 0.03) compared to 3.3 for walk-in clinics and 3.5 for family practices (see Table 3 of study). Regrettably, we have no idea whether these differences are meaningful, and they would appear not to be. Also noted in Table 3, "Satisfaction with physician's attitude" differences (82 to 87 to 90, presumably based on a visual analog scale, although this is not specified in the article) are of a similar magnitude -- and these differences are less than the clinically important VAS differences for pain, which are in the range of 13-15.
The most significant satisfaction difference (p < 0.001) was associated with waiting time. It is conceivable that the halo effect from "waiting dissatisfaction" could have influenced the other satisfaction parameters. Because waiting times were a key independent variable in this study, it is important to know what the waiting times in the EDs were. Unfortunately, these were not presented, and without them it is difficult to generalize the study findings beyond the study setting. For example, if waiting times in the participating EDs were 3-4 hours, the findings are not relevant to an ED where similar patients wait 35 to 40 minutes.
The bottom line is that, in this study, EDs scored highest in care quality and lowest in satisfaction. Unfortunately, the conditions studied do not reflect the ED's specific strengths, and the quality measures used may be clinically unimportant. In addition, the study is limited by potential sampling biases, failure to demonstrate that patients were comparable at baseline, failure to differentiate statistically significant from clinically important outcome differences, and difficulty in generalizing results to other settings. Nevertheless, EDs should ponder these data and determine whether they need to take steps to improve their patient service and patient satisfaction.
Dr. Andrew Affleck, 197 Chercover Dr., Thunder Bay ON P7G 1A6