Non-response bias and quality of care

We think about surveys a lot here at CCHSR. The GP Patient Survey and the Cancer Patient Experience Survey are two in particular which we do a lot of work with. As well as using data from the surveys to conduct research, we also look at the reasons why doctors, managers, practices, hospitals and others find it hard to engage with the results. Concerns about non-response are a common reason people give for not acting on survey findings. In a paper, published online recently, we take a look to see if these fears are well founded using data from the English Cancer Patient Experience Survey.

This question particularly matters for this specific survey because comparisons made between organisations using this data are high profile, and there is a push to dedicate increasing time and resource improving care on the basis of the results. If non-response biases were truly an issue it is possible that this resource could be better directed elsewhere. That said, the generic question as to whether biases introduced by non-response make comparisons between organisations unfair is a very useful one to answer and applies to a wide range of settings.

We found that hospitals where survey response rates are higher do have more positive experience scores. Part of this is explained by some hospitals having more of the type of patients that are both more likely to respond and to give positive responses. However, it would be wrong to suggest the remaining association between response rate and patient experience scores was driven by non-response. Our modelling suggested that if everyone who was sent a survey replied to that survey, the difference between the best and worse hospitals would be wider, and not narrower, than it is now. High survey response rates may in fact be a marker of efficient hospital administration, and better care, and variation in response rates between hospitals may be an indirect measure of care quality rather than a reflection of bias in survey findings.

So do we need to worry? Well, we did find that non-response may be responsible for an overestimation of the absolute national scores, so I would take those with a pinch of salt. But as for which hospitals do well and which do worse, I think we are on reasonably safe ground.


CL Saunders, MN Elliot, G Lyratzopoulos, GA Abel, Do Differential Response Rates to Patient Surveys Between Organizations Lead to Unfair Performance Comparisons?: Evidence From the English Cancer Patient Experience Survey. Medical Care, 2015

This entry was posted in Uncategorised. Bookmark the permalink. Both comments and trackbacks are currently closed.
  • The Cambridge Centre for Health Services Research (CCHSR) is a thriving collaboration between the University of Cambridge and RAND Europe. We aim to inform health policy and practice by conducting research and evaluation studies of organisation and delivery of healthcare, including safety, effectiveness, efficiency and patient experience.