Friday, January 24, 2020

Republishing Noble's Rules for better satisfaction surveys

About a decade ago (!)  I started to think about all the problems I was having trying to gather useful information on satisfaction surveys for my proficiency testing program and from my students.  Most of the results that I got back were either incomplete, or inconsistent.   Trying to look at the data was both an Advil moment and a waste of time.  
So I started experimenting with different approaches to see if I could generate better surveys and get better information.
In 2011 I wrote a block on getting better surveys through applying better rules.
Over the years I have revised them, but through the process I ended up corrupting the blog post.  

So for anyone interested, I am republishing the last version, put together in 2018

In the laboratory business we have always thought it was all about the science and not about the business.
But we were wrong.
ISO as well as WHO and CLSI (and before them, Deming and Crosby) all acknowledge the importance of “Customer satisfaction”.
It is not so much that the customer is always right, but that the customer should always have a voice and should be heard. There is an expectation to have some form of customer input on a regular basis, perhaps as often as once a year.
The reason that the standards development bodies have included this as a requirement and the basis for policy is because it doesn’t matter if you are an academic providing a course, or a laboratory providing documented information, or a manufacturer providing umbrellas, or a proficiency testing provider, or an equipment and reagents supplier, if your customers are not happy, then bad things start to happen.
In the private product or service sector that probably means customers stop coming. And that becomes the business killer.
In the public sector laboratory, the customer may not have a choice of which laboratory they have to use, but that won’t stop complaints, reputation slurs, increased threat of litigation. (Incidentally, this applies to accreditation bodies as well.)
Sooner or later you risk becoming the interest of the public and the media.  

Or even worse, think about the embarrassment and humiliation of a public inquiry.
All of those are major career killers.

So what to do. In the business world, the godsend solution for customer satisfaction has become the on-line survey. It is so easy to create an on-line survey and send it out to all your important customers. So easy, in fact, that it has become too easy. 

Anyone foolish enough to give your email address to a hotel or car-rental or restaurant gets inundated with surveys. We have become a world of survey send-outers and survey send-inners, and most of it is a waste of time.

Most surveys are poorly designed; are way too long, too complex, and far too diffusely focused. If a survey takes more than 2 -3 minutes to complete, you can guarantee that either it will not be completed, or will be completed with junk information. 

Also, you have to remember that responders  always have their own bias one way or another,and probably have interpreted the questions in ways that you never dreamed of. Creating most surveys has become high risk of being counter productive for addressing customer satisfaction. As they say “Fast, easy, slick and wrong”.

If you still feel compelled to resort to surveys, spent some time at setting them up so that you might get some information that you can consider. (We call that PDSA) . 

After years of learning the hard way, I figured out a set of simple rules  that anyone interested in developing a Satisfaction Survey can follow.  I arrogantly coined them as Noble's Rules for Successful Satisfaction Surveys Note:  They don't guarantee success, but not keeping them in mind will pretty much guarantee failure.

(1) Focus them to a single issue.
The more you try to pack into a survey, the worse it gets.  Pick a topic and get out.

(2) Ask the question that needs to be asked, even if you may not like the answer.  
It’s very easy to create surveys that will always give you positive feedback by simply avoiding any potentially controversial or challenging issues, but how can you study or learn what people think if you don’t open up the discussion.

(3) limit the survey to only a few questions , best is to keep it to 5-6 and NEVER more than 10, and make them as uncomplicated as possible . 
Get in, ask a few questions, and get out.  Don't give them a chance to get bored.

(4) make sure that it can always be completed in 3 minutes or less. Boredom is a guarantee for incomplete surveys loaded with random nonsense answers.  It would be better if they didn't send the response in, because the nonsense becomes pollution and the pollution leads to terrible interpretation. 

(5) Pre-test the questions to reduce (you can never avoid) ambiguity. 
Make your questions VERY simple.  Confusing questions get confusing answers.

(6) Avoid requiring an answer. That is the other  guaranteed invitation to bogus information. 
Making people answer questions, makes people angry.  Sometimes you can't avoid them, but keep them to an absolute minimum.

(7) Pick your audience and stick with it.  
General send-outs are a total waste of time.

(8) Where you can, avoid satisfaction surveys. 
More effective solutions for monitoring satisfaction is looking at objective measures.  For example, count how many complaints come in and how many are resolved within a specific time. 
Set up a system to catalogue every complaint, something that most laboratories never do. All those telephone and hall-way gripes are complaints and they need to be included.

You may not think they were important, but the person who mentioned them did.