Do student satisfaction surveys REALLY measure satisfaction?

The central message in Quality is monitor your customers and continually progress towards improvement.

I wish I could say that was an absolute truth in the which is in the arena of education and teaching, but in my observation, the best I can say is not so much.  I don’t think this is entirely a consequence of disinterest in wanting to set a quality agenda in teaching; it is also a lack of investigation, follow through and innovation.

It should also be clear that if your goal is only about customer satisfaction, then it is fair to say that you are stuck in the 1980s.  What the more appropriate goal is improvement which goes beyond satisfaction, or goes more under the current title of “customer delight”.  

I have raised this before.  Customer delight follows the model described by Kano, which talks about providing a service beyond the normal expectation, beyond satisfaction and creates a feeling of exceptional appreciation.  

To be fair, it is difficult to measure for satisfaction, if the sole tool are traditional student satisfaction surveys.  Surveys are at best marginal to credibly measuring satisfaction.  I created “Noble’s Rules” as a way to increase their potential.  But  even with the “Rules” surveys have nothing to offer for looking at  “customer delight”.   

When educators discovered surveys, either on-paper or on-line, they seemed like the perfect tool.  You create a bunch of questions, students answer them and you then can count the response.  If one teacher gets 7 As and 3 Bs and another gets 5As and 5Bs, the first must be better.  The problem is that most students soon learn there is little in the surveys for them.  This is a little game from which they quickly suffer survey fatigue and and boredom.  They all too quickly become robotic in their answers and far too predictable to be reliable.  Most students rank teachers on a 5-point scale with As or Bs most of the time, mainly because it is fast and easy.  Put down something else and then you get these other questions.  Too much work and not worth the effort.  

There are others who love to be outliers who feel empowered and throw in a few Cs and Ds.   Today we call this  the “twittering” of student surveys; the power of outliers when protected by anonymity.

If we really want to gather information, we need more objective, independent measures to determine if we are making progress.  

So let me tell you of a supplemental measurement tool that seems to be working for us to see if our audience likes what we are doing.  
In our certificate course for medical laboratory quality management, we do a lot of year-over-year update and revision,  Since few people (if any) take our course year over year, few are aware of how much the course changes over time. 

But when they finish the course they communicate with their organization manager or  employer and tell them about what they learned.  If they had a terrible experience, the message would likely be that the course was a waste of time. 

But what we are seeing is that organizations send us more and more candidates year over year.  This is happening in multiple provinces in Canada, and in a number of foreign countries.  Over the past 5 years our repeat business not only continues to occur, but many participants start registering earlier.  For example, this year our registrants started to come in early in September, with many coming from organizations who have sent people to us before.   

We see this as benefiting from shared information.  Participant A has a positive experience, and informs their colleague who then registers early to become Participant B, or perhaps, they inform their employer who this is inclined to send more workers to increase the pool of Quality trained persons.  Ultimately it is a Quadruple win: Participant A, Participant B, Employer and us.

So while we track individual opinions through satisfaction surveys, we can also track structural opinions by looking at where they come from, and were they likely coming by referral.

So here is my message:

·   (A) If you feel compelled to use student satisfaction surveys, be very skeptical of the information you gather

·        (B) If you feel you have no choice, at least improve your surveys with Noble’s Rules. 

·       (C) Better yet, find another indicator that is less subjective that satisfaction surveys, more independent and more measurable and more focused on structural issues of referrals.  (See Noble’s Rule (8)).

 

How Happy are your customers and how much do you care (Part 1)

Recently I have had the opportunity to start looking at laboratory Quality in a whole new light.   

Medical laboratories have only recently been introduced to the concept that they are businesses that have customers and that these customers have certain rights and expectations.   Quality oriented laboratories need to take customer satisfaction in mind because unhappy customers can make complaints that at a minimum can disrupt smooth administration, or they can create terrible publicity and public awareness, or the can sue.  In many parts of the world, they can drive business away and the laboratory can starve for work (and revenue).  

Unfortunately this is still all very new in the laboratory arena, and only the barest of minimums of activity are yet in place.  Few laboratories are much beyond the complaints form, and for those that are few are doing anything to really capture clinician or patient sentiment.  But to be fair, progress is slowly being made.

In many businesses, meeting satisfaction needs is only a minimum; more importantly they need to find the way to connect satisfaction to both business and revenue growth (welcome back, spend more, and bring a friend!).  This is being referred to as Service Excellence or “Customer Delight”.  (As an aside the term customer delight has been around more that 30 years, but in most English speaking countries it still sits very uncomfortably on the tongue.)

In the early 1980s Noriaki Kano, an academic in quality management and customer satisfaction wrote a lot about satisfaction.  Although his target audience was neither healthcare or medical laboratories, as I read about his model of satisfaction (referred to appropriately as the Kano Model) I can see how much of it directly is referable to the medical laboratory and quality improvement.

Kano wrote of 4 identifiers or attributes of product and service development in the context of customer satisfaction (A) Basic or Threshold (B) Performance or Linear, (C) Attraction or Delighters(!) and (D) Indifferent.

Basic satisfaction occurs when the customer gets used to having the product when they want it, then then get really annoyed when it is not available.   An example of this might be providing staff in the Emergency Department a simple point of care test to detect an infection, but then saying it will not be available for use on Wednesdays or the weekend.   

Performance of linear satisfaction is said to be positive when the service is performed and negative when the service is withdrawn.  A good example of this would be when an laboratory announces that in order to make the life of elder or very young patients easier, they offer to go to the patient’s home for sample collection rather than making them go the clinic, but then not providing the service when the driver or the collector goes on vacation for three weeks or if they decide they won’t collect the sample because maybe the patient has the flu.

Attraction or Delighter satisfaction occurs when the customer is REALLY pleased when the new service or product is available and enjoys its presence, but is not dissatisfied when it is not.  It is seen as something of special value.   It’s like when you go to the laboratory and get seen by a phlebotomist who is not only efficient and effective, but is also happy and congenial and helps take away the anxiety of visiting the laboratory.  Or perhaps when you receive a laboratory report there is an informative note that helps put the result into better perspective.  What Kano pointed out is that once an attraction/delighter action is first put into place, it is seen as something novel and keen, but overtime, everyone just assumes that it should be the norm, and gradually it shifts from being an attraction/delighter satisfaction attribute and becomes a basic one, that people expect all the time, and get really annoyed when it is no longer there.   

Finally there is what Kano called Indifferent Quality which describes quality efforts that the customer knows nothing about and is unaware if they are present of absent and have no tangible impact on satisfaction.   This with regret accounts for maybe 99% of the things that we do under the banner of Quality Management and Quality Improvement.  (We know, but nobody else is aware).

If (and that is a BIG if) documents like ISO15189 or other standards/guidelines start pushing for more innovation in laboratory customer service and more clients start demanding a higher level of attention and care, these concept are going to become very real and very familiar in your neighbourhood laboratory.

More to come,