Featured Post

Healthcare Customer Satisfaction: More Talk AND More Action

Healthcare Customer Satisfaction: More Talk AND More Action Customer satisfaction (Voice of the customer) is a recurrent th...

Tuesday, October 1, 2019

Do student satisfaction surveys REALLY measure satisfaction?

The central message in Quality is monitor your customers and continually progress towards improvement.

I wish I could say that was an absolute truth in the which is in the arena of education and teaching, but in my observation, the best I can say is not so much.  I don’t think this is entirely a consequence of disinterest in wanting to set a quality agenda in teaching; it is also a lack of investigation, follow through and innovation.
It should also be clear that if your goal is only about customer satisfaction, then it is fair to say that you are stuck in the 1980s.  What the more appropriate goal is improvement which goes beyond satisfaction, or goes more under the current title of “customer delight”.  

I have raised this before.  Customer delight follows the model described by Kano, which talks about providing a service beyond the normal expectation, beyond satisfaction and creates a feeling of exceptional appreciation.  

To be fair, it is difficult to measure for satisfaction, if the sole tool are traditional student satisfaction surveys.  Surveys are at best marginal to credibly measuring satisfaction.  I created “Noble’s Rules” as a way to increase their potential.  But  even with the “Rules” surveys have nothing to offer for looking at  “customer delight”.   
When educators discovered surveys, either on-paper or on-line, they seemed like the perfect tool.  You create a bunch of questions, students answer them and you then can count the response.  If one teacher gets 7 As and 3 Bs and another gets 5As and 5Bs, the first must be better.  The problem is that most students soon learn there is little in the surveys for them.  This is a little game from which they quickly suffer survey fatigue and and boredom.  They all too quickly become robotic in their answers and far too predictable to be reliable.  Most students rank teachers on a 5-point scale with As or Bs most of the time, mainly because it is fast and easy.  Put down something else and then you get these other questions.  Too much work and not worth the effort.  

There are others who love to be outliers who feel empowered and throw in a few Cs and Ds.   Today we call this  the “twittering” of student surveys; the power of outliers when protected by anonymity.

If we really want to gather information, we need more objective, independent measures to determine if we are making progress.  

So let me tell you of a supplemental measurement tool that seems to be working for us to see if our audience likes what we are doing.  
In our certificate course for medical laboratory quality management, we do a lot of year-over-year update and revision,  Since few people (if any) take our course year over year, few are aware of how much the course changes over time. 
But when they finish the course they communicate with their organization manager or  employer and tell them about what they learned.  If they had a terrible experience, the message would likely be that the course was a waste of time. 
But what we are seeing is that organizations send us more and more candidates year over year.  This is happening in multiple provinces in Canada, and in a number of foreign countries.  Over the past 5 years our repeat business not only continues to occur, but many participants start registering earlier.  For example, this year our registrants started to come in early in September, with many coming from organizations who have sent people to us before.   

We see this as benefiting from shared information.  Participant A has a positive experience, and informs their colleague who then registers early to become Participant B, or perhaps, they inform their employer who this is inclined to send more workers to increase the pool of Quality trained persons.  Ultimately it is a Quadruple win: Participant A, Participant B, Employer and us.

So while we track individual opinions through satisfaction surveys, we can also track structural opinions by looking at where they come from, and were they likely coming by referral.
So here is my message:

·   (A) If you feel compelled to use student satisfaction surveys, be very skeptical of the information you gather

·        (B) If you feel you have no choice, at least improve your surveys with Noble’s Rules. 

·       (C) Better yet, find another indicator that is less subjective that satisfaction surveys, more independent and more measurable and more focused on structural issues of referrals.  (See Noble’s Rule (8)).


Tuesday, August 27, 2019

Trust but Verify in Healthcare

I am NOT the first person who has thought about this.  I am NOT the first person who has experienced this.  This is NOT my first time to struggle with the dilemma of tests performed inside the medical system but outside the laboratory.  But it has reached a level that needs attention.

So about 7 months ago, my family doc thought that that it would be a good idea for me at my age to have a screening occult blood test done to rule out a silent bowel cancer.  Unfortunately the test did not rule out bowel cancer, but understand the test is VERY specific, which means it is detects blood, but VERY non-specific in that there are a bazillion reasons that one can have a positive blood test, and NOT have bowel cancer.  In fact the highest probability was that I do not and did not have any good reason to  suspect I have cancer.  But in the modern era, if there is even the tiniest suspicion one has to do a follow-up test, a bowel scoping (colonoscopy).

So a scoping is arranged and 6 months (!!) later, I  get an appointment and after going through all the preparations I have my test, which I am told indicates no cancer.  No surprise, but YAY, none the less. 

But for a guy who spends his time thinking about Quality and performance, this was a really unsatisfying experience because, what if the test result was wrong.  As we all know, it can happen.  

One thing is always certain… errors occur.  Maybe the doc doing the test is incompetent, or maybe he was distracted or maybe is under a lot of stress, too much work and not enough time.  Maybe the scope was defective.  Maybe the biopsy that was taken was taken incorrectly from the wrong place.  The reality is that our system makes the assumption that the scope doc is competent, but we have NOTHING but faith to back that up. 

In the laboratory we perform Quality Control regularly to ensure that our equipment works, our reagents work.  We can, to the extent possible, ensure that samples were collected properly and transported properly.  We can maintain quality indicators to monitor for incorrect reports, and we can most certainly monitor for complaints.   Further we have External Quality Assessment (Proficiency Testing) to monitor that our staff are proficient/competent.  And we have regular Quality Indicators that can measure and record all sorts of parameters to give us early detection of problems arising.  It is part of our norm, part of our routine.   It is called verification and validation of the process.

In clinical care we don’t have that or do that.   We have Trust me… Trust ME!  I didn’t see any evidence of cancer… Trust me.   I didn’t see anything that looked like a polyp… Trust me.  Or, I saw a polyp and took a biopsy correctly… Trust me.   Of course I know what I am looking at with my scope.  I am a good doctor… I was trained at the best place by the best people.  I always am careful… I never make mistakes… TRUST ME.

Let me argue, the days of trust are getting pretty thin, and I can see a time in the not-do-distant future that “Trust, but verify”  does not only apply to nuclear physics and bomb site laboratories.  There are many many organizations where Quality processes are the norm (maybe not in physician or lawyer offices or teachers and certainly not politicians) where regular audit, and re-certification is the routine.  Think about restaurants, and food chain, and medical equipment suppliers, and medical laboratories where audits to ISO international standards are routine in most countries around the world.  (Even politicians have a form of validation and verification  every 4 years!).  In the United States, if a medical laboratory does poorly on its proficiency testing, the facility can lose its ability to bill for federal funds.  In Canada, a laboratory can be closed until the quality requirements are addressed.

In the olden days, one could make the argument “Yes we agree, but there is no way one can make an assessment, and measurement, a determination.  That is no longer the case.  In the medical laboratory we can make samples that closely simulate typical laboratory sample proficiency testing is can be a real-to-life experience.  In the medical training arena there are dolls that look like and act like real patients.  Some of these can be and are used in examination and training settings.  We no longer rely solely on actors playing patients. 

What we need to do as individuals, as interested parties, as professionals start the movement.  Verification by demand. 

It won’t stop all errors, but it will provide documentation of meeting requirements and help professionals become aware of potential weaknesses, and provide our customers, our patients and their families with the confidence that we take Quality Monitoring and Quality Improvement seriously.


Thursday, July 18, 2019

Meeting the Needs

It is remarkable that 2019 marks 40 years since Philip Crosby wrote his seminal book Quality is Free and his declaration of the Four Absolutes of Quality, the first of which was “Quality is defined as conformance to requirements” by which he meant that if the customer has certain requirements and expectations, such as defect free and durability, precision, and date of delivery, and all are met, then the product is meeting the customer’s requirements and can be fairly called a Quality product.   Meeting Requirements or Meeting the Needs of the Customer is not a cornerstone of modern quality, it is the cornerstone. 

As we think about meeting needs, it becomes clear that needs go a long way from purely “technical” considerations.  Yes, precision, and timeliness are still critical needs for medical laboratory testing, but so are clinical relevance, and interpretation, and context, and clear language and of course getting the right information to the right person and not infringing of confidentiality.  Satisfaction is essential, but is only the first step.  In our arena, our customers extend to cover health care personnel working with the “patient” as well as their family and friends and contacts and caregivers, and public health and the institutions of health.  It is not that meeting the needs has become more complicated; rather it is that we have become more aware of what people want, need and expect.  

So in November 2019 we are going to bring the discussion to the larger table.  On November 24-26 a conference will be held here in Vancouver BC Canada to look broadly at the theme of Meeting the Needs.

The conference will be an ideal forum for people interested in Quality in the Medical Laboratory field including technologists, Quality specialists, students, trainees, administrators, and pathologists from BC, across Canada and even beyond.

As we gear up you will see increasing information at https://polqm.med.ubc.ca/

This would be a good time to start thinking about your own poster presentations.
More to come!!!