A truly successful Quality Moment.

Every two years we put on a POLQM Laboratory Quality Conference here in Vancouver.  The overarching theme is always the same - what’s new in laboratory quality for British Columbia and Canada and beyond.  We focus on topics like updates on key ISO standards (like ISO15189 and ISO22870) and on understanding risk for medical laboratories (ISO 22367) and medical devices.  All important topics for laboratorians to know.

This year we had an additional theme on “Meeting the Needs” with particular reference to Crosby and his definition of Quality as Meeting Requirements and the Measurement of Quality as the Price of Nonconformance, which we modified to the Costs and Consequences of Poor Quality, underscoring that all too often it is the customer who pays the consequence of our poor quality.

We had a lot of information on today issues like Quality Control of Cannabis and Impact of Gender Diversity on laboratory services, and the role of patients, and caregivers in the education of health professionals and learning the skills of Leadership.  Plus much more.

From my experience putting on conferences is NOT a money generating activity. If we break even we consider that a success.  If we lose a little or gain a little that is our target.  (The university is pretty clear that we are not-for-profit, but we are certainly not-for-loss!!).  If we take in a lot of money, that usually means that I charged too much. 

What I enjoy from putting on conferences is the satisfaction of knowing that we contribute to quality education and quality improvement in a most immediate sort of way.  People get together, they talk, they question, they challenge, they make presentations and verbalize what they are interested in, and then they go back to home with new thoughts, new ideas, with a new enthusiasm to create a better care environment for healthcare professionals and patients and their families and the community.    It is a lot like our putting on our virtual classroom courses, but even more immediate and more intense.  It is the ultimate quality and improvement moment.

Those who attended shot forward in their appreciation of how much laboratory quality is advancing.  Those that did not, did not. 

First let me emphasize that with our activities we focus on those present; and spend little time thinking about those who did not.  But this time I feel compelled to comment a little to the negative. 

From three jurisdictions we heard about spending freezes in healthcare, with particular reference to cuts in staff education.  Lots of funding for leaders and administrators but none for staff education.  Different funding pockets we were told; very unfortunate we were told; financial crisis management we were told.  All of it BS.   It gives us pause when we think about the current status of patient care when institutions put such a low priority on continuing staff education and quality improvement.

The most significant saving grace we experienced  were staff members who traveled from afar to get to the conference, using their own funding and using their own vacation time to attend.  These are the people who will save healthcare in the future.  

We fatigue of the tiresome expressions of privilege and entitlement and arrogance in folks who should know better.  Laboratory improvement is NOT derived from the high price help.  It comes from the people who do the work of making laboratories better.

For people interested in seeing what we discussed at our meeting, visit https://POLQM.med.ubc.ca/2019-polqm-quality-conference/2019-conference-presentations/    after December 6, 2019.

When Quality Conferences end, Real Quality Improvement ENDS

Do student satisfaction surveys REALLY measure satisfaction?

The central message in Quality is monitor your customers and continually progress towards improvement.

I wish I could say that was an absolute truth in the which is in the arena of education and teaching, but in my observation, the best I can say is not so much.  I don’t think this is entirely a consequence of disinterest in wanting to set a quality agenda in teaching; it is also a lack of investigation, follow through and innovation.

It should also be clear that if your goal is only about customer satisfaction, then it is fair to say that you are stuck in the 1980s.  What the more appropriate goal is improvement which goes beyond satisfaction, or goes more under the current title of “customer delight”.  

I have raised this before.  Customer delight follows the model described by Kano, which talks about providing a service beyond the normal expectation, beyond satisfaction and creates a feeling of exceptional appreciation.  

To be fair, it is difficult to measure for satisfaction, if the sole tool are traditional student satisfaction surveys.  Surveys are at best marginal to credibly measuring satisfaction.  I created “Noble’s Rules” as a way to increase their potential.  But  even with the “Rules” surveys have nothing to offer for looking at  “customer delight”.   

When educators discovered surveys, either on-paper or on-line, they seemed like the perfect tool.  You create a bunch of questions, students answer them and you then can count the response.  If one teacher gets 7 As and 3 Bs and another gets 5As and 5Bs, the first must be better.  The problem is that most students soon learn there is little in the surveys for them.  This is a little game from which they quickly suffer survey fatigue and and boredom.  They all too quickly become robotic in their answers and far too predictable to be reliable.  Most students rank teachers on a 5-point scale with As or Bs most of the time, mainly because it is fast and easy.  Put down something else and then you get these other questions.  Too much work and not worth the effort.  

There are others who love to be outliers who feel empowered and throw in a few Cs and Ds.   Today we call this  the “twittering” of student surveys; the power of outliers when protected by anonymity.

If we really want to gather information, we need more objective, independent measures to determine if we are making progress.  

So let me tell you of a supplemental measurement tool that seems to be working for us to see if our audience likes what we are doing.  
In our certificate course for medical laboratory quality management, we do a lot of year-over-year update and revision,  Since few people (if any) take our course year over year, few are aware of how much the course changes over time. 

But when they finish the course they communicate with their organization manager or  employer and tell them about what they learned.  If they had a terrible experience, the message would likely be that the course was a waste of time. 

But what we are seeing is that organizations send us more and more candidates year over year.  This is happening in multiple provinces in Canada, and in a number of foreign countries.  Over the past 5 years our repeat business not only continues to occur, but many participants start registering earlier.  For example, this year our registrants started to come in early in September, with many coming from organizations who have sent people to us before.   

We see this as benefiting from shared information.  Participant A has a positive experience, and informs their colleague who then registers early to become Participant B, or perhaps, they inform their employer who this is inclined to send more workers to increase the pool of Quality trained persons.  Ultimately it is a Quadruple win: Participant A, Participant B, Employer and us.

So while we track individual opinions through satisfaction surveys, we can also track structural opinions by looking at where they come from, and were they likely coming by referral.

So here is my message:

·   (A) If you feel compelled to use student satisfaction surveys, be very skeptical of the information you gather

·        (B) If you feel you have no choice, at least improve your surveys with Noble’s Rules. 

·       (C) Better yet, find another indicator that is less subjective that satisfaction surveys, more independent and more measurable and more focused on structural issues of referrals.  (See Noble’s Rule (8)).

 

Trust but Verify in Healthcare

I am NOT the first person who has thought about this.  I am NOT the first person who has experienced this.  This is NOT my first time to struggle with the dilemma of tests performed inside the medical system but outside the laboratory.  But it has reached a level that needs attention.

So about 7 months ago, my family doc thought that that it would be a good idea for me at my age to have a screening occult blood test done to rule out a silent bowel cancer.  Unfortunately the test did not rule out bowel cancer, but understand the test is VERY specific, which means it is detects blood, but VERY non-specific in that there are a bazillion reasons that one can have a positive blood test, and NOT have bowel cancer.  In fact the highest probability was that I do not and did not have any good reason to  suspect I have cancer.  But in the modern era, if there is even the tiniest suspicion one has to do a follow-up test, a bowel scoping (colonoscopy).

So a scoping is arranged and 6 months (!!) later, I  get an appointment and after going through all the preparations I have my test, which I am told indicates no cancer.  No surprise, but YAY, none the less. 

But for a guy who spends his time thinking about Quality and performance, this was a really unsatisfying experience because, what if the test result was wrong.  As we all know, it can happen.  

One thing is always certain… errors occur.  Maybe the doc doing the test is incompetent, or maybe he was distracted or maybe is under a lot of stress, too much work and not enough time.  Maybe the scope was defective.  Maybe the biopsy that was taken was taken incorrectly from the wrong place.  The reality is that our system makes the assumption that the scope doc is competent, but we have NOTHING but faith to back that up. 

In the laboratory we perform Quality Control regularly to ensure that our equipment works, our reagents work.  We can, to the extent possible, ensure that samples were collected properly and transported properly.  We can maintain quality indicators to monitor for incorrect reports, and we can most certainly monitor for complaints.   Further we have External Quality Assessment (Proficiency Testing) to monitor that our staff are proficient/competent.  And we have regular Quality Indicators that can measure and record all sorts of parameters to give us early detection of problems arising.  It is part of our norm, part of our routine.   It is called verification and validation of the process.

In clinical care we don’t have that or do that.   We have Trust me… Trust ME!  I didn’t see any evidence of cancer… Trust me.   I didn’t see anything that looked like a polyp… Trust me.  Or, I saw a polyp and took a biopsy correctly… Trust me.   Of course I know what I am looking at with my scope.  I am a good doctor… I was trained at the best place by the best people.  I always am careful… I never make mistakes… TRUST ME.

Let me argue, the days of trust are getting pretty thin, and I can see a time in the not-do-distant future that “Trust, but verify”  does not only apply to nuclear physics and bomb site laboratories.  There are many many organizations where Quality processes are the norm (maybe not in physician or lawyer offices or teachers and certainly not politicians) where regular audit, and re-certification is the routine.  Think about restaurants, and food chain, and medical equipment suppliers, and medical laboratories where audits to ISO international standards are routine in most countries around the world.  (Even politicians have a form of validation and verification  every 4 years!).  In the United States, if a medical laboratory does poorly on its proficiency testing, the facility can lose its ability to bill for federal funds.  In Canada, a laboratory can be closed until the quality requirements are addressed.

In the olden days, one could make the argument “Yes we agree, but there is no way one can make an assessment, and measurement, a determination.  That is no longer the case.  In the medical laboratory we can make samples that closely simulate typical laboratory sample proficiency testing is can be a real-to-life experience.  In the medical training arena there are dolls that look like and act like real patients.  Some of these can be and are used in examination and training settings.  We no longer rely solely on actors playing patients. 

What we need to do as individuals, as interested parties, as professionals start the movement.  Verification by demand. 

It won’t stop all errors, but it will provide documentation of meeting requirements and help professionals become aware of potential weaknesses, and provide our customers, our patients and their families with the confidence that we take Quality Monitoring and Quality Improvement seriously.

TRUST ME!!!