Artificial Intelligence and The Future of External Quality Assessment -

 

 

I have been away for a while.   I apologize for the delay in writing.

M

I have been watching with some interest all the activity that surrounds Generative AI through using ChatGPT4.  The idea that you can ask a computer a question and it scans the electronic datasphere and gathers a whole bunch of “information” and then compresses it into a number of sentences which can be read in a form of basic interpretable English intrigues me.  It took little time for me to understand why there is so much interest (and fear) about this new tool.

Since it is readily available at a kind-of-affordable cost, I figured I should learn a little more about what it can and cannot do.

Being of a certain age, the first thing I did was buy the book “AI For Dummies”.  For my generation there is a “Dummies” book for almost every topic you can possibly want.  It was not perfect, but then again what could I expect?

The next thing was to jump in with a question: "What is the future of laboratory external quality assessment?"

Almost as soon as I got my fingers of the keyboard, ChatGPT4 started to respond.  It took all of maybe 5 seconds.

What I got back impressed me to the extend that the response was on-topic which means that it was framed around what I understand as  laboratory external quality assessment.

The response was broken down into sections which included:

1.    Integration of artificial intelligence (AI)  to revolutionize the medical laboratories by improving accuracy, speed, and efficiency.   It would do this through analyzing data, and assist through interpreting test results, and help identify potential errors or inconsistencies.

2.    Digitalization and remote testing: The increasing digitalization of healthcare could lead to a shift from traditional paper-based proficiency testing to online platforms.

3.    Remote proficiency testing could allow laboratories to participate in testing programs from anywhere, reducing the need for physical samples and paperwork. This would streamline the testing process, increase accessibility, and facilitate real-time performance monitoring.

4.    Personalized proficiency testing: As healthcare moves towards personalized medicine, proficiency testing may also become more tailored to individual laboratory needs. Laboratories may receive customized samples or challenges that align with their specific testing capabilities or areas of expertise. This approach could help laboratories identify and address any weaknesses or gaps in their performance more effectively.

5.    Continuous quality improvement: Proficiency testing programs are likely to focus more on continuous quality improvement rather than simply measuring a lab's performance. This could involve providing laboratories with detailed feedback and recommendations for improvement based on their performance in proficiency testing. Collaborative learning networks and knowledge sharing platforms may play a role in facilitating continuous quality improvement among laboratories.

6.    Regulatory changes and standardization: Changes in regulatory requirements, such as the implementation of new quality management systems or accreditation standards, may affect the future of proficiency testing. Standardization of proficiency testing methods and metrics across different regions and countries could improve comparability and harmonization of laboratory performance assessment.

Some of this I found interesting, but with apologies to the gurus of artificial intelligence, while the subject was correct, for a lot of the response the tense was wrong.  Most of it dealt with what is the near or distant past, or was just wrong.  

 For example while there is interest in virtual microscopy that can be on-line, for the microbiology and chemistry and blood banking, it is difficult to imagine that test competency can be monitored without physical samples.

Also the notion of personalized proficiency testing has been a fact  for many years.  Laboratories already select the companies, and the sample products they want to receive.  While there may be some refinements, they will likely be minor.

What does sound interesting and maybe even futuristic, the use of EQA to monitor knowledge and performance on continuous improvement and knowledge of regulation changes, is already in place in some EQA programs already.  In our program we call that para-examination EQA. 

So here is what I have learned…  computers have reached a new point where they are able to access the whole datasphere and process large amount of data on a wide range of specific topics here and now.  Their memory systems can be trained to look for specific words and patterns, frame them in a new way and restructure it into something different.   And present that is maybe new and novel, but not ready to take over the world.

Interesting? … Yes.   Helpful?...  In some ways.  Definitive?  ….  Not Yet.

It reminds me of another new thought (????) roaming across the drivelsphere.   "… have a vision of what can be, unburdened by what has been. " 

 

 

 

Teaching Points about EQA.

 

I was going through some of my past teaching files and came across a presentation that I put together for a guest presentation in a class at the local Institute for Technology in 2016. I had already been active in proficiency testing for many years.  Often when I I look at back at my teaching  wish I had done it differently.  This time is different.  I think the comments were actually spot on, and are still both relevant and insightful. 

These were  students in their first year of training to become laboratory technologists.  Some had some university background, but most were direct from high school.  I am not sure where or how they picked up the idea that working in a medical laboratory might be a good job. 

Few had any prior to medical education or laboratories, and I’m pretty sure that none of them had any prior education or training in the concepts of Quality. 

My first task thus was to establish a familiar foundation to which which they could relate.

“Most science laboratory students are familiar with Proficiency Testing, but probably under another name.  Often they are given samples by their teacher, which  they are expected to test, to prove that they have learned a procedure properly and are able to perform tests correctly.  This is usually referred to as receiving and testing “unknowns”.  If “unknowns” are important during training, they are even more important during professional clinical practice.”

Then I tried to make the point the more often than not when mistakes do happen, they are usually not the person’s fault.  They happen.

“Proficiency testing is used for detecting a broad array of potential errors.  Sometimes there are human errors that are made as a result of slips, or distractions, or mistakes as a result of misinterpretations or misunderstandings.  Sometimes, people forget procedures or are unaware of changes because they have been away because of illness or extended vacation or are off regular laboratory rotations.  Other times errors are more systemic, the result of silent problems with equipment or reagents, errors or confusions within the procedure. “

Students need to understand that while making errors can happen, they are always a big deal, and can put lives at risk.

But, physicians and patients don’t care about laboratory problems; they  want, indeed expect and demand the information they receive to be the right answer, accurate and interpretable and relevant and on-time.  Quality assessment  tools, including proficiency testing allow the laboratory and appropriate authorities to be aware of competencies on a regular basis.”

Next I introduced to the students the concept of Quality through Assessment, Prevention and Improvement. 

“Proficiency Testing is an important component of the total Laboratory Quality Assessment  which also includes accreditation, internal audits,  quality indicators and customer satisfaction surveys.  Each Quality Measure gives a view of an aspect of the whole organization.  Proficiency Testing can challenge every aspect of laboratory performance including  every phase of laboratory performance (pre-examination, examination, post-examination) as well as activities that surround testing such as safety, quality control (peri-examination)… every phase of laboratory performance”

I wanted to warn them about the consequences of not taking Proficiency Testing seriously. 

“With regret, some laboratories staff do not understand or appreciate why they are expected to participate in proficiency testing by authorities or accreditation bodies.  Some of them may even try to suck the value out of PT/EQA through deception (sending the sample to a reference laboratory for investigation) or overwork (not processing the sample with their usual procedure or with their usual workers), and worst, failure to Investigate why and how errors occurred.”

 

I summed up with…
“In Summary, Proficiency Testing is an internationally embraced Laboratory  Quality Assessment measurement tool similar in purpose as a grown-up extension of student “unknowns”.   It is a  technique that lets laboratories  demonstrate their quality and competence and at the same time provides the opportunity to discover silent system problems that lead to laboratory error.  Proficiency Testing can and should be a valuable  part of laboratory life for the full extent pf a laboratory based career.”

As I look back now, I’m pretty pleased with the message I put forward, now some 7 years ago.  As I recall the class response was positive.   I wish I had had the opportunity to keep in contact and later survey the students over the intervening years, to see if any part of the message actually stuck.

The renaming of Quality Improvement?

 

I have had the opportunity to be actively involved with the American Society for Quality (ASQ).  I am the current Chair of the Healthcare Quality and Improvement Committee (HQIC).  I truly enjoy being  actively involved  because Healthcare Quality has become a personal passion.  Through HQIC I collaborate with like-minded people with a similar sense of passion. 

Last month the World Conference on Quality and Improvement was held in Philadelphia.  Unfortunately, I missed it because of another commitment.  Next year. 

But wait a second… in the last 3 sentences I used a phrase “Quality and Improvement” twice.  Prior to the last little while, I don’t think I have ever said or written Quality AND Improvement before.  I have talked and taught for years about Quality Improvement and Continual Quality Improvement and Continuous Quality Improvement, but never has the phrase Quality AND Improvement been part of that process.   Joseph Juran wrote volumes about Quality Improvement as did W. Edwards Deming, and Philip Crosby and while all of them were involved in ASQ they never used the interloping AND.

So where did it come from?

In 1947 the American Society for Quality Control (ASQC) was created as a union of some seven small organizations.   Each year they would hold an annual meeting (the ASQC Congress).

In 1997, on the 50^th anniversary of ASQC, the name of the organization was shorted to American Society for Quality (ASQ), I imagine because the organization appreciated that Quality had become more diverse and involved many more aspects than just Quality Control.

Along with the conference name change, the  meeting name similarly changed to Annual ASQ Congress.

Then eight years later in 2004, ASQ joined forces with several Quality organizations for other countries around the world, and they collectively decided there was value in merging all their congresses into a single world event which they called the World Congress on Quality AND Improvement;  and the interloping "AND" first appeared!

I wonder perhaps that was done to accommodate a common English translation for a conference including primary English and non-English speaking  countries.

In any event, jump forward anther 15 years to 2019, the ASQ goes through another modernizing name exercise.  Within the Healthcare Division there is a committee named Healthcare Technical Committee that needed a name revision.  They decided that a good name would  not be Healthcare Quality Improvement Committee (HQIC) but rather Healthcare Quality and Improvement Committee, still with the acronym HQIC.   

I don’t think any other committee in ASQ followed a similar path.   I did a global check of ASQ.org and found only one reference to an ASQ Committee with the phrase Quality and Improvement in its title, the aforementioned HQIC

Five years ago is not that long ago, however none of the people I have asked can remember why they opted for Quality AND Improvement.  Was there an ASQ organizational directive (now lost and forgotten) that recommended that all committees addressing Quality Improvement harmonize  their names with the conference name? 

So in the absence of a human memory, I decided to go the non-human route,  specifically  ChatGPT.  If AI doesn’t know the answer, then no answer exists.

So I asked the following… “What is the difference between ‘Quality Improvement’  and ‘Quality AND Improvement’? and in about 2 seconds, I got the following:

"Quality improvement" is a systematic, formal approach to the analysis of performance and efforts to improve it. In healthcare, for instance, it involves methodologies to improve care, patient outcomes, and other systems processes with an aim to reduce errors.

On the other hand, "quality and improvement" isn't a recognized term in the same sense but can be understood literally. It refers to the two distinct concepts of "quality" (the standard of something as measured against other things of a similar kind) and "improvement" (the act of making something better). This phrase might be used in a general context where one is talking about both the quality of something and the improvement of something, but they may not necessarily be related.

In my opinion this is an amazingly intuitive and nuanced response.  Scary that it comes from a machine.   Even if it is only the computations of a non-thinking machine, I don’t think it can be disregarded.  Somewhere in its algorithm  the machine recognized that Quality and Improvement are two separate processes that can function either together or  independently.  Just because an organization implements a Quality Improvement exercise doesn’t necessarily ensure there will be a measurable or meaningful improvement in the organization's  Quality. 

Seems to me there are two explanations for the interloping "AND"; either the folks back in that meeting in 2019 were truly visionary and  insightful, or it was a typographical error that never got caught. 

Or maybe they were just being smart alecs.

What I find interesting is that the term Quality and Improvement rings true in my ear, and I find the argument of recognizing the two activities as distinct compelling.  Maybe it is time for all references to Quality Improvement to adjust to the new AI standard.

I suspect Joseph and Edwards and Philip would enjoy the discussion.