Quality Education: what's it good for?

We are putting together a Master’s in Laboratory Quality Management which we expect will be available both in an on-line option and an on-site option.  The on-line will primarily be for folks who are working and intend to continue to work.  The on-site will be for folks who want to take some time away and spend some time doing primary research in medical laboratory quality.  I will continue to write on the subject as the program becomes ready for prime-time.

In the process of putting the proposal together, we did a survey of the folks who participated in the UBC Certificate Course.  Rather than invite and re-invite folks to respond to the survey we made the decision to take the responders from the first pass.  Both approaches can be equally useful, as long as you take methodology  into consideration as you interpret the results.

So after 5 days we gathered about 25 percent of course attenders.  The distribution of folks from across British Columbia and across Canada, and from outside Canada were essentially the same as the total group.  In addition the responders  equally included participants from the early years, the middle years, and the recent years, and few from the current group.   As we did not put any pressure on this group we can interpret this as a reasonably representative group.

Of interest to us was that about 90% belief there is a market for people with a Master’s degree today, and that the same percentage see that the market will be much stronger over the next 5-10 years.  That is what I believe too.

One finding that interested me, and disappointed me at the same time, was that only about 65% of responders currently are holding positions involved directly  in Quality.  We didn’t ask how many had held quality positions but had changed jobs, in large part because that was not the primary point of the survey  But what was interesting was that folks who had taken the course but were not actively engaged in Quality positions still believed there are opportunities out there, and that the market will continue to get stronger.

Which all got me to thinking; there are all sorts of positions for people interested in and trained in Quality.  The most obvious position is as part of a quality team or a part of laboratory  management.  But there are lots of other places as well.  There are positions with all the quality partners, like the proficiency testing group and the accreditation bodies.  There are positions within public or civil service, especially within the Ministries (or Departments) of Health and the health authorities, where knowledge of laboratory quality is important.  There are positions with service and equipment and reagent suppliers who need to be aware of validation, verification, stability, and measurement uncertainty.  And there are consultant positions, especially as we in Canada move more and more towards official inquiries for laboratory error.  And importantly there are increasing opportunities within education with the schools of technology, as well as the medical schools.

Bottom line appears to be that Quality in health and in particular the medical laboratory is a growth opportunity; today and tomorrow, and from the folks who responded to our survey, for the next 5-10 years plus.

Human Factors and Safety and Quality (?)

In 1986 Briggs Phillips published a small, probably near forgotten, but important study in laboratory safety (see in Laboratory Safety: Principles and Practices.  ASM Press 1986).  The author was able to identify two groups of people: one group of 33 who had experienced at least one laboratory accident in the last 2 years, and another group of 33 who had been accident free during the same 2 years.   The two groups were matched by job classification.    

The two groups were interviewed and surveyed for additional information.   

While not matched for the following characteristics, all including age, years of education, years on the job, weight, height and gender and marriage status were identical.  In addition, wearing glasses, medication use, showed no differences.  Also of interest, accidents outside the laboratory, and driving records were also equal.  

The factors that had significant differences were (A) smoking  (B) the number of accidents and injuries and infections acquired in the 2 years prior to the current 2 years (C) personal attitude towards safety (D) opinions of the safety awareness of their supervisors and co-workers, and (E) importance of training and communication versus personal experience.  In all these, the accident involved were more likely smokers (remember this was 25 years ago), had a long history of accidents,  believed that technique was more important than attitude, but thought that personal experience was more important than training.   

The accident free group were significantly more critical of the supervisors and significantly more critical of their co-workers safety consciousness, and believed that safety attitude and awareness were more important than having techniques and equipment.  

[If you are safety aware you will avoid getting into the high risk behaviour of rushing or allowing distraction.  If you are not safety aware, you are more likely to assume that equipment will allow you to go faster and increase risk.]

The message of the study points to importance of attitude and awareness in the avoidance of having a laboratory accident or injury.  

So why am I bringing up 25 year old studies?  Two reasons.

First, a recent study in ASQ’s Quality Management Journal V18, N:1, 2011 on factors associated with continuous  improvement in patient safety  by E. Naveh et al came to a similar finding of the relative importance of opinions on attitude and priority.

Secondly, and more importantly, what made the first study possible was that records were maintained on who had been involved in a laboratory accident or injury.  In many laboratories today, we have virtually no records of Quality accidents, and where we do have records,  the names of the people are commonly deleted.  So in many (most?) laboratories if we wanted to study the human characteristics associated with increased error, it would be impossible because we have no place to even begin.

Deming wisely called to “Drive out Fear” and I agree with that.  He also said that most error was systemic, and I agree with that too.  But Quality in the medical laboratory is a new field, and it is important that we study why errors occur.  Confidentiality is important, and so if avoidance of a sense of retribution.  But let’s not “throw the baby out with the bathwater”.  Continuous improvement requires having data to study.

That’s why they call it PDSA.

Communicating Quality and the Principles of Adult Education

In recent while, I have been engaged in the active promotion of  medical laboratory quality in a variety of forums, including writing this web-log, providing an on-line course, hosting single-site lunch time education,  hosting the resident seminar series, and organizing and participating in workshops and conferences.  All of them have provided me with the opportunity to consolidate my own philosophy towards Quality.  

There is a lot of differences between all these activities; some have been held in Canada and others are international, some are one-off events while others short-term multiple events, and others yet are long multiple events, some of them are at no-charge to the participants while others are at substantial cost. Some of the participants want to be participating while others they are there because they feel compelled for one reason or another.  It would be easy to say that the only thing they have in common is a common topic, but there is something of far greater commonality; all the participants are adults.

Teaching adults is a special event because adults learn in their own special way.  There is tons written on adult learning.   For a quick summary you can visit Principles of Adult Learning at the University of Hawaii site.  

There are lots of reasons that people attend courses.  For learning, the most successful reasons are when they believe there is something in it for them; it is a topic of interest or there is an opportunity for career advancement.  Sometimes it is less about knowledge, and more about the social aspects; is a way to meet and connect with others, or is better than just being bored.  These folks may not be focused on learning, but they might connect.  Sometimes folks attend solely because they have to; it is a requirement and attendance is being taken.  This never turns out well.

Adults are autonomous and self-directed, and learn on top of their accumulated knowledge and life experiences, which means we learn what we want and how we want and accept the knowledge if it makes sense and is consistent with what we already know.  And we learn if we see the purpose to the information and especially it is meets our own goals.  And we are practical and pragmatic learners; if the information is not organized and timely then we turn-off. 

Adults tend to learn best if motivated, especially if we can link the information to what we do as a matter of competence or promotion, or job enrichment.  (Some say we learn better if there is money on the line.)  

For the educator, this means that the successful ones make the information relevant to what people do, is consistent with what they already know and link the knowledge to competence and enrichment.  Keep the information organized and timely and interesting.  For me, I have been involved as an adult educator for over 30 years.  When I think back to what I was like when I started, I can say with confidence that I am a lot better today than I was back then.  And one of these days I may actually get it right.  

One of the most successful educators I know has a formula.  He has 3 points to make (4 points a maximum).  He makes his points and reinforces them with stories and humor.  When he has delivered his points and told his stories, he stops.  I’m pretty good at the selecting 3-4 points and the telling stories.  I am pretty weak at the humor, and tend talk until I run out of time, rather that knowing when to stop.

Memo to self:  with widely divergent groups, some folks are there because they want to be there; others are not.  Remember the motivation; point of why knowing about Quality is an important   component of competency and enrichment, and can have an impact on promotion.  Be organized and know when to stop talking.  After than, the rest is largely out of your control, so don't worry about it.

m

PS: I am accumulating the post seminar series data now.  Will share the results soon.

PPS:  Big changes at www.POLQMWeekendWorkshop.ca  Lots of NEW information.

Quality Education

Today I started my Quality Management Seminar Series for residents and graduate students in Pathology and Laboratory Medicine.  It is the second time that we have done the series, the first time being in January 2009.  I know that in Canada this is the only Quality Management Seminar Series provided for residents.  I am sure that there are other centers around the world that provide this information, but I do not know where.  Such is the reality of the communication networks in Quality.

A few things of note.  This year I have applied a PDSA to the series by having a pre-course survey which will be compared and analyzed to a post-course survey.  The results of the pre-course survey are in themselves instructive.  The survey included 10 knowledge questions that were pretty basic.   All questions were answered as single right answer with the list of responses being randomly positioned to avoid answering bias (if you don't know the right answer, then answer the 2nd in the list).  The style of questions included "what is the international standard for medical laboratory quality and competence?" or "in which phase of the medical laboratory cycle do most errors occur?". 
Of the 10 questions, the best response was 91% of responders getting the right answer, and the poorest response was 0 getting the right answer.  The mean, median, and mode for correct response was all around 40%.   So as they say, there is room for knowledge improvement.  When we have the post-course results collected and analyzed, I think the information derived will be a useful contribution to the resident education literature. 
I was tempted to attach the quiz to MMLQR, but I learned from the last survey that is unlikely to get many responses.  If anyone is interested, if you contact me I will send you a link to the survey.

Today's presentation included an introduction on why residents in training need to be more aware of Quality issues.  I reinforced the message with a series of news releases from the Canadian press highlighting high-profile error in the medical laboratory.  The list was put together by Robert Michel (of the Dark Report) and shared at the Quality Confab in San Antonio, and I thank Robert for letting me use this as an instructional exercise.   I was able to expand this list with a few more events.  The point of the exercise was to point out that once the public starts shining the light that results in change, it tends to not turn the light off.  And further, shining light does not result in confidence building; indeed it points out that public trust and confidence are fragile.  It is fair to say that for many the era of "I am a doctor. You can trust me"  is over.

The seminar series will continue for 3 more weeks.

While not posted yet, the presentations will be available on www.POLQM.ca

m

PS:  I have not mentioned the POLQM Weekend Workshop for a while. 
We have put together a brilliant group of speakers on a variety of topics highly relevant to education and practice of medical laboratory Qualitology.   Plus we are inviting posters and podium presentations.
Please visit www.POLQMWeekendWorkshop.ca

Predictions in Qualitology - 2011

As the year slowly creeps to a close and the next year gallops forward, I start to wonder what 2011 will look like for medical laboratory quality.

1: Flavor of the decade?
It is unlikely that Quality is going off the agenda in the next while in health care.  Across Canada the media has decided to keep an eye on the laboratories, waiting for the bad thing to happen.  We are unlikely to disappoint. 
This will put increasing pressure on provincial governments and maybe (but unlikely) Ottawa. 
Organizations like the Royal College will have a hard time turning away.

The validity of Crosby will ring true in health: the costs of non-conformance, in NOT doing it right the first time, are too high.  I don't know if we will ever actually achieve zero tolerance for error, but 2011  will be a pressure point.

2: Jobs, Jobs, Jobs
As the world comes out of recession and folks again have money to spend, the jobs will return.  This will be true in both the private and the public sector.  Near the top of the list will be the positions postponed or sacrificed along the way.  People over machines.  Buying bigger and better analyzers is unlikely to be seen as the best way forward.  But institutions will still be cautious and a major priority will be effective use of money and more effective monitoring. 
And that will mean more interest for more quality team positions.
Add to this, increasing political pressures (as mentioned above) and there will be even more Quality positions.  Think of 2011 and 2012 as the equivalent of what SARS did for Infection Control.

3: Knowledge is King.
In Canada, we had our Royal College meeting and the importance of shared knowledge in Quality was seen as priority number ONE.  In the US the audience for quality is growing, as is the number of laboratories seeking supplemental accreditation.  The number of folks coming to our training programs is increasing from around the world.  So the message is clear in healthcare that more folks need more information, and we will see the sharers of the that knowledge in more demand.
Again as mentioned above, the wave of new positions will require a wave of new educational opportunities.
Universities and Colleges will become more actively engaged in Quality.

4: Conferences - maybe
Organizations are increasingly leery of  conferences as good vehicles for continuing education.  Airfares, hotels, meals are very expensive.  So how does this fit with Knowledge is King".

First the number of on-line courses and conferences and confabs will increase using a wider array of communication tools that will promote connectivity.  Video conferencing, collective conversation,creative use of networking software will be an increasing part of the on-line education experience.
Second, when conferences are held, they will attracting an audience will be tougher as folks get more selective.
The conferences that survive will be fewer but better.

Successful conferences will be the ones in the right place, and the right time, with the right information, and the right contacts. For example, Vancouver in June at the POLQM Quality Weekend Workshop (visit www.POLQMWeekendWorkshop.ca )


So I'm looking forward towards an exciting year coming up with lots to keep us all busy.
See you next year!

m

Message to self

Let me start by saying that I do a lot of surveys. To date my account has 82 surveys completed and an additional 3 currently active.  I survey students regularly during courses, and annually we do at least one customer satisfaction survey for CMPT.  I have done surveys for the International Organization for Standardization (ISO) and for International Laboratory Accreditation Cooperation (ILAC).  Over the years I have become adept at creating surveys that address the issues that I want addressed.

Last week's survey was my first experiment of linking a survey to a discussion website like MMLQR.  I would not call it a totally successful experiment. 

When I look at the reported results, first I noted that we are attracting a variety of laboratory Quality professionals from Canada and internationally.  There are some positive trends.  Based on a 6-point Likert scale, this site was ranked either as Excellent or Good by near everyone with regard to variety of topics, relevance, clarity, accessibility, and refreshment.  There were no "poor" or "unacceptable" responses.  The same was true for the Overall assessment.

So this is all good, Yes?
Well it provides documentation that supports impressions based on the progressively increasing readership, and it confirms that the people that I am interested in engaging in conversation aer finding the site.  But based upon the number of tracked page views, it looks like less than 4 percent of people connecting to MMLQR have responded to the survey.

With the information that I can garner, I don't know how many people opened the survey but chose to anwer no questions, but I assume that that is a very small number

So there is a problem, but a generalisable problem.  Almost all the surveys I have done in the past have been to a closed or faily closed population, where I could go back to the group and try again and again.  This is a survey to an open population.  In that regard it is similar to attempting to do a satisfaction survey of people exiting a laboratory patient service centre, or of physicians that use laboratory services.  
In all these situations, one can generate a denominator of how many potential responses there could have been.  The challenge is how does one increase the numerator without generating bias, both positive and negative.

In the laboratory setting, one might try creating focus groups as definable groups, but one has to create incentives to garner participation.  This is potentially expensive and would only be applicable if one risked confidentiality breaches.  One could combine focus groups with electronic surveying, but still one would have to have identifiers to work with.
In this setting, I have no access to identifiers, and no obvious inducements that might entice a response.

So it is back to the drawing board with some questions to be asked, like who do I want to attract to the survey, and how can I entice them to participate, and what is a sufficient cluster, and what kinds of questions will capture the information that I want.  And maybe to affirm why I want to generate the information.

It's kind of like my own PDSA

If I come up with some answers I will try again.
In the meantime if you are in the 4%, many thanks for participating.

In the meantime, I am going to take a few days off and come up with my predictions and resolutions for a happy and Quality 2011.

For those of you who celebrate the day, Merry Christmas.

m

The Science of Qualitology

I like the ASQ’s Quality Management Journal because it publishes articles in a science and experimental structure that I understand and expect to see in a journal.  The article that I was looking at was analyzing factors associated with Quality in hospital settings. (seeR.E Carter, S.C. Lonial, and P.S. Raju.  2010.  Impact of Quality Management on Hospital Performance: An Empirical Investigation.  QMJ.  17(4): 8-24).

The study design was based on a survey sent to hospital executives in 175 organizations in mid-US (Kentucky, Ohio, Tennessee, Minnesota, and Mississippi)). The surveys were sent to Hospital CEOs who were in turn supposed to pass them on to senior folks like the VP administration, Quality manager, Support services manager, Director of nursing.  This was very ambitious.

The conclusions they came to were what I would expect; when it comes to quality size and stress matter.  The more uncertainty in the institution, the larger the institution, the less likely they were to have “measurable” evidence of Quality. 

The “measure” of Quality in this study looked at 5 markers for financial performance, 4 markers of market/service development and 4 markers of quality outcomes.  That, in my opinion was a set unlikely to give a clear picture of hospital quality.

And that brings me to my point. 
What are the objective measures that one can monitor as an indicator for success or failure for introduction of Quality activities in medical laboratories?
Not success in accreditation or proficiency testing scores. They are too readily manipulated  (see   M.A. Noble.  2007.  Does External Evaluation of Laboratories Improve Patient Safety?    Clinical Chemistry and Laboratory Medicine.  Clin Chem Lab Med.  45(6):753-756). 

Not numbers of reported incidents or OFI’s.  They are too open to flexible interpretation.  OFI reports, if anything are like unemployment rates.  A downward movement in rates may mean more people are being employed, or it may mean that fewer people are bothering to look.  And a rise may mean more people are unemployed, or it may mean more people are hopeful and are again more actively looking.  In the same way  a rise in the OFI’s rate may mean more problems are being identified and reported meaning poorer Quality, or it may mean more engagement leading to more reporting meaning better Quality.

How about client or staff satisfaction?  Maybe, but again, very manipulatable and too vague.
And in Canada, financial stability or instability are completely inappropriate since 99 percent (or more) of resources come from the government purse.

So we have a dilemma.  For good studies we need measurable and interpretable and  monitorable outcomes on both a micro- and macro-  basis. We do this on a micro- scale all the time (call that Quality Indicators).  But to move from interesting to convincing and compelling, we will need to define our macro- outcomes as well. 

For Quality to create a lasting imprint in medical laboratories, we are going to have to speak the language of laboratory personnel, pathologists and technologists.  We will need the language of science and experimentation. outcome and conclusion. 

Any and all ideas are most certainly welcome.
m

PS: Absence of strong interpretable measures makes grant funding difficult, maybe impossible.  I have learned this the hard way.