{"id":2753,"date":"2013-02-10T12:50:10","date_gmt":"2013-02-10T12:50:10","guid":{"rendered":"http:\/\/www.saasoft.com\/blog\/?p=2753"},"modified":"2013-02-10T12:50:10","modified_gmt":"2013-02-10T12:50:10","slug":"the-writing-on-the-wall-part-ii","status":"publish","type":"post","link":"https:\/\/hcse.blog\/?p=2753","title":{"rendered":"The Writing on the Wall &#8211; Part II"},"content":{"rendered":"<p style=\"text-align: left\"><a href=\"http:\/\/www.improvementscience.co.uk\/blog\/wp-content\/uploads\/2013\/02\/Who_Is_To_Blame.gif\"><img loading=\"lazy\" decoding=\"async\" class=\"alignleft size-full wp-image-2765\" src=\"http:\/\/www.improvementscience.co.uk\/blog\/wp-content\/uploads\/2013\/02\/Who_Is_To_Blame.gif\" alt=\"Who_Is_To_Blame\" width=\"156\" height=\"195\" \/><\/a>The retrospectoscope is the favourite instrument of the forensic cynic &#8211; the expert in the <em>after-the-event-and-I-told-you-so rhetoric<\/em>. The\u00a0rabble-rouser for the lynch-mob.<\/p>\n<p style=\"text-align: left\">It feels better to\u00a0retrospectively nail-to-a-cross the person who committed the Cardinal Error of Omission, and leave them there in emotional and financial pain as a visible lesson to everyone else.<\/p>\n<p style=\"text-align: left\">This form of\u00a0public feedback has been used for centuries.<\/p>\n<p style=\"text-align: left\">It is called barbarism, and it has no place in a modern civilised society.<\/p>\n<hr \/>\n<p style=\"text-align: left\">A more\u00a0constructive question to ask is:<\/p>\n<p style=\"text-align: left\">&#8220;<em>Could the evolving Mid-Staffordshire crisis have been detected earlier &#8230; and avoided<\/em>?&#8221;<\/p>\n<p style=\"text-align: left\">And this question exposes a tricky problem: it is much more difficult to predict the future than to explain the past.\u00a0 And if it could have been detected and avoided earlier, then how is that done?\u00a0 And if the how-is-known then is everyone else in the NHS using\u00a0this know-how\u00a0to detect and avoid their own evolving Mid-Staffs crisis?<\/p>\n<p style=\"text-align: left\">To illustrate how it is currently done\u00a0let us use the actual\u00a0Mid-Staffs data. It is conveniently available in Figure 1 embedded in Figure 5 on Page 360 in Appendix G of Volume 1 of the first Francis Report.\u00a0 If you do not have it at your fingertips I\u00a0have\u00a0put a copy of it below.<\/p>\n<p style=\"text-align: left\"><a href=\"http:\/\/www.improvementscience.co.uk\/blog\/wp-content\/uploads\/2013\/02\/MS_RawData.jpg\"><img loading=\"lazy\" decoding=\"async\" class=\"size-full wp-image-2754 aligncenter\" src=\"http:\/\/www.improvementscience.co.uk\/blog\/wp-content\/uploads\/2013\/02\/MS_RawData.jpg\" alt=\"MS_RawData\" width=\"465\" height=\"380\" srcset=\"https:\/\/hcse.blog\/wp-content\/uploads\/2013\/02\/MS_RawData.jpg 465w, https:\/\/hcse.blog\/wp-content\/uploads\/2013\/02\/MS_RawData-300x245.jpg 300w\" sizes=\"auto, (max-width: 465px) 100vw, 465px\" \/><\/a><\/p>\n<p style=\"text-align: left\">The message does not exactly leap off the page and smack us between the eyes does it? Even with the benefit of hindsight.\u00a0 So what is the problem here?<\/p>\n<p style=\"text-align: left\">The problem is one of ergonomics. Tables of numbers like this are <strong>very<\/strong> difficult for most people to interpret, so they create a risk that we ignore the data or that we just jump to the <em>bottom line\u00a0<\/em>and miss the real message. And It is very easy to miss the message when we compare\u00a0the results for the current period with the previous one &#8211;\u00a0a very bad habit that is spread by accountants.<\/p>\n<p style=\"text-align: left\">This was a slowly emerging crisis so we need a way of seeing it evolving and the better way to present this data is as a time-series chart.<\/p>\n<p style=\"text-align: left\">As we are most interested in safety and outcomes, then we\u00a0would reasonably look at the outcome we do <strong>not<\/strong> want &#8211; i.e. mortality. \u00a0I think we will all agree that it is an easy enough one to measure.<\/p>\n<p style=\"text-align: left\"><img loading=\"lazy\" decoding=\"async\" class=\"size-full wp-image-2755 aligncenter\" src=\"http:\/\/www.improvementscience.co.uk\/blog\/wp-content\/uploads\/2013\/02\/MS_RawDeaths.jpg\" alt=\"MS_RawDeaths\" width=\"523\" height=\"344\" srcset=\"https:\/\/hcse.blog\/wp-content\/uploads\/2013\/02\/MS_RawDeaths.jpg 523w, https:\/\/hcse.blog\/wp-content\/uploads\/2013\/02\/MS_RawDeaths-300x197.jpg 300w\" sizes=\"auto, (max-width: 523px) 100vw, 523px\" \/>This is the raw mortality data from the table above, plotted as a time-series chart.\u00a0 The green line is the average and the red-lines are a measure of variation-over-time. We can all see that the raw mortality is increasing and the red flags say that this is a statistically significant increase. Oh dear!<\/p>\n<p style=\"text-align: left\"><em>But hang on just a minute &#8211; using raw mortality data like this is invalid because\u00a0we all know that\u00a0the people are getting older, demand on our hospitals is rising,\u00a0A&amp;Es are busier, older people have more illnesses, and more of them will not survive their visit to our hospital. This rise in mortality may actually just\u00a0be because we are doing more work.<\/em><\/p>\n<p style=\"text-align: left\">Good point! Let us plot the activity data and see if there has been an increase.<\/p>\n<p style=\"text-align: left\"><a href=\"http:\/\/www.improvementscience.co.uk\/blog\/wp-content\/uploads\/2013\/02\/MS_Activity.jpg\"><img loading=\"lazy\" decoding=\"async\" class=\"size-full wp-image-2756 aligncenter\" src=\"http:\/\/www.improvementscience.co.uk\/blog\/wp-content\/uploads\/2013\/02\/MS_Activity.jpg\" alt=\"MS_Activity\" width=\"523\" height=\"344\" srcset=\"https:\/\/hcse.blog\/wp-content\/uploads\/2013\/02\/MS_Activity.jpg 523w, https:\/\/hcse.blog\/wp-content\/uploads\/2013\/02\/MS_Activity-300x197.jpg 300w\" sizes=\"auto, (max-width: 523px) 100vw, 523px\" \/><\/a><\/p>\n<p style=\"text-align: left\">Yes &#8211; indeed the activity has increased significantly too.<\/p>\n<p style=\"text-align: left\"><em>Told you so! And it looks like the activity has gone up more than the mortality. Does that mean we are actually doing a <strong>better<\/strong> job at keeping people alive? That sounds\u00a0like a\u00a0more positive\u00a0message for the Board and the Annual Report. But how\u00a0do we present that message? What about as a ratio of mortality to activity? That will make it easier to compare ourselves with other\u00a0hospitals.<\/em><\/p>\n<p style=\"text-align: left\">Good idea! Here is the Raw Mortality Ratio chart.<\/p>\n<p style=\"text-align: left\"><a href=\"http:\/\/www.improvementscience.co.uk\/blog\/wp-content\/uploads\/2013\/02\/MS_RawMortality_Ratio.jpg\"><img loading=\"lazy\" decoding=\"async\" class=\"size-full wp-image-2759 aligncenter\" src=\"http:\/\/www.improvementscience.co.uk\/blog\/wp-content\/uploads\/2013\/02\/MS_RawMortality_Ratio.jpg\" alt=\"MS_RawMortality_Ratio\" width=\"523\" height=\"344\" srcset=\"https:\/\/hcse.blog\/wp-content\/uploads\/2013\/02\/MS_RawMortality_Ratio.jpg 523w, https:\/\/hcse.blog\/wp-content\/uploads\/2013\/02\/MS_RawMortality_Ratio-300x197.jpg 300w\" sizes=\"auto, (max-width: 523px) 100vw, 523px\" \/><\/a><em>Ah ha. See! The % mortality is falling significantly over time. Told you so.<\/em><\/p>\n<p style=\"text-align: left\">Careful. There is an unstated assumption here. The assumption\u00a0that the case mix is\u00a0staying the same over time. This pattern could also be the impact of us doing a greater proportion of lower complexity and lower risk work. \u00a0So we need to\u00a0correct this raw mortality data for case mix complexity &#8211; and we can do that by using data from all NHS hospitals to give us a frame of reference.\u00a0Dr Foster can help us with that because it is quite a complicated statistical modelling process. What comes out of Dr Fosters black magic box is the Global Hospital Raw Mortality (GHRM) which is the expected number of deaths for our case mix <strong>if<\/strong> we were an &#8216;average&#8217; NHS hospital.<\/p>\n<p style=\"text-align: left\"><a href=\"http:\/\/www.improvementscience.co.uk\/blog\/wp-content\/uploads\/2013\/02\/MS_ExpectedMortality_Ratio.jpg\"><img loading=\"lazy\" decoding=\"async\" class=\"size-full wp-image-2760 aligncenter\" src=\"http:\/\/www.improvementscience.co.uk\/blog\/wp-content\/uploads\/2013\/02\/MS_ExpectedMortality_Ratio.jpg\" alt=\"MS_ExpectedMortality_Ratio\" width=\"523\" height=\"344\" srcset=\"https:\/\/hcse.blog\/wp-content\/uploads\/2013\/02\/MS_ExpectedMortality_Ratio.jpg 523w, https:\/\/hcse.blog\/wp-content\/uploads\/2013\/02\/MS_ExpectedMortality_Ratio-300x197.jpg 300w\" sizes=\"auto, (max-width: 523px) 100vw, 523px\" \/><\/a><\/p>\n<p style=\"text-align: left\">What this says is that the\u00a0NHS-wide raw mortality risk appears to be falling over time (which may be\u00a0for a wide variety of reasons but that is outside the scope of this conversation). So what we now need to do is compare this global raw mortality risk with our local raw mortality risk \u00a0&#8230; to give the Hospital Standardised Mortality Ratio.<\/p>\n<p style=\"text-align: left\"><a href=\"http:\/\/www.improvementscience.co.uk\/blog\/wp-content\/uploads\/2013\/02\/MS_HSMR.jpg\"><img loading=\"lazy\" decoding=\"async\" class=\"size-full wp-image-2761 aligncenter\" src=\"http:\/\/www.improvementscience.co.uk\/blog\/wp-content\/uploads\/2013\/02\/MS_HSMR.jpg\" alt=\"MS_HSMR\" width=\"523\" height=\"344\" srcset=\"https:\/\/hcse.blog\/wp-content\/uploads\/2013\/02\/MS_HSMR.jpg 523w, https:\/\/hcse.blog\/wp-content\/uploads\/2013\/02\/MS_HSMR-300x197.jpg 300w\" sizes=\"auto, (max-width: 523px) 100vw, 523px\" \/><\/a>This gives us the\u00a0Mid Staffordshire Hospital\u00a0HSMR chart.\u00a0\u00a0The blue line at 100 is the\u00a0reference average &#8211; and what this chart says is that Mid Staffordshire hospital had a consistently higher <strong>risk<\/strong> than the average case-mix adjusted mortality risk for the whole NHS. And it says that it got even worse after 2001 and\u00a0that it stayed consistently\u00a020% higher\u00a0after 2003.<\/p>\n<p style=\"text-align: left\"><em>Ah! Oh dear! That is not such a positive message for the Board and the Annual Report. But how did we miss this evolving safety catastrophe?\u00a0 We had the\u00a0Dr Foster data from 2001<\/em><\/p>\n<p style=\"text-align: left\">This is not a new problem\u00a0&#8211; a similar thing happened in\u00a0Vienna\u00a0between 1820 and 1850\u00a0with maternal deaths caused by Childbed Fever. The problem was detected by Dr Ignaz Semmelweis who also discovered a simple, pragmatic solution to the problem: hand washing. \u00a0He blew the whistle but unfortunately those in power did not like the implication that they had been the cause of thousands of avoidable mother and baby deaths. \u00a0Semmelweis was vilified and ignored, and he did not publish his data until 1861. And even then the story was buried in tables of numbers. \u00a0Semmelweis went mad trying to convince the World that there was a problem.\u00a0\u00a0<a title=\"The Story of Ignaz Semmelweis\" href=\"http:\/\/www.saasoft.com\/baseline\/semmelweis.html\" target=\"_blank\" rel=\"noopener\">Here is the full story<\/a>.<\/p>\n<p style=\"text-align: left\">Also, time-series charts were not invented until 1924 &#8211; and it was not in healthcare &#8211; it was in manufacturing.\u00a0These tried-and-tested\u00a0safety and quality improvement tools are only slowly\u00a0diffusing\u00a0into healthcare\u00a0because the barriers to innovation\u00a0appear somewhat impervious.<\/p>\n<p style=\"text-align: left\">And the pores have been clogged even more by the social poison called &#8220;cynicide&#8221; &#8211; the emotional and political toxin exuded by cynics.<\/p>\n<p style=\"text-align: left\"><em>So how could we detect a developing crisis earlier &#8211; in time to avoid a catastrophe?<\/em><\/p>\n<p style=\"text-align: left\">The first step is to\u00a0estimate the excess-death-equivalent. Dr Foster does this for you.<a href=\"http:\/\/www.improvementscience.co.uk\/blog\/wp-content\/uploads\/2013\/02\/MS_ExcessDeaths.jpg\"><img loading=\"lazy\" decoding=\"async\" class=\"size-full wp-image-2763 aligncenter\" src=\"http:\/\/www.improvementscience.co.uk\/blog\/wp-content\/uploads\/2013\/02\/MS_ExcessDeaths.jpg\" alt=\"MS_ExcessDeaths\" width=\"523\" height=\"344\" srcset=\"https:\/\/hcse.blog\/wp-content\/uploads\/2013\/02\/MS_ExcessDeaths.jpg 523w, https:\/\/hcse.blog\/wp-content\/uploads\/2013\/02\/MS_ExcessDeaths-300x197.jpg 300w\" sizes=\"auto, (max-width: 523px) 100vw, 523px\" \/><\/a>Here is\u00a0the data from the table plotted as a time-series chart that shows that the estimated-excess-death-equivalent per year. It has an average of 100 (that is two per week) and the average should be\u00a0close to zero. More worryingly the\u00a0number was increasing steadily over time up to 200 per year in 2006 &#8211; that is about four excess deaths per week &#8211; on average.\u00a0 It is important to remember that HSMR is a risk ratio\u00a0and mortality is a multi-factorial outcome. So the excess-death-equivalent estimate does <strong>not<\/strong>\u00a0imply that\u00a0a clear causal chain will be evident in specific deaths. That is a complete misunderstanding of the method.<\/p>\n<p style=\"text-align: left\"><em>I am sorry &#8211; you are losing me with the statistical jargon here. Can you explain in plain English what you mean?<\/em><\/p>\n<p style=\"text-align: left\">OK. Let us use an example.<\/p>\n<p style=\"text-align: left\">Suppose we\u00a0set up\u00a0a tombola\u00a0at the village fete and\u00a0we sell 50 tickets with the expectation that the winner bags all the money. Each ticket holder has the same 1 in 50 risk of\u00a0winning the wad-of-wonga\u00a0and a\u00a049 in 50 risk of losing their small stake. At the appointed time\u00a0we spin the barrel to mix up the ticket stubs then we\u00a0blindly draw one ticket out. At that instant the 50 people with an equal\u00a0risk changes to one\u00a0winner and 49 losers. It is as if the grey fog of risk instantly\u00a0condenses into a precise, black-and-white, yes-or-no, winner-or-loser, reality.<\/p>\n<p style=\"text-align: left\">Translating this concept back into HSMR and Mid Staffs &#8211; the estimated 1200 deaths are the just the\u00a0&#8220;condensed risk of harm equivalent&#8221;. \u00a0So, to then conduct a retrospective case note analysis of specific deaths looking for\u00a0the specific cause would be equivalent to trying to retrospectively work out the reason the particular winning ticket in the\u00a0tombola was picked out. It is a search that is doomed to fail. To then conclude from this fruitless search that HSMR is invalid, is only to compound the\u00a0delusion further.\u00a0 The actual problem here is\u00a0ignorance and misunderstanding of the basic Laws of Physics and Probability, because our brains are not good at solving these sort of problems.<\/p>\n<p style=\"text-align: left\"><em>But Mid Staffs is a particularly\u00a0severe example and\u00a0 it only shows up after years of data has accumulated. How would a hospital that was not as bad as this know they had a risk problem and know sooner? Waiting for years to accumulate enough data to prove there was a avoidable problem in the past is not much help.\u00a0<\/em><\/p>\n<p style=\"text-align: left\">That is an excellent question. This type of\u00a0time-series chart is not very sensitive to small changes when the data is noisy and sparse &#8211; such as when you plot the data on a month-by-month\u00a0timescale and avoidable deaths are actually an uncommon outcome. Plotting the annual sum\u00a0smooths out this variation and makes the trend easier to see, but it delays the diagnosis further.\u00a0One way to increase the sensitivity\u00a0is to plot the data as a cusum (cumulative sum) chart &#8211; which is conspicuous by its absence from the data table. It is the running total of the estimated\u00a0excess deaths. Rather like the running total of swings in a game of golf.<\/p>\n<p style=\"text-align: left\"><a href=\"http:\/\/www.improvementscience.co.uk\/blog\/wp-content\/uploads\/2013\/02\/MS_ExcessDeaths_CUSUM.jpg\"><img loading=\"lazy\" decoding=\"async\" class=\"size-full wp-image-2764 aligncenter\" src=\"http:\/\/www.improvementscience.co.uk\/blog\/wp-content\/uploads\/2013\/02\/MS_ExcessDeaths_CUSUM.jpg\" alt=\"MS_ExcessDeaths_CUSUM\" width=\"523\" height=\"344\" srcset=\"https:\/\/hcse.blog\/wp-content\/uploads\/2013\/02\/MS_ExcessDeaths_CUSUM.jpg 523w, https:\/\/hcse.blog\/wp-content\/uploads\/2013\/02\/MS_ExcessDeaths_CUSUM-300x197.jpg 300w\" sizes=\"auto, (max-width: 523px) 100vw, 523px\" \/><\/a>This is the cusum chart of excess deaths and you will notice that it is not plotted with control limits. That is because it is invalid to use standard control limits\u00a0for\u00a0cumulative data.\u00a0 The important feature of the cusum chart is the slope and the deviation from zero. What is usually done is an <em>alert threshold<\/em> is plotted on the cusum chart and if the measured cusum crosses this alert-line then the\u00a0alarm bell should go off &#8211; and the search then focuses on the precursor events: the Near Misses, the Not Agains and the Niggles.<\/p>\n<p style=\"text-align: left\"><em>I see. You make it look easy when the data is presented as pictures. But aren&#8217;t we still missing the point?\u00a0Isn&#8217;t this\u00a0still after-the-avoidable-event analysis?<\/em><\/p>\n<p style=\"text-align: left\">Yes! An avoidable death should be a\u00a0Never-Event in a designed-to-be-safe healthcare system. It should\u00a0never happen. There should be no coffins\u00a0to count. To get to that stage we need to apply exactly the same approach to the Near-Misses, and then the Not-Agains, and eventually the Niggles.<\/p>\n<p style=\"text-align: left\"><em>You mean we have to use the SUI data and the IR1 data and the complaint data to do this &#8211; and also\u00a0ask our staff and patients about their Niggles?<\/em><\/p>\n<p style=\"text-align: left\">Yes. And it is not the number of complaints that is the most useful metric &#8211; it is the appearance of the cumulative sum of the complaint severity score. And we need a method for diagnosing and treating the cause of the Niggles too. We need to convert the feedback information into effective action.<\/p>\n<p style=\"text-align: left\"><em>Ah ha! Now I understand what the role of the Governance\u00a0Department is: to\u00a0apply the tools and techniques\u00a0of Improvement Science proactively.\u00a0 But our Governance Department have not been trained to\u00a0do this!<\/em><\/p>\n<p style=\"text-align: left\">Then that is\u00a0one place to start &#8211; and their role needs to evolve from Inspectors and Supervisors to Demonstrators and Educators &#8211; ultimately everyone\u00a0in the organisation needs to be a competent\u00a0Healthcare Improvementologist.<\/p>\n<p style=\"text-align: left\"><em>OK &#8211; I now now what to do next. But wait a minute. This is going to cost a fortune!<\/em><\/p>\n<p style=\"text-align: left\">This is just one small first step.\u00a0\u00a0The next step is to redesign the processes so the errors do not happen in the first place. The cumulative cost saving from eliminating the\u00a0repeated checking, correcting, box-ticking, documenting,\u00a0investigating, compensating and insuring is much much more than the one-off investment in learning safe system design.<\/p>\n<p style=\"text-align: left\"><em>So the Finance Director should be a champion for safety and quality too.<\/em><\/p>\n<p style=\"text-align: left\">Yup!<\/p>\n<p style=\"text-align: left\"><em>Brill. Thanks.\u00a0And can I ask one more question?\u00a0I do not want to appear to skeptical but how do we know we can trust that this risk-estimation system\u00a0has been designed and implemented correctly? How do we know we are not being bamboozled by statisticians? It has happened before!<\/em><\/p>\n<p style=\"text-align: left\">That is the best question yet.\u00a0 It is important to remember that\u00a0HSMR is counting <em>deaths in hospital <\/em>which means that it is not actually the risk of harm to the patient that is measured &#8211; it is the <strong>risk to the reputation of\u00a0hospital<\/strong>! So the answer to your question is that you demonstrate your deep understanding of the rationle and method of risk-of-harm\u00a0estimation by listing\u00a0all the ways that such a system could be deliberately &#8220;gamed&#8221; to make the figures look better for the hospital. And then go out and\u00a0look for hard evidence of all the &#8220;games&#8221; that you can\u00a0invent. It is a sort\u00a0of creative\u00a0poacher-becomes-gamekeeper detective exercise.<\/p>\n<p style=\"text-align: left\"><em>OK\u00a0&#8211; I sort of get what you mean. Can you give me\u00a0some examples?<\/em><\/p>\n<p style=\"text-align: left\">Yes. The\u00a0HSMR method is based on deaths-in-hospital so discharging a patient from hospital before they die will make the figures look better. Suppose\u00a0one hospital has more access to\u00a0end-of-life care in the community\u00a0than another: their HSMR\u00a0figures would look better even though exactly the same number of people died. Another is that the\u00a0HSMR method is weighted towards admissions classified as &#8220;emergencies&#8221; &#8211; so if a hospital admits more patients as &#8220;emergencies&#8221;\u00a0who are not actually very sick and discharges them quickly then this will inflated their estimated deaths and make their actual mortality ratio look better &#8211; even though\u00a0the risk-of-harm\u00a0to patients has not changed.<\/p>\n<p style=\"text-align: left\"><em>OMG &#8211; so if we have pressure to meet 4 hour A&amp;E targets and we get paid more for an emergency admission than an A&amp;E attendance then admitting to an Assessmen Area and discharging within one day will\u00a0actually reward the hospital financially, operationally and by apparently reducing their HSMR even though there has been no difference at all to the care that patients actually recieve?<\/em><\/p>\n<p style=\"text-align: left\"><em>Y<\/em>es. It is an inevitable outcome of the current system design.<\/p>\n<p style=\"text-align: left\"><em>But that means that if I am gaming the system and my HSMR is not getting better then the risk-of-harm to patients is actually increasing and my HSMR system is giving me false reassurance that everything is OK.\u00a0\u00a0\u00a0Wow! I can see why some people might not want that realisation to be public knowledge. So what do we do?<\/em><\/p>\n<p style=\"text-align: left\">Design the system so that the rewards are aligned with\u00a0lower risk of harm to patients and improved outcomes.<\/p>\n<p style=\"text-align: left\"><em>Is that possible?<\/em><\/p>\n<p style=\"text-align: left\">Yes. It is called a Win-Win-Win design.<\/p>\n<p style=\"text-align: left\"><em>How do we learn how to do that?<\/em><\/p>\n<p style=\"text-align: left\">Improvement Science.<\/p>\n<p style=\"text-align: left\"><span style=\"color: #ff0000\">Footnote I:<\/span><\/p>\n<p style=\"text-align: left\"><span style=\"color: #ff0000\">The graphs tell a story but they may not create a useful sense of perspective. It\u00a0has been said that there is a 1 in 300 chance that if you\u00a0go to hospital you will not leave alive for avoidable causes. What! It cannot be as high as 1 in 300 surely?<\/span><\/p>\n<p style=\"text-align: left\"><span style=\"color: #ff0000\">OK &#8211; let us use the published\u00a0Mid-Staffs data to test this hypothesis. Over 12 years\u00a0there were about 150,000 admissions and an estimated 1,200 excess deaths (if all the risk were concentrated into the excess deaths which is not what actually happens). That means a 1 in 130\u00a0odds of an avoidable death for every admission! That is <span style=\"text-decoration: underline\">twice<\/span> as bad as the estimated average.<\/span><\/p>\n<p style=\"text-align: left\"><span style=\"color: #ff0000\">The Mid Staffordshire statistics are\u00a0bad enough;\u00a0but the NHS-as-a-whole statistics are cumulatively\u00a0worse because there are 100&#8217;s of other <\/span><span style=\"color: #ff0000\">hospitals that are each generating not-as-obvious\u00a0avoidable mortality. The data is very &#8216;noisy&#8217; so it is difficult even for a statistical expert to separate the\u00a0message from the morass.<\/span><\/p>\n<p style=\"text-align: left\"><span style=\"color: #ff0000\">And remember &#8211; that\u00a0 the &#8220;expected&#8221; mortality is estimated from the average for the whole NHS &#8211; which means that if this average is higher than it could be then\u00a0there is a statistical bias and we are being falsely reassured by being\u00a0&#8216;not statistically significantly different&#8217; from the pack.<\/span><\/p>\n<p style=\"text-align: left\"><span style=\"color: #ff0000\">And remember too\u00a0&#8211; for every patient and family that suffers and avoidable death there are many more that have to\u00a0live with\u00a0the consequences of\u00a0<span style=\"text-decoration: underline\">avoidable\u00a0but non-fatal<\/span> harm.\u00a0\u00a0That is called avoidable morbidity.\u00a0 This is what the risk really means &#8211; everyone has a higher risk of some degree of avoidable harm. Psychological and physical harm.<\/span><\/p>\n<p style=\"text-align: left\"><span style=\"color: #ff0000\">This challenge is not\u00a0just about preventing another\u00a0Mid Staffs &#8211; it is about preventing 1000&#8217;s of avoidable deaths\u00a0and 100,000s of patients avoidably harmed every year in &#8216;average&#8217;\u00a0NHS trusts.<\/span><\/p>\n<p style=\"text-align: left\"><span style=\"color: #ff0000\">It is not a mass conspiracy of bad nurses, bad doctors, bad managers or bad policians that\u00a0is the root cause.<\/span><\/p>\n<p style=\"text-align: left\"><span style=\"color: #ff0000\">It\u00a0is <span style=\"text-decoration: underline\">poorly designed processes<\/span>\u00a0&#8211; and they are poorly designed because\u00a0the nurses, doctors and managers have not learned how to design better ones.\u00a0\u00a0And\u00a0we do not know\u00a0how because\u00a0we were not trained to.\u00a0\u00a0And that education gap was an accident &#8211; an unintended error of omission.\u00a0\u00a0<\/span><\/p>\n<p style=\"text-align: left\"><span style=\"color: #ff0000\">Our urgently-improve-NHS-safety-challenge requires a system-wide\u00a0safety-by-design educational and cultural transformation.<\/span><\/p>\n<p style=\"text-align: left\"><span style=\"color: #ff0000\">And that is possible because the knowledge\u00a0of how to design, test and implement inherently safe processes\u00a0exists. But it exists outside healthcare.<\/span><\/p>\n<p style=\"text-align: left\"><span style=\"color: #ff0000\">And that safety-by-design training is\u00a0a worthwhile investment because safer-by-design processes cost less to run because they require less checking, less documenting,\u00a0less correcting &#8211; and all the valuable nurse, doctor and manager time freed up by that can be reinvested in more care, better care\u00a0and designing even better processes and systems.<\/span><\/p>\n<p style=\"text-align: left\"><span style=\"color: #ff0000\">Everyone Wins &#8211; except the cynics who have a choice: to eat humble pie or leave.<\/span><\/p>\n<p style=\"text-align: left\"><span style=\"color: #000080\">Footnote II:<\/span><\/p>\n<p style=\"text-align: left\"><span style=\"color: #000080\">In the debate that has followed the publication of the Francis Report a lot of scrutiny has been applied to the method by which an\u00a0<em>estimated<\/em> excess\u00a0mortality number is created and it is necessary to explore this in a bit more detail.<\/span><\/p>\n<p style=\"text-align: left\"><span style=\"color: #000080\">The\u00a0HSMR is an estimate\u00a0of relative risk &#8211; it\u00a0does not say that a set of specific patients were the ones who came to harm and the rest were OK. So looking at individual deaths and looking for the specific causes is\u00a0to completely misunderstand the method. So looking at the actual deaths individually and looking for identifiable cause-and-effect paths is an\u00a0misuse of the message.\u00a0\u00a0When very few if any are found to conclude that HSMR is flawed is an error of logic and exposes the ignorance of the analyst further.<\/span><\/p>\n<p style=\"text-align: left\"><span style=\"color: #000080\">HSMR\u00a0is not perfect though &#8211; it has weaknesses.\u00a0\u00a0It is a benchmarking process\u00a0the&#8221;standard&#8221; of 100 is always moving because the collective goal posts are moving &#8211; the reference is always changing . HSMR is estimated using data submitted by hospitals themselves &#8211; the clinical coding data.\u00a0\u00a0So the main\u00a0weakness is that it is\u00a0dependent on the quality of the clinicial coding &#8211; the errors of comission (wrong codes) and the errors of omission (missing codes). Garbage In Garbage Out.<\/span><\/p>\n<p style=\"text-align: left\"><span style=\"color: #000080\">Hospitals use clinically coded data for other reasons &#8211; payment. The way hospitals are now paid\u00a0is based on the volume and complexity of that activity &#8211; Payment By Results (PbR) &#8211;\u00a0using what are called Health Resource Groups (HRGs). This is a better and fairer\u00a0design because hospitals with more complex (i.e. costly to manage) case loads get paid more per patient on average.\u00a0 The\u00a0HRG for each patient is determined by their clinical codes &#8211; including what are\u00a0called the comorbidities &#8211;\u00a0the other things that\u00a0the patient has wrong with them. More comorbidites means more complex and more risky so more money and more risk of death\u00a0&#8211; roughly speaking.\u00a0 So when PbR came in it becamevery important to code fully\u00a0in order\u00a0to get paid &#8220;properly&#8221;.\u00a0 The problem was that before PbR the coding errors went largely unnoticed &#8211; especially the comorbidity coding. And the errors were biassed &#8211; it\u00a0is more likely to omit a code than to have an incorrect code. Errors\u00a0of omission are harder to detect.\u00a0This meant that by more complete coding (to attract more money) the estimated casemix complexity would have gone up compared with the historical reference.\u00a0So as actual (not estimated)\u00a0NHS mortality has gone down slightly then the HSMR yardstick becomes even more distorted.\u00a0 Hospitals that did not keep up with the Coding Game would look worse even though\u00a0 their actual risk and mortality may be unchanged.\u00a0 This is the fundamental design flaw in all types of \u00a0benchmarking based on self-reported data.<\/span><\/p>\n<p style=\"text-align: left\"><span style=\"color: #000080\">The actual problem\u00a0here is even more serious. PbR is actually a payment for activity &#8211; not a payment for\u00a0outcomes.\u00a0It is calculated from what it cost to run the average NHS hospital using a technique called Reference Costing which is the same\u00a0method that manufacturing companies used to decide what price to charge for their products. It has another name &#8211;\u00a0Absorption Costing.\u00a0\u00a0The highest performers in the manufacturing world no longer use this out-of-date method. The implication of using Reference Costing and PbR in the NHS are profound and dangerous:<\/span><\/p>\n<p style=\"text-align: left\"><span style=\"color: #000080\">If NHS hospitals in general have poorly designed processes that create internal queues and require more bed days than\u00a0actually necessary then the cost of that &#8220;waste&#8221; becomes built into the future\u00a0PbR tariff. This means average length of stay (LOS) is financially rewarded. Above average LOS is financially penalised and below average LOS makes a profit.\u00a0 There is no financial pressure to improve beyound average. This is called the Regression to the Mean effect.\u00a0 Also\u00a0LOS is not a measure of quality &#8211; so there is a to shorten length of stay for purely financial reasons &#8211; to generate a surplus to use to fund growth and capital investment.\u00a0 That pressure is non-specific and indiscrimiate.\u00a0 PbR is necessary but it is not sufficient &#8211; it requires\u00a0an quality of outcome metric to complete it.\u00a0\u00a0\u00a0\u00a0 <\/span><\/p>\n<p style=\"text-align: left\"><span style=\"color: #000080\">So the PbR system is based on an out-of-date cost-allocation model and therefore leads\u00a0to the very problems that are contributing to the MidStaffs crisis &#8211; financial pressure causing quality failures and increased risk of mortality.\u00a0 MidStaffs may be a chance victim of a combination of factors coming together like a perfect storm &#8211; but those same\u00a0factors are present throughout the NHS because they are built into the current design.<\/span><\/p>\n<p style=\"text-align: left\"><span style=\"color: #000080\">One solution is to move towards a more\u00a0up-to-date financial model called\u00a0stream costing. This\u00a0uses the similar data\u00a0to reference costing but it estimates the &#8220;ideal&#8221; cost\u00a0of the &#8220;necessary&#8221; work to achieve the intended outcome. This\u00a0stream cost becomes the focus for improvement &#8211; the streams where there is the biggest gap between the\u00a0stream cost and the reference cost are the focus of the redesign activity. Very often the root cause is just poor operational policy design; sometimes it is quality and safety design problems. Both are solvable without\u00a0investment in extra capacity. The result is a higher quality, quicker, lower-cost stream. Win-win-win. And in the short term that \u00a0is rewarded by a tariff income\u00a0that exceeds\u00a0cost and a lower HSMR.<\/span><\/p>\n<p style=\"text-align: left\"><span style=\"color: #000080\">Radically redesigning the financial model\u00a0for\u00a0healthcare is not a quick fix &#8211; and it requires a lot of other changes to happen first. So the sooner we start the sooner we will arrive.\u00a0<\/span><\/p>\n","protected":false},"excerpt":{"rendered":"<p>The retrospectoscope is the favourite instrument of the forensic cynic &#8211; the expert in the after-the-event-and-I-told-you-so rhetoric. The\u00a0rabble-rouser for the lynch-mob. It feels better to\u00a0retrospectively nail-to-a-cross the person who committed the Cardinal Error of Omission, and leave them there in emotional and financial pain as a visible lesson to everyone else. This form of\u00a0public feedback &hellip; <\/p>\n<p class=\"link-more\"><a href=\"https:\/\/hcse.blog\/?p=2753\" class=\"more-link\">Continue reading<span class=\"screen-reader-text\"> &#8220;The Writing on the Wall &#8211; Part II&#8221;<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[5,6,7,17,19,21,22,24,30,33,35,38,42,43,45,46,48,49],"tags":[],"class_list":["post-2753","post","type-post","status-publish","format-standard","hentry","category-4n-chart","category-6m-design","category-baseline","category-examples","category-fish","category-governance","category-healthcare","category-improvementology","category-operations","category-quality","category-reflections","category-safety","category-how","category-why","category-what","category-teach","category-trust","category-victimosis"],"_links":{"self":[{"href":"https:\/\/hcse.blog\/index.php?rest_route=\/wp\/v2\/posts\/2753","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/hcse.blog\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/hcse.blog\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/hcse.blog\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/hcse.blog\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=2753"}],"version-history":[{"count":0,"href":"https:\/\/hcse.blog\/index.php?rest_route=\/wp\/v2\/posts\/2753\/revisions"}],"wp:attachment":[{"href":"https:\/\/hcse.blog\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=2753"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/hcse.blog\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=2753"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/hcse.blog\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=2753"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}