{"id":4530,"date":"2016-01-31T10:28:31","date_gmt":"2016-01-31T09:28:31","guid":{"rendered":"http:\/\/www.saasoft.com\/blog\/?p=4530"},"modified":"2016-01-31T10:28:31","modified_gmt":"2016-01-31T09:28:31","slug":"the-magic-black-box","status":"publish","type":"post","link":"https:\/\/hcse.blog\/?p=4530","title":{"rendered":"The Magic Black Box"},"content":{"rendered":"

\"stick_figure_magic_carpet_150_wht_5040\"<\/a>It was the appointed time for Bob and Leslie’s regular coaching session as part of the improvement science practitioner programme.<\/p>\n

<Leslie><\/em> Hi Bob, I am feeling rather despondent today so\u00a0please excuse me in advance if\u00a0you hear a lot of “Yes, but …”<\/em> language.<\/p>\n

<Bob><\/em>\u00a0I am sorry to hear that Leslie. Do you want to talk about\u00a0it?<\/p>\n

<Leslie><\/em> Yes, please.\u00a0 The trigger for my gloom was\u00a0being sent on a mandatory training workshop.<\/p>\n

<Bob><\/em> OK. Training to do what?<\/p>\n

<Leslie><\/em> Outpatient demand and capacity planning!<\/p>\n

<Bob> But you know how to do that already, so what is the reason you were “sent”?<\/p>\n

<Leslie><\/em>\u00a0Well, I am no longer\u00a0sure I know how to it.\u00a0 That is why I am feeling so blue.\u00a0 I went more out of curiosity and I came away utterly\u00a0confused and with my confidence shattered.<\/p>\n

<Bob><\/em>\u00a0Oh dear!\u00a0We had better start at the beginning.\u00a0 What was the purpose of the workshop?<\/p>\n

<Leslie><\/em> To train everyone in how to use\u00a0an Outpatient Demand and Capacity planning\u00a0model, an Excel one that we were told to download along with the User Guide.\u00a0 I think it is part of a national push to improve waiting times for outpatients.<\/p>\n

<Bob><\/em> OK.\u00a0On the surface that sounds reasonable. You have designed and built your own Excel flow-models already; so where\u00a0did the trouble start?<\/p>\n

<Leslie><\/em>\u00a0I will\u00a0attempt to explain.\u00a0 This was a paragraph in the instructions. I felt OK with this because my Improvement Science training has given me a very good understanding of basic demand and capacity theory.<\/p>\n

\"IST_DandC_Model_01\"<\/a><Bob><\/em> OK.\u00a0 I am guessing that other delegates may have felt less comfortable with this. Was that the case?<\/p>\n

<Leslie><\/em> The training workshops are targeted at Operational Managers and the ones I spoke to actually felt that they had a good grasp of the\u00a0basics.<\/p>\n

<Bob><\/em> OK. That is encouraging, but a warning bell is ringing for me. So where did the trouble start?<\/p>\n

<Leslie><\/em> Well, before going to the workshop I decided to read the User Guide\u00a0so that I had some idea of\u00a0how this magic tool worked.\u00a0 This is where I started to wobble – this paragraph specifically …<\/p>\n

\"IST_DandC_Model_02\"<\/a><\/p>\n

<Bob><\/em>\u00a0H’mm. What did you make of that?<\/p>\n

<Leslie><\/em> It was complete gibberish to me and I felt like an idiot for not understanding it.\u00a0 I went to the workshop in a bit of a panic\u00a0and\u00a0hoped that all would become clear. It didn’t.<\/p>\n

<Bob><\/em> Did the User Guide explain what\u00a0‘percentile’ means in this context, ideally with some visual charts to assist?<\/p>\n

<Leslie><\/em> No and the use of ‘th’ and ‘%’ was\u00a0really confusing too.\u00a0 After that I sort of went into a mental fog and none of the workshop made much sense.\u00a0 It was all about practising using the tool without any understanding of\u00a0how it worked. Like a black magic box.<\/p>\n

\n

<Bob><\/em> OK.\u00a0\u00a0I can see why you were confused, and do not worry, you are not an idiot.\u00a0 It\u00a0looks like the author of the User Guide has unwittingly used some very confusing\u00a0and ambiguous terminology here.\u00a0\u00a0So can you talk me through what you\u00a0have to do to use this magic box?<\/p>\n

<Leslie><\/em> First we have to enter some of our historical data; the number of new referrals\u00a0per week for a year; and the referral and appointment dates for all patients for the most recent three months.<\/p>\n

<Bob><\/em> OK. That sounds very reasonable.\u00a0 A run chart of historical demand and the raw event data for a Vitals Chart\u00ae\u00a0is where I would start the measurement phase\u00a0too – so long as the data\u00a0creates a valid 3 month reporting\u00a0window.<\/p>\n

<Leslie><\/em> Yes, I though so too … but that is not how the black box model seems to work. The weekly demand is used to draw an SPC chart,\u00a0but the event data seems to disappear into\u00a0the innards of the black box, and recommendations pop out of it.<\/p>\n

<Bob><\/em> Ah ha!\u00a0 And let me guess the relationship between the\u00a0term ‘percentile’ and the SPC chart of weekly new demand was not explained?<\/p>\n

<Leslie><\/em> Spot on.\u00a0 What does percentile mean?<\/p>\n

\n

<Bob><\/em>\u00a0It is statistics jargon. Remember that we have talked about the distribution of the data around the average\u00a0on a BaseLine chart; and how we use the histogram feature\u00a0of BaseLine to show it visually.\u00a0 Like this example.<\/p>\n

\"IST_DandC_Model_03\"<\/a><Leslie><\/em> Yes. I recognise that. This chart shows a stable system of demand with an average of around 150 new referrals per week and the variation distributed above and below the average in a symmetrical pattern, falling off\u00a0to\u00a0zero around the upper and lower process limits.\u00a0\u00a0I believe that you said that over 99% will\u00a0fall within the limits.<\/p>\n

<Bob><\/em> Good.\u00a0\u00a0The blue histogram on this chart is called a probability distribution function<\/em>, to use the\u00a0terminology of a statistician.<\/p>\n

<Leslie><\/em> OK.<\/p>\n

<Bob><\/em> So, what would happen if we created a Pareto chart of demand using the number of patients per week as the categories and ignoring the\u00a0time aspect? We are allowed to do that if the behaviour is stable, as this chart suggests.<\/p>\n

<Leslie> <\/em>Give me a minute, I will\u00a0need to do a rough sketch. Does this look right?<\/p>\n

\"IST_DandC_Model_04\"<\/a><\/p>\n

<Bob><\/em> Perfect!\u00a0 So if you now convert the Y-axis to a percentage scale so that 52 weeks is 100% then where does the average weekly demand of about 150 fall? Read up from the X-axis to the line then across to the Y-axis.<\/p>\n

<Leslie><\/em>\u00a0At about 26 weeks or\u00a050% of 52 weeks.\u00a0\u00a0Ah ha!\u00a0 So that is what a percentile means!\u00a0 The 50th percentile is the average, the zeroth percentile is around the lower process limit and the 100th percentile is around the upper process limit!<\/p>\n

<Bob><\/em> In this case the 50th percentile is the average, it is not always\u00a0the case\u00a0though.\u00a0 So where is the 85th percentile line?<\/p>\n

<Leslie><\/em> Um, 52\u00a0times 0.85 is 44.2 which, reading across from the\u00a0Y-axis then down to the X-axis\u00a0gives a weekly demand of about\u00a0170 per week.\u00a0 That is about the same as the average plus one sigma according to the run chart.<\/p>\n

<Bob>\u00a0Excellent.\u00a0The Pareto chart that you have drawn is called a cumulative probability distribution function<\/em> … and that is usually what percentiles refer to. Comparative Statisticians love these but often omit to explain their rationale to non-statisticians!<\/p>\n

\n

<Leslie><\/em>\u00a0Phew!\u00a0 So, now I can see that the 65th percentile is just above average demand, and 85th percentile is\u00a0above that.\u00a0\u00a0But in the confusing paragraph\u00a0how does that relate to the phrase\u00a0“65% and 85% of the time”?<\/p>\n

<Bob><\/em> It doesn’t. That is the really, really<\/strong> confusing part of\u00a0 that paragraph. I am not surprised that you looped out at that point!<\/p>\n

<Leslie><\/em>\u00a0OK. Let us leave that for another conversation.\u00a0 If I ignore that bit then does the rest of\u00a0it make sense?<\/p>\n

<Bob><\/em> Not yet alas.\u00a0We need to dig a bit deeper. What would you say are the implications of this message?<\/p>\n

\n

<Leslie><\/em> Well.\u00a0 I know that if our flow-capacity is less than our average demand then we will guarantee to create\u00a0an unstable queue and chaos. That is the Flaw of Averages trap.<\/p>\n

<Bob><\/em> OK.\u00a0\u00a0The creator of this tool seems to know that.<\/p>\n

<Leslie><\/em> And my outpatient manager colleagues are always complaining that they do not have enough slots to book into, so I conclude that our current flow-capacity is just above\u00a0the\u00a050th percentile.<\/p>\n

<Bob><\/em> A reasonable\u00a0hypothesis.<\/p>\n

<Leslie><\/em> So to\u00a0calm the chaos the\u00a0message is saying\u00a0I will need to increase my flow capacity up to the 85th percentile\u00a0of demand which is from about 150 slots per week to 170 slots per week. An increase of 7% which implies a 7% increase in costs.<\/p>\n

<Bob><\/em> Good.\u00a0 I am pleased that you did not fall into the intuitive trap that a increase from the 50th to the 85th percentile implies a 35\/50 or 70% increase! Your estimate of 7% is a reasonable one.<\/p>\n

<Leslie><\/em> Well it may be theoretically reasonable but it is not practically possible. We are exhorted to reduce costs by at least that amount.<\/p>\n

<Bob><\/em>\u00a0So we have a finance versus governance bun-fight\u00a0with the operational managers caught in the middle: FOG. That is not the end of the litany of woes … is there anything about Did Not Attends in the model?<\/p>\n

\n

<Leslie><\/em> Yes indeed!\u00a0We are required to enter the\u00a0percentage of DNAs and\u00a0what we do with them. Do we\u00a0discharge them\u00a0or re-book them.<\/p>\n

<Bob><\/em> OK.\u00a0Pragmatic reality is always\u00a0much more interesting than academic rhetoric and this aspect of the real system rather complicates things, at least for a comparative statistician. This is where the smoke and mirrors will appear and they will be hidden inside the black magic box.\u00a0 To solve this conundrum we need to understand the relationship between demand, capacity, variation and yield … and it is rather counter-intuitive.\u00a0 So, how would you approach this problem?<\/p>\n

<Leslie><\/em> I would use the 6M Design\u00ae framework and I would start with a map and\u00a0not with a model; least of all a magic black box\u00a0one that I did not design, build and verify myself.<\/p>\n

<Bob><\/em> And how do you know that will work any better?<\/p>\n

<Bob><\/em> Indeed you did.\u00a0 So shall we do this one as an ISP-2 project?<\/p>\n

<Leslie><\/em>\u00a0An excellent suggestion. \u00a0I already feel my confidence flowing back and I am looking forward to this new challenge. Thank you again Bob.<\/p>\n

\n","protected":false},"excerpt":{"rendered":"

It was the appointed time for Bob and Leslie’s regular coaching session as part of the improvement science practitioner programme. <Leslie> Hi Bob, I am feeling rather despondent today so\u00a0please excuse me in advance if\u00a0you hear a lot of “Yes, but …” language. <Bob>\u00a0I am sorry to hear that Leslie. Do you want to talk … <\/p>\n

Continue reading “The Magic Black Box”<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[6,7,8,14,15,17,20,22,26,30,42,43,45,46],"tags":[],"class_list":["post-4530","post","type-post","status-publish","format-standard","hentry","category-6m-design","category-baseline","category-bobles","category-delivery","category-design","category-examples","category-flow","category-healthcare","category-isp","category-operations","category-how","category-why","category-what","category-teach"],"_links":{"self":[{"href":"https:\/\/hcse.blog\/index.php?rest_route=\/wp\/v2\/posts\/4530","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/hcse.blog\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/hcse.blog\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/hcse.blog\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/hcse.blog\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=4530"}],"version-history":[{"count":0,"href":"https:\/\/hcse.blog\/index.php?rest_route=\/wp\/v2\/posts\/4530\/revisions"}],"wp:attachment":[{"href":"https:\/\/hcse.blog\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=4530"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/hcse.blog\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=4530"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/hcse.blog\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=4530"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}

<Leslie><\/em> Because at the One Day ISP Workshop<\/a> I saw it work with my own eyes. The queues, waits and chaos just evaporated.\u00a0 And it cost nothing.\u00a0 We already had more than enough “capacity”.<\/p>\n