{"id":4609,"date":"2016-03-19T11:14:31","date_gmt":"2016-03-19T10:14:31","guid":{"rendered":"http:\/\/www.saasoft.com\/blog\/?p=4609"},"modified":"2016-03-19T11:14:31","modified_gmt":"2016-03-19T10:14:31","slug":"type-ii-error","status":"publish","type":"post","link":"https:\/\/hcse.blog\/?p=4609","title":{"rendered":"Type II Error"},"content":{"rendered":"

\"figure_pointing_out_chart_data_150_clr_8005\"<\/a>It was the time for Bob and Leslie’s regular Improvement Science\u00a0coaching session.<\/p>\n

<Leslie><\/em> Hi Bob, how are you today?<\/p>\n

<Bob><\/em>\u00a0I am getting over a winter cold but otherwise I am good.\u00a0 And you?<\/p>\n

<Leslie><\/em> I am OK and I\u00a0need to talk something through with you because I suspect you will be able to\u00a0help.<\/p>\n

<Bob><\/em> OK. What is the context?<\/p>\n

<Leslie><\/em> Well, one of the projects that I am involved with is looking at the elderly unplanned admission stream which accounts for less than half of our unplanned admissions but more than half of our bed days.<\/p>\n

<Bob><\/em> OK.\u00a0So what were you looking to improve?<\/p>\n

<Leslie><\/em> We want to reduce the average length of stay so that we free up beds to provide resilient space-capacity to ease the 4-hour A&E admission\u00a0delay niggle.<\/p>\n

<Bob><\/em> That sounds like a very reasonable strategy.\u00a0 So have you\u00a0made any changes and measured any improvements?<\/p>\n

<Leslie>\u00a0<\/em>We worked through the 6M Design\u00ae sequence. We\u00a0studied the current system,\u00a0diagnosed some\u00a0time traps and bottlenecks, redesigned the ones we could influence, modified the system, and continued to measure to monitor the effect.<\/p>\n

<Bob><\/em> And?<\/p>\n

<Leslie><\/em> It feels<\/em> better but the system behaviour charts do not show an improvement.<\/p>\n

<Bob><\/em> Which charts, specifically?<\/p>\n

<Leslie><\/em> The BaseLine XmR charts of\u00a0average length of stay for each week of activity.<\/p>\n

<Bob><\/em> And you locked the limits when you made the changes?<\/p>\n

<Leslie><\/em> Yes.\u00a0And there still were no red flags. So that means our changes have not had a significant effect. But it definitely feels better. Am I deluding myself?<\/p>\n

<Bob><\/em> I do not\u00a0believe so.\u00a0Your subjective assessment is very likely to be accurate. Our Chimp OS 1.0<\/em> is very good at some things!\u00a0I think the issue is with the tool you are using to measure the change.<\/p>\n

<Leslie><\/em> The XmR chart?\u00a0 But I thought that was THE tool to use?<\/p>\n

<Bob><\/em> Like all tools it is designed for a specific purpose.\u00a0 Are you familiar with the term Type II Error<\/em>.<\/p>\n

<Leslie><\/em> Doesn’t that come from research? I seem to remember that is the error we make when we have an under-powered study.\u00a0 When our sample size\u00a0is too small to confidently detect the change in the mean that we are looking for.<\/p>\n

<Bob><\/em> A perfect definition!\u00a0 The same error can happen when we are doing before and after studies too.\u00a0 And\u00a0when it does, we see the pattern\u00a0you have just described: the process feels better but we do not see any red flags\u00a0on our BaseLine\u00a9 chart<\/em>.<\/p>\n

<Leslie><\/em> But if our changes only have a small effect how can it feel better?<\/p>\n

<Bob><\/em> Because\u00a0some changes have cumulative effects and we omit to measure them.<\/p>\n

<Leslie><\/em> OMG!\u00a0 That makes complete sense!\u00a0 For example, if\u00a0my bank balance is stable my average income and average expenses are balanced over time. So if\u00a0I make a small-but-sustained\u00a0improvement to\u00a0my expenses, like using\u00a0lower cost generic label products, then I will see a cumulative benefit over time to the balance, but not the monthly expenses; because the noise swamps the signal on that chart!<\/p>\n

<Bob><\/em>\u00a0An excellent analogy!<\/p>\n

<Leslie><\/em> So the XmR chart\u00a0is not the tool for this<\/strong> job. And if this is the only tool we have then we risk making a Type II error. Is that correct?<\/p>\n

<Bob><\/em> Yes. We do still use an XmR chart first though, because if there is a big enough and fast enough shift\u00a0then the XmR chart\u00a0will reveal it.\u00a0 If there is not then\u00a0we\u00a0do not give up just yet; we reach for our more sensitive shift detector tool.<\/p>\n

<Leslie><\/em> Which is?<\/p>\n

<Bob><\/em> I will leave you to ponder on that question.\u00a0 You are a trained designer now so it is time to put your designer hat on and first consider the purpose of this new tool, and then\u00a0create the outline\u00a0a fit-for-purpose design.<\/p>\n

<Leslie><\/em> OK, I am on the case!<\/p>\n","protected":false},"excerpt":{"rendered":"

It was the time for Bob and Leslie’s regular Improvement Science\u00a0coaching session. <Leslie> Hi Bob, how are you today? <Bob>\u00a0I am getting over a winter cold but otherwise I am good.\u00a0 And you? <Leslie> I am OK and I\u00a0need to talk something through with you because I suspect you will be able to\u00a0help. <Bob> OK. … <\/p>\n

Continue reading “Type II Error”<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[6,7,8,13,15,24,25,28,36,42,43,45,46],"tags":[],"class_list":["post-4609","post","type-post","status-publish","format-standard","hentry","category-6m-design","category-baseline","category-bobles","category-chimpware","category-design","category-improvementology","category-information","category-metaphors","category-resilient","category-how","category-why","category-what","category-teach"],"_links":{"self":[{"href":"https:\/\/hcse.blog\/index.php?rest_route=\/wp\/v2\/posts\/4609","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/hcse.blog\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/hcse.blog\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/hcse.blog\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/hcse.blog\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=4609"}],"version-history":[{"count":0,"href":"https:\/\/hcse.blog\/index.php?rest_route=\/wp\/v2\/posts\/4609\/revisions"}],"wp:attachment":[{"href":"https:\/\/hcse.blog\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=4609"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/hcse.blog\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=4609"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/hcse.blog\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=4609"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}