May 29, 2011

ASQ Future - Baye Conditional Probability Risk Human Errors and Ducks

About three years ago I attended one of the ACTKM annual Knowledge Management Conferences. The 2008 conference covered a huge range of items, including Bayesian Networks. I'd parked that in the bottom of the memory banks to be honest - until I encountered a PEX article on Baye by Phil Mercy who does Six Sigma consulting with Motorola.

I started to reflect on how Baye's approach could be seen through a lens of Risk, Complexity & Human Errors - key issues in the Future of Quality - a subject raised by ASQ's Paul Borawski (see his recent post).  And so follows the output of my learning journey into this maze ...

There's a lot of maths in Bayesian analysis - however Phil's explanatory style did help me begin to deal with it : 
Phil starts out with catching birds in a trap and determining if they are ducks or not :
"If it walks like a duck and quacks like a duck, is it possible it's not a duck?  ... and examines this oft-quoted maxim and describes how Bayes’ theorem of conditional probability makes raw data useful for making business decisions.

“If it walks like a duck, and quacks like a duck, then it’s a duck.”
This oft quoted maxim is intuitively ‘Correct’ and accurately describes the human reasoning process. Evidence about an object, in this case whether it waddles or quacks, is used to help determine the nature of that object, i.e. whether it’s a duck or not. When the weight of evidence builds up in favour of any single outcome, then a human will deduce that this result is the correct one. “If it walks like a duck, and quacks like a duck, then it’s a duck”.
This seems ad hoc, and not analytically sound, but in practice this method works really well to guide our day to day decisions.

After all, when it comes down to it, we’re all in the business of turning raw data into a correct decision for our business.
Luckily for us analytical types, there is a Mathematical formalism for this technique:  Bayes’ theorem of conditional probability.
The probability of an event A occurring is changed if we know something about a related event B

P(A|B) = P(B|A).P(A)/P(B)
… and in English - The probability of A, given that B has occurred, is the probability of B, given that A has occurred, times the probability of A, all over the probability of B
If we know that event A normally occurs when event B has already occurred, then knowing something about B may well change your view of A. For complex systems with multiple events A,B,C … etc.  being considered, a Bayesian Belief Network is often used to model the likelihood of an outcome. You’ll find Bayes used in a number of high technology areas such as complex risk analysis, data mining, machine data learning, artificial intelligence and language recognition.
He then goes on to describing determing the likelihood of a bird being a duck, or not, with the use of a Quackometer & Waddleometer ...  By using both types of evidence we’ve improved our success rate and now only 5/100 decisions are wrong.
Of course such reasoning may have been applied to the likelihood of their being Black Swans.ie as in European thinking several centuries ago  when they referred to impossible events "as being as likely as a Black Swan". All such reasoning tipped on its head when European explorers came to Australia as explained by Nasim Taleb in his book on Black Swans & how resilient organizations prepare for the seemingly impossible.
It did seem to me that the Bayes Concepts could be useful for analysing probability,  risk & consequences in Decisionmaking. It aligned with questions by ASQ's Paul Borawski in his A View from Q blog post on Quality & Disasters.

Consider an analysis in Japan of risk of a bad event affecting the population & economy :

  • P(E) :  Probability of serious Earthquake
  • P(T) : Probability of serious Tsunami
  • P(NE) : Probability of serious Nuclear Event
if we reviewed each of these independently then perhaps the probability might be low


but if we collectively factor in other key aspects, then we might alter our estimation of the risk of such a serious event occurring :
  • P(EqT) : Probability that our understanding of Earthquake risk is inaccurate &/or inadequate
  • P(Econ Eng) : Probability of using an Economic Approach to Engineering Design &/or Construction= less robust design
  • P (Project Cost Drivers) : Probability of Project Cost Drivers being weighted higher than robust Engineering Design risk aspects
  • P(Op Mgmt) : Probability that equipment has not been operated, maintained &/or safety/training done correctly
  • P(Eng) : Probability of Engineers incapable of controlling serious Nuclear Event
  • P(Rc) : Probability of Risk Changes if initial understanding of design, operational maintenance, safety circumstances has altered or if any of these have actually been altered

So even if we did all of the above analysis and it suggested a serious risk of generating a situation which is unlikely to be  tolerable - do we dismiss this as pessimistic engineering reasoning then optimistically run this through the lens of Evidence Based Reasoning - ie which runs as something like : 
 as we haven't seen any problems in so many years in recent memory, so
 therefore we reckon it will be okay &
we'll deal with the consequences if & when they ever arise ?
And then over time, when such cataclysmic events have not occurred, do we begin to believe that the approaches taken years before to deal with the risk prevention & response are okay,
even though they not been tested in fact
ie  as such serious situations have not actually arisen to verify that they are robust enough
so we delude ourselves  into believing that these measures have worked so far & thus have made make us invulnerable ? And so we do not review the analysis on which all this is based ?  sort of like Churchill's belief in Fortress Singapore ?

And  is it then still considered as unlikely as 17th Century Europeans thought Black Swans ? Hard to make sense of ? So we do not even prepare for responding to such serious scenarios ? And if we had prepared - would it mean a huge number of incredibly long & complex procedures that go largely unread ? And what of the failure to recognise the magnitude of the cataclysmic events even as they are beginning to unfold? What seems to me too often to be the Myers Briggs Type ISTJ or ESTJ factor among some technocrats & government bureaucrats ? Reassure at all costs that everything is in control - New Scientist 9 April 2011 p 10.
And then even if you did recognise the risk of this calamity and responded - what if there were other risk situations in your organization or community - and to deal with preventing the calamity, you in fact effectively starved those other risks of the organizational resources & managerial attention that they needed to be adequately addressed ? Even worse if there has been a culture of assuring management that these other risk areas do exist but are effectively under control - sort of Emperor's New Clothes Syndrome ?

You could probably swap the above earthquake scenario with any of the following ?
  • GFC - previous erroneous view that economic world consists largely of simple independent transaction markets - New Scientist October 28 2008
  • 2004 Tsunami - our understanding of subduction earthquake behaviour is evolving - New Scientist 23 April 2011 p6
  • 2011 Christchurch Earthquake - was not previously identified as an earthquake zone - New Scientist 26 Feb 2011 p 4
  • 2010 BP Gulf Disaster
  • 2010 Toyota Recall Crisis
  • 2011 - Queensland Floods - Wivenhoe Dam - Inquiry told that predicted rainfall events not factored into Dam Operations - even in Flood ? No major review of Operations Manual since 1985 ?
  • 2009 Victorian Bushfires -Inquiry Report -  Planning & Response deficiencies
  • September 11 2001
So how do we help people to deal with such risks when the maths to derive their likelihood is so abstract & complex ? What does it mean for those in Quality Management - the guardians of standard procedures  & records management ? What of Knowledge Management - capturing & sharing lessons learned & making them findable/comprehensible/embeddable ? Storytelling - however recognising that albeit currently popular in many KM circles,  Storytelling methodology on its own, is never going to be a sufficient panacea ? Especially when you read a review on complexity in our world eg Braden R Allenby & Daniel Sarawitz on "We've made a World We Cannot Control" - New Scientist - 14 May 2011 p 28



And developing people, communities & organizations to proactively & reactively face complexity with resilience rather than denial ?


Simplifying where it is possible ?


I've seen Professor James Reason's Human Errors  Swiss Cheese Model on multifactors  in causing serious events and perhaps it is a mental model to deal with complexity ? refer DSTO report ' A Review of Accident Modelling Approaches for Complex Critical Sociotechnical Systems"
And perhaps by moving into the Realms of Complexity Thinking ? People such as Bruce Waltuck aka @ Complexified (see his Future of Quality post) and Nick Milton are seeing the overlaps of complexity, quality & information/knowledge transfer - management ...

ASQ's Paul Borawski recently challenged us as ASQ Global Influential Voices for Quality to reflect on What is the Future of Quality?
Is this perhaps the future direction for quality management a future ISO 9001:2015  - to be more upfront about more aggressively & openly addressing Risk ?

May 19, 2011

ASQ WCQI11 World Conference on Quality Improvement 2011 Day 3 May 18 2011

Thoughts shared from Day 3 #WCQI11 :

  • Updating ISO 19011 on Auditing
  • Engaging your employees & team in quality
  • Understanding the Why's of Human Errors - impacts on Procedures / IT Hardware
  • Change
  • Supply Chain Management processes

ASQ 7:00am via HootSuite

Missing #WCQI11? What was your favorite#WCQI11 experience? What did you learn that you will take back?

ASQ6:46am via HootSuite

RT @thebigqbyjuran: At the BIG Q BLOG, video recap of keynote speakers from this year's @ASQ #WCQI11 http://www.juran.com/blog/?p=255

ASQ2:45am via HootSuite

Great blog--glad you enjoyed #WCQI11! RT @dlbrecken#WCQI11 Love the conference. Check out my blog at http://t.co/AC2v9RP


psuahr1:43am via TweetCaster

You'll never accomplish it as a team unless you engage your team. #WCQI11 Don't give up on your dreams but take your employees with you.

psuahr1:41am via TweetCaster

Managers - how pitiful we must feel about ourselves if we have to step on the people who work for us to feel good about ourselves.#WCQI11


QualityBob1:33am via Twitter for iPhone

#WCQI11 Bryant echoes what I've said for years... Apply Lean first then Six Sigma. Lean gets rid of junk; 6s improves what's left

psuahr1:33am via TweetCaster

Apply lean first then six sigma to achieve significant gains #WCQI11


ASQ_FW14161:25am via Mobile Web

Bryant: Don't give up - you have to crawl before you walk or run #WCQI11 #ASQ


psuahr1:25am via TweetCaster

Don't give up before you cross the finish line.#WCQI11

RT @psuahr: More teams! Lots of noise... Raise the voice of Quality! #WCQI11http://lockerz.com/s/102682251


ASQ_John1:23am via TweetCaster

RT @psuahr: If it doesn't mean anything to you they're going to see right through it. #WCQI11

psuahr1:23am via TweetCaster

Send the employee an e-mail and copy their management to praise them - build their self esteem - engage them. #WCQI11


ASQ_FW14161:13am via Mobile Web

Bryant: Overcoming resistance to change - work with the ones who will work with you.#WCQI11 #ASQ


psuahr1:11am via Mobile Web

RT @ASQ_FW1416: Bryant: build passion...set std, partner w clients, listen...want engaged clients #WCQI11 #ASQ

psuahr1:11am via TweetCaster

The first step towards change is the hardest.#WCQI11


psuahr1:08am via TweetCaster

How do you build passion - set the standard for others to follow #WCQI11

psuahr1:07am via TweetCaster

Find the things that are going well and replicate them. #WCQI11

ASQ_FW14161:06am via Mobile Web

Bryant: "You only hear one complaint out of 10"#WCQI11 #ASQ


ASQ_FW14161:04am via Mobile Web

Bryant: "Many times we reach success at the expense of our employees" #WCQI11 #ASQ


psuahr1:04am via TweetCaster

Improving employee/client engagement will yield $B #WCQI11

ASQBuffalo1:04am via Twitter for iPad

Give employees tools & equipment they need good work happens. #WCQI11


psuahr1:03am via TweetCaster

If you put your employees first, the employees will take care of the clients. #WCQI11

psuahr1:02am via TweetCaster

If we truly engage your employees, you're going to see it in the output. #WCQI11

QualityBob1:02am via Twitter for iPhone

Ron Bryant #WCQI11 - Hard times do not dictate the outcome - you do.#employee_engagement

ASQBuffalo1:02am via Twitter for iPad

Empoyee engagement increases output...duh#WCQI11


ASQBuffalo12:59am via TweetCaster

RT @psuahr: More teams! Lots of noise... Raise the voice of Quality! #WCQI11http://lockerz.com/s/102682251

psuahr12:58am via TweetCaster

Applying engagement to your quality programs#WCQI11 http://lockerz.com/s/102684078


ASQ_FW141612:44am via Mobile Web

@complexified Top 3 takeaways-Single Point Lessons, Dealing with Resistance, Standard Work 4 Leaders #WCQI11 #ASQ

ASQ12:35am via HootSuite

New video--highlights from #WCQI11 keynote speakers--Barbara Corcoran, J.J. Irani, Bennie Fowler and Brian Joiner: http://ow.ly/4Xkpk


ASQ12:24am via HootSuite

@ASQ Influential Voice @dblevy0925 is covering #WCQI11 on his blog, David on Quality. Great read! http://ow.ly/4XlUm


psuahr12:12am via TweetCaster

Must have robust requirements when safety is critical. #WCQI11

qadvocate12:11am via Mobile Web

Unknown - will #ASQ revise CQA to address expected new auditor competence section in ISO19011? #WCQI11


qadvocate12:09am via Mobile Web

Johnson: ISO19011 is intended as a general guideline to audit mgmt systems, not for certification (ISO 17021 applies) #WCQI11#ASQ


psuahr12:03am via TweetCaster

Http://www.iaqg.sae.org/scmh supply chain management handbook - it's free! #WCQI11


psuahrMay 18, 11:57pm via TweetCaster

Don't leave it up to the auditor to define your processes. #WCQI11

qadvocateMay 18, 11:57pm via Mobile Web

Johnson: Clause 5 will have significant changes due to expansion to cover other mgmt systems. #WCQI11 #ASQ


qadvocateMay 18, 11:39pm via Mobile Web

In final session at #WCQI11 - W19 on ISO19011 audit standard update #ASQ


qadvocateMay 18, 11:35pm via Mobile Web

@complexified most meaningful idea rcvd at#WCQI11 -Read's SCARF model for dealing w resistance. I'll apply in interactions w sr mgmt#ASQ

ASQBuffaloMay 18, 11:31pm via HootSuite

RT @ASQ: Cool! Great idea. @SteelyQueen I am curating conf tweets into my Sharepoint blog to share with my org in Australia #WCQI11thks guys

ASQMay 18, 11:22pm via HootSuite

@ASQBuffalo Thanks for live tweeting#WCQI11! Looking forward to your blog post.

Use the knowledge of human error causal factors to avoid failures at the handoff stage#WCQI11


ASQBuffaloMay 18, 10:50pm via Twitter for iPad

Faulty procedures are created by human error#WCQI11

ASQBuffaloMay 18, 10:47pm via Twitter for iPad

Errors in hardware almost always human error in the management of the hardware designers of processes #WCQI11

dblevy0925May 18, 10:46pm via Twitter for BlackBerry®

Ben Marguglio delivers presentation on Human Error Causal Factors with passion and gusto #asq #wcqi11


ASQBuffaloMay 18, 10:45pm via Twitter for iPad

When do don't conceive of the need that is cognitive based human error #WCQI11


ASQBuffaloMay 18, 10:31pm via Twitter for iPad

Ben marguglio is amazing in #WCQI11 @ASQhuman error causal factors session!

5SsupplyMay 18, 10:24pm via Twitter for iPhone

Humana uses lunch & learn, social media and distance learning/technology to get employees engaged in #lean #ASQ #WCQI11

ASQBuffaloMay 18, 10:20pm via Twitter for iPad

Value Baedeker error based on behavior when the individual does not accept the requirement expecation or need #WCQI11

ASQBuffaloMay 18, 10:18pm via Twitter for iPad

Cognition- base error absent the ability to process the knowledge (memorize, understand, apply, analyze, synthesize, or evaluate) #WCQI11

ASQBuffaloMay 18, 10:17pm via Twitter for iPad

Knowledge based error is error base on behavior absent the reciept of the knowledge of the requirement,expectation or need #WCQI11

ASQBuffaloMay 18, 10:15pm via Twitter for iPad

Human error causal factor taxonomies. Rule-based. Knowledge-based. Skill-based#WCQI11

ASQBuffaloMay 18, 10:12pm via Twitter for iPad

Human error is NOT malicious compliance, malicious behavior, good decision with bad outcomes #WCQI11

ASQBuffaloMay 18, 10:11pm via Twitter for iPad

Human error is behavior that is expected to create a result and does not #WCQI11

ASQBuffaloMay 18, 10:11pm via Twitter for iPad

Understand errors that occur in the human creation of plans #WCQI11

ASQBuffaloMay 18, 10:09pm via Twitter for iPad

What is human error? 7 universally applicable human error causal factors coming soon.#WCQI11

ASQBuffaloMay 18, 10:07pm via Twitter for iPad

Coaching is to reinforce good practices#WCQI11

ASQBuffaloMay 18, 10:06pm via Twitter for iPad

Can't eliminate human hazards we must develop barriers to mitigate + ameliorate it.#WCQI11

 

Posted via email from kerrieannesfridgemagnets's posterous