Bulletproof Problem-Solving (Book Summary)

Bulletproof Problem-Solving by Charles Conn and Robert McLean

Foreword

• 7 Easy Steps to Bulletproof Problem-solving
  • Define the Problem
  • Disaggregate the Issues
  • Prioritize the Issues, Prune the Tree
  • Build a Workplan and Timetable
  • Conduct Critical Analyses
  • Synthesize Findings from the Analysis
  • Prepare a Powerful Communication

Introduction Problem-solving for the Challenges of the Twenty-First Century

• The persona of problem-solving organizations
  • a drive to be working on the right problems
  • addressing root causes
  • engaging teams around short duration work plans
  • allocating responsibilities and timelines with accountability.
  • The World Economic Forum in its Future of Jobs Report placed complex problem-solving at #1 in its top 10 skills for jobs in 2020.
• We see few problems that can ever be solved without disaggregation into component parts.
• Sometimes no amount of regression analysis is a substitute for a well-designed, real-world experiment that allows variables to be controlled and a valid counterfactual examined.

Chapter One Learn the Bulletproof Problem-solving Approach

• Problem-solving means the process of making better decisions on the complicated challenges of personal life, our workplaces, and the policy sphere.
• We encourage you to think of problem-solving as an iterative process rather than a linear one. This cycle can be completed over any timeframe with the information at hand. Once you reach a preliminary end point, you can repeat the process to draw out more insight for deeper understanding.
• We often use the expression, “What’s the one-day answer?” This means we ask our team to have a coherent summary of our best understanding of the problem and a solution path at any point in the project, not just at the end.
• We use logic or issue trees to visualize and disaggregate problems. We employ several types, including hypothesis trees and decision trees.
• Use a hypothesis to bring forth the arguments to either disprove it or support it.

Chapter Two Define the Problem

• Getting problem definition right, including boundaries, is essential to good problem-solving and can be an essential competitive advantage.
• Good problem statements have a number of characteristics. They are:
  • Outcomes focused: A clear statement of the problem to be solved, expressed in outcomes, not activities or intermediate outputs.
  • Specific and measurable wherever possible.
  • Clearly time bound.
  • Designed to explicitly address decision-maker values and boundaries, including the accuracy needed and the scale of aspirations.
  • Structured to allow sufficient scope for creativity and unexpected results — too narrowly scoped problems can artificially constrain solutions.
  • Solved at the highest level possible, meaning for the organization as a whole, not just optimized for a part or a partial solution.
• The problem statement worksheet that captures the whole context:

• Constant iteration allows the team to hone its understanding and therefore to sharpen its strategies to achieve the desired outcome — at the same time keeping all the stakeholders onside as the process runs through time.
• When we worked for McKinsey, we often saw problems that benefited from redefinition to a higher level.
• When possible, it is advantageous to allow flexibility in the scope or width of your problem-solving project.
• Wherever you can, target your problem-solving efforts at the highest level at which you can work, rather than solving for the interests only of smaller units.

Chapter Three Problem Disaggregation and Prioritization

• Any problem of real consequence is too complicated to solve without breaking it down into logical parts that help us understand the drivers or causes of the situation.
• Types of logic trees:

• Trees should have branches that are MECE which stands for “mutually exclusive, collectively exhaustive.”
• Inductive trees show probabilistic relationships, not causal ones.
• Good prioritization in problem-solving makes solutions come faster and with less effort.
• We don’t want to retain elements of the disaggregation that have only a small influence on the problem, or that are difficult or impossible to affect.
• Prioritize problems where both potential scale of impact your ability to influence are high
• Use of constructive challenging and “what you’d have to believe” questions can help get the process out of ruts and foster more creativity in solution paths.

Chapter Four Build a Great Workplan and Team Processes

• The workplan is the place to get specific about your initial hypotheses, clarify what outputs you want from analysis, and assign the parts so that everyone knows what they are doing and by when.
• Best-practice approaches to work planning:
  • We don’t do any analyses that aren’t guided by very clear and testable hypotheses.  We never go off and build a model without a very good idea about what question it answers. There is no vague “I’ll look into X or Y.”
  • We sharpen our thinking even more by requiring that we can visualize what form the output might take (we call this dummying the chart), so we know if we would want it if we had it.
  • We are very careful about the order in which we do analyses. Do knock-out analyses first.
  • We are very specific about who is doing what by when.  No confusion about responsibilities for deadlines.
  • We have workplans to go out only 2-3 weeks, and longer-term study plans to rough out later periods.
• Model workplan:

• Focus your work on the 20 % of the problem that yields 80 % of the benefit.
• Our approach is to do short, but highly specific, workplans that focus on the most important initial analyses, perhaps stretching out two to three weeks, and constantly revise them as new insights come from the team’s work. We couple these with rougher project plans, usually in Gantt chart format, that cover the fixed milestone dates and to ensure the overall project stays on track from a time perspective.
• There are many ways to structure one-day answers, but the classic way employed in McKinsey and elsewhere is to organize them in three parts:
  • A short description of the situation that prevails at the outset of problem-solving. This is the state of affairs that sets up the problem.
  • A set of observations or complications around the situation that creates the tension or dynamic that captures the problem. This is typically what changed, or what went wrong that created the problem.
  • The best idea of the implication or resolution of the problem that you have right now. At the beginning this will be rough and speculative. Later it will be a more and more refined idea that answers the question, “What should we do?”
• What is it that good problem-solving teams do better in their approach to work-planning and analysis?
  • They are hypothesis driven and end-product oriented.
  • They porpoise frequently between the hypothesis and data. They are flexible in the face of new data.
  • They look for breakthrough thinking rather than incremental improvements.
• The most important biases to address are:
  • Confirmation bias: Confirmation bias is falling in love with your one-day answer.
  • Anchoring bias: Anchoring bias is the mistaken mental attachment to an initial data range or data pattern that colors your subsequent understanding of the problem.
  • Loss aversion: Loss aversion, and its relatives, the sunk cost fallacy, book loss fear, and the endowment effect, are a failure to ignore costs already spent (sunk) or any asymmetric valuing of losses and gains.
  • Availability bias: Availability bias is use of an existing mental map because it is readily at hand, rather than developing a new model for a new problem, or just being influenced by more recent facts or events.
  • Over-optimism: Over-optimism comes in several forms including overconfidence, illusion of control or simply failure to contemplate disaster outcomes.
• To lessen the impacts of cognitive biases:
  • Diversity in team members
  • Always try multiple trees / cleaves:
  • Try adding question marks to your hypotheses.
• Employ Team Brainstorming Practices:
  • Obligation to dissent
  • Role playing – Try acting out your interim solutions from the perspective of clients, suppliers, other family members, citizens … whoever isn’t you.
  • Dialectic standard – every idea or hypothesis must be met with its antithesis and challenged, before joining the learning together in synthesis.
  • Perspective taking – Perspective taking is the act of modeling another team member’s assertion or belief (especially if you don’t agree) to the point that you can describe it as compellingly as the other.
  • Constructive confrontation – To disagree without being disagreeable. One of the great tools we both used in McKinsey is “What would you have to believe? “to accept a particular thesis or viewpoint.
  • Team distributed voting – One approach we have used is to assign each team member 10 votes; the most senior person votes last
  • Solicit outside views (but be careful with experts). The normal thing is to interview experts — but the risk is that they just reinforce the dominant or mainstream view and therefore smother creativity. Try talking to customers, suppliers, or better yet players in a different but related industry or space.
  • Explicit Downside Scenario Modeling and Pre-Mortem Analysis.
  • Broaden your data sources: The best problem solvers that reflects an active openness to new ideas and data, and a suspicion of standard or conventional answers

Chapter Five Conduct Analyses

• Don’t jump right into building giant models until they have a clear understanding of whether and where complex tools are required.
• Smart analysis starts with heuristics and summary statistics to assess the magnitude and direction of the key problem levers.
  

• We see many common errors that relate to the distribution of outcomes. These include placing too much emphasis on the mean outcome, typically called the base case, and insufficient weight on outcomes that are one or even two standard deviations from the mean in a normal distribution.
• The Sherlock Holmes approach of painting a picture of the problem by asking who, what, where, when, how, and why is a powerful root-cause tool to quickly focus problem-solving. Ask “Why? five times.

Chapter Six Big Guns of Analysis

• Bayesian statistics, regression analysis, Monte Carlo simulation, randomized controlled experiments, machine learning, game theory, or crowd-sourced solutions
• First-cut data analysis often points to direction of causality and size of impact, which are critical to evaluating the results of complex models later.
• Data-fishing expeditions or unfocused analysis that “boil the ocean” are likely to result in inefficient problem-solving.
• Our preferred sequence, not surprisingly, is to start with clearly defining the problem and forming initial hypotheses. Then get to know your data by looking at the mean, median, and mode, as well as other summary statistics.
• One answer is the natural experiment, also called a quasi-experiment: If you can’t run an experiment yourself, look to see if the world has already run it — or something like it — for you.

Chapter Seven Synthesize Results and Tell a Great Story

• Done right, your conclusions are an engaging story, supported with facts, analyses, and arguments that convince your audience of the merits of your recommended path.
• Our recommended process is iterative at each stage and driven by the interaction of the strong hypotheses of your one-day answers with the analyses of your workplan.
• Where possible, the most powerful visualization is to show each graphic as branches on your revised tree structure.
• Return to your problem definition worksheet and remind yourself:
  • What problem are we trying to solve? How has this evolved?
  • What are the key criteria for success that our decision maker (which may be yourself) set out in advance? It is important to reflect these explicitly in your story.
  • Did you honor the boundaries of the problem set by the decision maker? If not — which may be for good reasons around creativity or deciding to relax a constraint to open up new possibilities — you need to make the case here.
• Sometimes it is best to carefully lead the audience from situation to observation to resolution, which are your recommended actions. But our bias in most circumstances is to lead by answering the question “What Should I Do?” and then summarize the situation and key observations that support action.
• it was not one that the local management team wanted to hear. In circumstances like this, it can make sense to use a revealed approach to your arguments,

Chapter Eight Problem-solving with Long Time Frames and High Uncertainty

Chapter Nine Wicked Problems

• These problems typically involve multiple causes, major values disagreements among stakeholders, unintended consequences, or substantial behavior change in order for the problem to be solved. Terrorism, environmental degradation, and poverty are often proffered as examples of wicked problems.

Chapter Ten Becoming a Great Problem Solver

• Great problem-solving consists of:
  • good questions that become sharp hypotheses
  • a logical approach to framing and disaggregating issues
  • strict prioritization to save time
  • solid team processes to foster creativity and fight bias
  • smart analytics that start with heuristics and move to the right big guns
  • a commitment to synthesize findings and turn them into a story that galvanizes action.
• Take the time up front to really understand your problem.
• Get started with nothing more than a problem statement.
• Try several cuts at the tree.
• Use a team wherever you can.
• Make the right investment in a good work-plan.
• Start your analysis with summary statistics, heuristics, and rules of thumb to get a feel for the data and the solution space.
• Don’t be afraid to employ big analytic guns when required.
• Treat the seven-steps process like an accordion.
• Don’t be intimidated by any problem you face.