In the last several years, the use of statistics has come under attack most likely due to presidential polling in the last two U.S. elections (regardless of political persuasion). While I’m sad as a statistician to see this, the good news is that sellers who grasp a few basics can zig while others zag.
In this post, I’m going to teach you how to compare two proportions (i.e. two percentages).
Let’s take a set of findings from the HubSpot Research Global Sales Enablement Survey published in October 2020. Check out the very provocative figure below.
According to the figure, outperforming companies focused on pipeline velocity and on competitive & market data. In contrast, underperforming companies focused on rep activity data, performance against quota, cross/upsell, forecasting, and ABM. What the fcuk?#%
Well, just because it does not make sense does not necessarily mean it is wrong. However, the authors of the study do have a duty to publish findings that are actually statistically valid. How should they have assessed validity? How can you, a normal B2B sales professional with maybe one stats class assess validity? Well, I show you how now.
Let’s take the first set of columns – 46% of outperforming companies focus on rep activity data while 52% of underperforming companies focus on rep activity data. Is the 46% statistically different than the 52%? Well, the answer is that it depends on sample sizes. If one sampled a million people, then yes. Let’s look at HubSpot’s sample.
The report says that there were “over 500” survey responses. I’ll use 500 here since I know that not every respondent answers every question so 500 might even more above the sample size (“n”) for this question.
HubSpot showed that 38% of respondents reported overperforming against revenue goals this year and 40% underperformed. That means 190 (=500 x 38%) overperformed and 200 underperformed.
(As an important side note, I imagine that whether or not a company outperformed or underperformed in the last year has almost everything to do with an external factor – COVID – and rather little to do with internal factors. But, we will ignore that little detail for now.)
So, 46% of 190 overperforming companies, or 87, focused on rep activity data. 52% of 200 underperforming companies, or 104 focused on rep activity.
There is a somewhat intimidating formula for comparing two proportions that looks like this:
But, we don’t need to use that since this is the 21-st century and there is a site for everything on the Internet. We will use this calculator where anyone (yes, even math-phobic people) can get quick answers to complex questions.
Here are the results of inputting the #s we just got:
OK, so how do we interpret this? Just look at the red text. You can see that comparing 46% (87 of 190 overperforming companies) to 52% (104 of 200 underperforming companies) is not significant. In other words, with this sample size, one cannot conclude there is a difference between 46% and 52! In even more other words, it is incorrect and untrue to say that underperforming companies focused more on rep activity than overperforming ones.
With a z-test and a standard 95% confidence interval, any time the p-value is greater than 0.05, we say the results are non-significant. Here the p-value was 0.2187 for the rep activity proportion comparison.
What about all the other data in chart? Here is a summary:
| Test | Over | Under | p-value | Significant? |
| Rep activity data | 46% | 52% | 0.22 | no |
| Pipeline velocity | 31% | 23% | 0.08 | marginal |
| Competitive & market data | 44% | 34% | 0.04 | yes |
| Performance against quota | 38% | 56% | 0.00 | yes |
| cross-upsell | 30% | 42% | 0.01 | yes |
| Forecasting | 47% | 56% | 0.08 | marginal |
| ABM | 30% | 33% | 0.52 | no |
So, what can we really conclude from this survey (again, ignoring the fact that the researchers did not control for the impact of COVID).
- Over-performing companies focused disproportionately on competitive & market data.
- Under-performing companies focused disproportionately on performance against quota and on cross/upsell.
- There is only marginal evidence that under-performing companies focused more on forecasting and that over-performing companies focused more on pipeline velocity. To confirm, they should pursue a second, larger study.
- There is ZERO evidence to say anything one way or the other about whether a company outperforms or underperforms by focusing more/less on rep activity data or ABM.
My hypothesis is that, adjusting for COVID, the highest performing companies focus on leading indictors, especially activity, measures of effectiveness, and customer health.
Hope that helps.