When Can You Realistically Expect to Outperform the Average?
I tend to feel unconfident in my opinions. Partly, it's because I might look bad if my views are wrong. Another reason is that most of the time, the people in the room share similar views to mine. I feel I have little to add. What makes me special compared to the collective intelligence of humankind? At best, I am just average. Also, being born into a Chinese family, I have been taught to be modest—don't feel satisfied, there is always room for improvement. Combining all these factors, I am conservative in my opinions and tend to start with an average estimation of my performance in many aspects of life. Is there an objective way to predict the probability of outperforming the average? This seems to be an important question as it affects life choices.
To answer this question, we must first ask: with all of humankind's collective intelligence and evolution, why is the world still so imperfect and inefficient? First, decision makers are not always the beneficiaries. For example, as a developer, you have little incentive to modify existing problematic code if it works. Developers tend to add features to existing codebases instead of refactoring them. After all, what's the point of solving predecessors' mess if my feature can be added? Second, there is information asymmetry. Not everyone shares the same information because transmission incurs costs—whether it's the simple cost of sending (like in the pre-Internet age) or the opportunity cost where sharing information may reduce competitive advantages. Third, it's hard to coordinate change (the system is stuck in a bad Nash equilibrium). Often, change requires effort from multiple stakeholders with different incentives and priorities. Any party alone cannot improve the system and may even hurt themselves if they try while others don't. The classic example is the Prisoner's Dilemma, where both prisoners betray each other even though the best collective outcome would be for both to cooperate.
Given this inefficiency and inadequacy, how can we realistically expect to outperform? It remains rare because not all inefficiencies are exploitable—a $20 bill on the subway platform exists because it may be hard to retrieve—and not all inadequacies, like Nash equilibrium problems, are fixable. I believe that imperfect things in life exist for reasons. The first step is understanding why inefficiencies and inadequacies persist. Then we can estimate whether we have a reasonable chance to solve them. If something is valuable and easy to exploit or fix, someone should have already done it. However, whether an imperfection is valuable or solvable is relative. Sometimes a problem means nothing to the world but is crucial to you. Consider a market or customer segment that's unimportant to a big company but makes perfect sense for a smaller one—that's how startups find serviceable addressable markets. Ultimately, you must consider whether the world, or whichever group you're comparing yourself to, has devoted enough attention and consideration to the problem you're trying to solve. Your boss may spend only a few minutes thinking about a problem you've contemplated for a month.
Focus and attention enable you to outperform. You need to identify winnable battles.
Further readings:
