What Is Survivorship Bias (And Why It Ruins Backtest Data)
Survivorship bias is simple: you only see the winners. The 5% of EAs that work get published, promoted, and downloaded. The 95% that fail silently disappear. You never see them. You never compare them. You only see the winners and assume they represent the average.
In trading, this is catastrophic. A trader finds an EA on MQL5 with a 47% annual return on the backtest. 500 downloads. Four-star rating. Backtested over 10 years. They download it, run it live, and lose 28% in the first month.
Why? Because that EA wasn't one of the winners. It was number 487 in the version history. The first 486 versions all failed. The developer kept tweaking parameters until one version looked good on a backtest. Then they published it.
The Math Behind 95% Failure Rate
Here's what the data shows:
- 95% of published EAs fail their first 90 days live according to an analysis of MT5 marketplace trends and broker reports. That's not guessing—that's what the data says.
- Of the remaining 5%, most experience a 30-50% drawdown within the first 6 months, even if profitable in backtest.
- The average EA shows 40-60% annual returns on backtest, but 2-8% in live trading when accounting for slippage, spread, and real-world conditions.
Why the gap? Because the backtest was optimized for the past. Not the future.
Why Backtests Lie: Three Killers of Published EAs
1. Curve Fitting (Overfitting)
A developer runs a strategy against 10 years of historical data. They tweak RSI settings, moving average lengths, and entry points. They keep adjusting until it looks perfect. 47% annual return. No single losing month. It looks like a money printer.
What they did was fit the strategy to that specific 10-year period. Not the market. That exact dataset. When live market conditions shift—volatility changes, trend reversals, news events—the EA collapses.
2. Look-Ahead Bias
The EA backtests using perfect information about tomorrow's price action to make today's decision. Your backtest software closes a position at "next bar close" at the exact optimal price. Live trading doesn't work that way. You get slipped 2-5 pips. Your exit is 30 seconds late. By then, the move is gone.
3. Slippage and Spread Disappearance
Backtests assume zero slippage and tight spreads. In reality: live spreads widen during volatile news. Slippage is 1-5 pips on limit orders, 5-20 on market orders. A 40-pip profit target becomes a 30-pip reality. A 50-pip breakeven becomes a -5 pip loser. Survivorship bias ensures you never hear about these failures.
The Hidden Cost of "Free" Published EAs
You know what's expensive? Free EAs.
The trader who downloads a free EA spends:
- 2-5 hours setting it up
- $500-$2,000 in live account losses testing it
- Another 10-20 hours backtesting variations trying to fix it
- 3-6 months waiting to see if it's actually profitable
They lose $1,500 over 100+ hours of work to save $300 on a custom EA. That's not cheap. That's expensive.
And it still doesn't work.
How to Spot a Fake EA vs. One That Actually Works
Real EAs have these markers:
- Honest drawdown disclosure. A real EA says "42% maximum drawdown occurred in March 2023." Not "12% average drawdown." Real is specific and scary. Fake hides it.
- Forward testing proof, not just backtests. Backtests can be faked. A live trading account statement can't. If the developer won't show 60-90 days of live results from a real account, it's untested on real conditions.
- Parameters explained, not hidden. A good EA tells you why the RSI is set to 14 and the TP is 45 pips. A suspicious EA says "optimized parameters" and won't tell you.
- Limitations named explicitly. "Works in trending markets, fails in consolidation." "Requires 4+ hour timeframe." "Struggles in news volatility." Real developers know where their EA breaks. Fake ones claim it works everywhere.
Why Custom EAs Beat the Survivor Bias Trap
Here's what separates a custom EA from Alorny from a published one:
A custom EA is built for YOUR specific strategy and YOUR trading conditions. Not optimized for 10 years of historical data. Not curve-fitted to a lucky streak. Built for what you actually trade.
We backtest it. We forward-test it on a demo account for 30-60 days. We walk you through the live transition so you see the realistic slippage and spread impact before risking real capital. Then we test it again live.
The result: an EA that's honest about its edge, transparent about its failures, and built to survive regime changes instead of just fit the past.
That costs $100-$500 depending on complexity. A month of testing "free" EAs costs more than that and delivers nothing. At Alorny, we've completed 660+ projects on MQL5. Working demo in 45 minutes. Full delivery in hours, not weeks.
Key Takeaways
- 95% of published EAs fail live because survivorship bias hides the failures. You only see the lucky ones.
- Backtests lie by default. Curve fitting, look-ahead bias, and zero-slippage assumptions create fake performance.
- The cost of free is expensive. Months of testing + account losses + wasted hours beats $300 for a custom solution.
- Real EAs show honest drawdowns and live proof, not backtest fantasies. If they won't show a live account statement, it's untested.
- Custom EAs beat survivor bias because they're built for your strategy, not optimized for history.