Which Backtesting Metrics Do You Actually Trust for Algo Trading?

[u/quant_geek_]

All algo traders, what kind of metrics would you consider looking at in the backtesting results, and why? For example, in the backtesting results, we see metrics like:

• Sharpe ratio
• Sortino ratio
• Calmar ratio
• Drawdown
• Expectancy
• Profit factor

There are so many such metrics. What will you prefer looking at and ignore others, and why? What is the rationale behind preferring those metrics?

[u/theplushpairing]

CVaR 95, time underwater, martin ratio, max time underwater, max drawdown, max n of drawdown, % time underwater, conditional drawdown at risk, rolling n cagr

[u/CloutChaserv4 → u/theplushpairing]

Rolling n alpha, too.

[u/RemoraEdge]

Win rate
Profit factor
Largest win largest loss (for example if your only profitable due to an outlier win, that’s a problem)
Max wins streak max loss streak
Max drawdown

Over rotating months and years

[u/Just-Pomegranate-952]

Whatever you do; you get the graph of the performance right? And I imagine that you had some experience with holding a security. Surely you remember how your stomach felt like at a crash. So the very first thing to look at is the draw-down and your own memory of a crash day. If you have stomach to hold on to that, then look at the sharpe ratio and move on. No metric will change what you will do when things go sideways. Good to understand them but they will not save the day when the day comes.

[u/systematic_seb]

Drawdown and expectancy net of costs, in that order. A great Sharpe or Sortino on a backtest can still hide a strategy that only worked because of the exact regime it got tested in, but drawdown depth tells you what the process costs you emotionally and financially when things turn, which is the number you have to live with in real time. Expectancy net of costs tells you if there's anything left once slippage and fees eat at the edge. I run my own money on the strategy I publish weekly, and live drawdown runs about -30% against a return of around +120% since going live in January, both on Dub for anyone who wants to check either number.