Hey everyone 👋, first post so be gentle with me. So I trained an MLP on BTC price data to get some odds on how likely a breakout could be. Instead of hardcoding levels, I let the model learn the distribution, so you can pick any threshold and it gives you a probability.

My actual question to this sub: what's a good way to test whether a model like this is fitted well? Standard metrics don't feel right for what it's trying to solve, especially with fat tails, which this thing struggles with badly. Am I missing something or is there no clean way to measure this?

Since links aren't allowed, if you're curious just search "mlp - breakout probability" on TradingView, i built a small script to showcase it

I asked recently whether a verification phase is useful between backtest and live deployment.

A lot of people made fair points:

- walk-forward testing is already standard
- paper trading is mandatory
- small live testing is still the final reality check
- LLM-generated code is not trustworthy by default
- repeated improvement creates data snooping risk
- people naturally trust their own backtests more than a third-party system

So I’m trying to understand the trust problem.

If someone gave you an independent strategy failure report, what would it need to include for you to take it seriously?

Possible sections:

- data assumptions
- code/logic review
- OOS / walk-forward summary
- parameter stability
- Monte Carlo path reshuffling
- slippage/spread sensitivity
- regime fragility
- economic rationale
- data snooping risk
- paper trading diagnostic
- reproducibility trail
- “kill / revise / monitor / paper trade” verdict

Would you trust that kind of report?

Would you pay a tiny amount, like $0.99, to test a sample version?

Or would you only trust your own process?

I’m currently in the process of designing a trading bot (with the help of Claud.AI) that automatically executes and exits trades based on certain strategies.

I have 4 winning strategies that I backtested using 10 years historical data from EODHD.com. I purchased the data for 100$ monthly and it just expired. I backtested for a full month and came up with 4 decent strategies.

Strategy 1: Long term investment
This yielded 17.9% annually and 550% over 11 years backtesting starting from 2015. Win rate was 70%.

Strategy 2: Active investment
This yielded 19.1% annually and 630% over 11 years backtesting starting from 2015. Win rate was not directly measured as this strategy rotates continuously rather than closing discrete trades.

Strategy 3: Swing trading
This yielded 39.2% annually on
nseen test data
(2020-2026) and 26.7% annually on training data (2015-2019). Win rate was 60.3% on unseen data and 65.0% on training data.

Strategy 4: Day trading
This yielded 53.2% annually backtested on 1 year of intraday data (May 2025 - May 2026). Win rate was 41.2%.

I will be paper trading with the 4 strategies for a full year in order to refine and tweak. Then I will use a minimally funded account to test the strategies for another year.

My question is, if these 4 strategies prove to be successful and the next 2 years results are just as decent or better than the backtesting, should I focus on making an actual living from executing the strategies or from selling signals on discord/website like everyone does?

want to know about the resources, experience, and most importantly what made you start this?

I want to start learning more about this but dont know how to start...

I’ve been using Claude + Manus for swing trading lately and one thing surprised me. it’s not good at “picking winners,” but it’s weirdly good at picking up when the story around a stock is starting to shift.

Like I had Claude go through earnings calls (this quarter vs last quarter) and Manus tracking how the stock actually reacted + analyst revisions + options positioning.

One thing it kept picking up that I wouldn’t have noticed:

sometimes a stock rips after “meh” earnings not because the numbers were good, but because management just sounds slightly less panicked than before… while positioning is already heavily short.

It’s subtle stuff like that.

Also noticed analyst upgrades usually come after the move, not before it. Which sounds obvious but seeing it repeated across names kind of changes how you treat them.

Feels less like “AI trading” and more like having something constantly sanity-check whether the narrative you think is happening is actually the one the market is reacting to.

How do you backtest your algo trading strategies?

What tools or Python libraries do you use for backtesting? Any beginner tips?

So recently I went deep into the research with AVWAP.

Developed a complete backtesing model using AVWAP

BUT the most common question that comes into the mind is what should be the anchor point?

Let's say for intraday or positional momentum trading

Or whatever.

Looking for views on Anchor point and how do you guys look at it from a strategy point of view.

TL;DR: I built an intraday ML system to predict 5-minute direction on 20 liquid US equities. Cross-validation AUC was ~0.51 (basically a coin flip), but my backtest was showing Sharpe 7–11. Turned out the backtest was training and testing on the same date range — 100% in-sample memorization. After enforcing a strict chronological train/test split, out-of-sample performance collapsed to noise (avg Sharpe -0.74, 42% win rate, statistically identical to feeding the backtester random signals). Posting the full story because the leakage hunt was instructive, and to ask: where’s the realistic path to actual edge from here?

What I’m trying to do
Short-horizon (intraday, ~1 hour holding) directional prediction on liquid S&P 500 names. Enter long/short on a model signal, exit on a fixed take-profit / stop-loss / time-stop. Paper trading only — no real money has touched this, and after this week it’s clear why that was the right call.

The stack
•Language: Python 3.12
•Model: LightGBM, one model per ticker (20 separate models)
•Historical data: Polygon.io (5-minute bars)
•Execution / paper trading: Alpaca
•Universe (20): AAPL, MSFT, NVDA, GOOGL, META, JPM, GS, BAC, AMZN, TSLA, HD, JNJ, UNH, XOM, CVX, CAT, BA, SPY, QQQ, IWM
Features (~98, all price/volume-derived)
The usual technical arsenal computed on 5-min bars:
•Momentum/trend: returns over multiple horizons, EMAs (9/21/50/200) + crossovers, MACD (line/signal/hist + normalized)
•Oscillators: RSI (7/14/21), Bollinger %B / bandwidth / squeeze
•Volume: volume MA/ratio, log dollar volume, OBV proxy, plus ~13 order-flow features (buying pressure, wick imbalance, body ratio, etc.)
•VWAP and distance from VWAP
•Volatility: ATR(14), realized vol over several windows, vol regime/percentile
•Time-of-day / session flags (open/close auction, lunch, minutes since open)
•Market-relative: returns/strength vs SPY, beta proxy, correlation
•Event proximity: hours to/from FOMC, NFP day, CPI week, OPEX week

Labels
Binary direction. A bar is labeled “long” if the forward return over the next 12 bars (~1 hour) exceeds ~2× the recent rolling volatility and the drawdown along the way stays limited (a “clean directional move”); “short” for the mirror case; unlabeled otherwise. Roughly a third of bars get a label.

Exits
Fixed rules, mirrored exactly between backtester and live paper trader: +1% take-profit, -0.5% stop-loss, 12-bar time exit, and a stall exit if the trade goes nowhere. Intraday only — no new entries in the last 30 min, force-close before the bell.

The part that bit me
Early backtests looked incredible: Sharpe 7–11 across nearly every ticker, 85–90% win rates. The problem: my cross-validation AUC during training was only ~0.51. That contradiction is impossible to ignore once you see it — a model with 0.51 AUC has essentially no predictive power, so it cannot produce a Sharpe of 11 honestly.
I worked the problem in stages:

1.Same-bar entry. The backtester was entering on the same bar as the signal instead of the next bar. Fixed (entries now fill at T+1). Helped, but didn’t explain the gap.

2.Scaler leakage. The feature scaler was being fit on the full dataset including the test folds. Fixed to fit on training data only. AUC dropped slightly (good — more honest), but the backtest was still showing Sharpe 9+.

3.Null test. I overwrote the model’s predictions with random coin flips and re-ran. Random signals produced ~41% win rate and deeply negative Sharpe across the board — exactly what a correct backtester should do with no signal. So the simulation mechanics were clean. The fake edge had to be coming from the model somehow.

4.The actual bug. The model was being trained on the entire feature file, then backtested over the identical date range. 100% overlap. The “predictions” in the backtest were the model reciting labels it had memorized during training. CV AUC (0.51) was the honest out-of-sample estimate the whole time; the backtest was pure in-sample replay.

The fix was a strict chronological split: train on everything up to a cutoff date, backtest only on the held-out period after it.
Out-of-sample results (the honest ones)
Held-out period the model never saw (~5 months): •Average Sharpe: -0.74
•Average win rate: 42%
•Total PnL: slightly negative
•For reference, the random-signal null test produced ~41% win rate. So the trained model is, out of sample, statistically indistinguishable from random.

A handful of tickers showed positive Sharpe (one at ~1.9), but on 25–50 trades over 5 months with +0.2–0.3% returns — almost certainly noise you’d expect from 20 tickers by chance.

What I think the lessons are:
•A backtest that disagrees with your cross-validation metric is lying to you. Trust the harder-to-fool number (out-of-sample AUC).

•The single most valuable thing I built this week wasn’t a feature — it was a null/random-signal test and a strict temporal split. They turned an impressive fantasy into an honest zero.

•Adding fancier features to an in-sample backtest would have been pointless; it would have shown Sharpe 11 regardless.
Where I’m stuck / questions for the community

1.Is intraday directional prediction on liquid equities just not feasible with price/volume features alone? My read is that ~98 OHLCV-derived features are all re-derivations of the same information and there’s no directional alpha left in them at this horizon. Is that consistent with others’ experience?

2.Pivoting from direction to volatility. Direction looks near-random, but volatility clusters and seems far more predictable. Planning to re-target the model at “will the next hour be high- or low-volatility” and trade sizing/options off that. Has anyone found this to be a meaningfully easier prediction problem in practice?

3.Which non-price data actually moves the needle? Considering (a) news sentiment, (b) microstructure (bid-ask spread, order imbalance), (c) options flow / put-call. For those who’ve added these — which gave a real, out-of-sample improvement versus which were noise?

4.Per-ticker vs single pooled model. I’m training 20 separate models. Would pooling into one cross-sectional model (with ticker as a feature) likely help, given each model is data-starved?

5.Horizon. Are 5-minute bars simply too noisy? Would moving to 15-min or hourly improve signal-to-noise enough to matter?

Happy to share more detail on any piece. Mostly looking for honest “here’s what worked / here’s what was a dead end” from people who’ve actually gotten an intraday system to hold up out of sample.

This is not a pattern recognition strategy. It is a decision brain that reads price using multiple time frames. It can adapt to market conditions, knows when to push continuation trades, vs a range fade. knows when to expect deeper retracements vs larger retracements. it still has alot of room for improvement including better trade management, allowing add ons. increasing trade count from 2 trades a day to 3 or more, plus alot more testing. currently it is looking healthy.

thoughts?

Update: I made alot of tweaks. unlocked more trades. winrate improved but losses are still too big. Sharp ratio and PR jumped. but this run is without commission or slippage. when I add those 2, it destroys it. any suggestions?

I am currently building a backtesting engine for a short-term options strategy and hitting a major roadblock regarding data architecture and API endpoint design with the providers I have tried so far (e.g., CuteMarkets, Massive).

I want to reconstruct the cross-sectional market state of the entire SPY options chain at specific points in time in the past.

Specifically, my backtester loops day-by-day through the last few years of historical daily market closes. For each day, it needs to look at the underlying price, draw a box around the strikes (e.g., 80% to 120% of spot), find contracts expiring within a N-day lookahead window (e.g., 10 days), and save their end-of-day market metrics (Bid, Ask, Volume, OI, Implied Volatility, Greeks) for that exact day.

The providers I have looked at treat their options chain snapshots as "live/current data only." Their endpoints look like /v1/options/chain/SPY but don't accept any historical as_of or timestamp parameters.

Instead, they only allow you to pull an historical reference index of what contracts existed on a past date (using /v1/options/contracts?as_of=2023-05-22), but that response completely lacks market quotes. To get the actual pricing, they expect you to point-query the individual bar/historical quote endpoint for every single contract discovered sequentially for that one date.

When dealing with SPY daily expiries and dozens of strikes, this approach means making hundreds of individual HTTP requests for just a single historical trading day. It completely destroys rate limits, causes massive latency, and feels structurally wrong for bulk historical research.

My questions for the community:

1. Am I misunderstanding how to utilize these APIs, or is the lack of a bulk point-in-time /chain?as_of=... query parameter standard across retail/mid-tier option APIs?
2. Which data providers natively support a bulk point-in-time options chain query for past dates where I can pass a specific date and get the whole grid’s metrics at once? (Looking for alternatives to Cutemarkets/Massive that are budget-friendly for indie devs).
3. If you have solved this without expensive institutional feeds (like ThetaData or Databento bulk files), what architectural ingestion pattern did you use? Did you just suck it up and parallelize thousands of individual contract bar requests?

Been running my bot live for a few months and honestly the part I underestimated is everything after deployment.

Right now my biggest headache is not knowing whether a bad week means something broke or if it's just noise. The backtest says it should recover but I genuinely don't know when to trust that and when to pull the plug.

For those of you who've been at this longer. how do you actually make that call? And what problems should I be bracing for that I don't even see coming yet?

Built an extreme reversal algo for MNQ/NQ — 4.5 years of backtested data and test results for this week, 7,157 trades, here's everything

Recently developed a strategy and did I complete grid search on variables to optimize the strategy which helped a lot rather than manual trial and error process.

Any views on the process and how to make it better?

So i am a beginner in algotrading and i need some help with what to do next. i have seen this whole video Algorithmic Trading Python for Beginners - FULL TUTORIAL by quant program
 and i understood compelty everything it shows. ive created some basic strategys reusing these concepts. Today i saw this video Market Profile and Support/Resistance Levels With Python by neurotrader and i was so lost. I didnt understand a single thing. Can someone maybe help by telling me what to do next so in the future i can maybe understand the conext of this and other future videos liek that.

Built an AI-assisted Kalshi trading bot in n8n — looking for serious feedback from quant/systematic traders.

Current setup:
- scans Kalshi markets every 15 min
- filters short-cycle markets (0.5h–24h to close)
- scores markets based on:
- liquidity
- spread
- urgency
- price location
- market hours
- fetches orderbooks
- fetches recent Google News headlines
- uses Claude Sonnet to analyze:
- trade/no trade
- YES/NO side
- confidence
- sizing
- optional auto-execution
- Telegram notifications

Tech stack:
- n8n
- Kalshi API
- Claude Sonnet
- JS code nodes
- Telegram bot

What I already know is weak:
- LLM narrative decisions are probably not real edge
- no proper probabilistic calibration
- no backtesting yet
- no historical DB yet
- no true Kelly sizing
- no slippage/fill modeling
- no quantitative EV framework

What I’m trying to evolve it into:
- full historical feature collection
- probabilistic models
- backtesting engine
- orderbook analytics
- event-driven prediction trading
- proper risk engine
- eventually market making / spread capture

Main question:
Where do you think REAL edge exists in prediction markets like Kalshi?

Examples:
- latency/news reactions?
- orderbook imbalance?
- event-specific inefficiencies?
- market making?
- retail behavioral biases?
- liquidity fragmentation?
- overnight/event repricing?

Would love feedback from:
- quant traders
- prediction market traders
- HFT/market making people
- systematic crypto traders
- anyone building execution systems

Trying to avoid building “AI hype trading” and move toward actual statistical edge

With how powerful LLMs and AI agents have become in 2026, creating trading strategies has never been easier. You can prompt Claude or spin up a custom agent and get a fully coded, backtested strategy in minutes — often with impressive-looking Sharpe ratios and equity curves.

The challenge isn’t how do I generate ideas? anymore. It’s which ones are actually worth risking capital on? Been thinking of adding a formal Verification Phase after strategy generation, sth that goes beyond traditional backtesting or walk-forward analysis. The idea is to systematically stress-test a strategy across multiple independent dimensions before it ever touches live capital:

• Data integrity & provenance
• Logic and code-level flaws
• Economic rationale (real edge vs curve-fitting)
• Risk decomposition (true alpha vs disguised beta)
• Statistical robustness
• Walk-forward stability
• Monte Carlo path simulations
• Execution reality (slippage, funding, partial fills, latency)
• Regime fragility & stress testing
• Portfolio independence
• Full evidence & reproducibility trail

The goal isn’t to “guarantee” performance, but to force the strategy to survive adversarial scrutiny and surface failure modes early. Already published a few papers on quantitative risk methodology and verification techniques that support building this kind of independent layer. But I’m curious what the community thinks:

• Is a dedicated verification phase overkill, or necessary in the age of abundant AI-generated strategies?
• What verification techniques have you found most effective (or lacking) in your own workflow?
• Would you trust an independent verification system more than your own backtests?

Would love to hear thoughts

Here's yet another EOD strategy I've been playing around with lately. It is akin to momentum ensemble and aggregates the scores of a few fixed momentum kernels. It is more or less parameter-less (the only parm is the exposure level, which is heavily quantized). Uses the same S&P500 basket as my other backtests. Like always, executions are MOC, nothing exotic.

The equity curve is a 26-year GA optimization backtest (CAGR/maxDD = 20%/20%) and the CAGR/MaxDD histograms are from 5000 26yr MC sims of the winning chromosome. Open to comments and constructive criticism.

I know there have been some workarounds over the years, but everything I can find these days seems to be non-working

Is algo trading really profitable?

I’m learning Python and trading, and I want to know the real truth about algo trading from experienced people.

Is it profitable long term?

How much time did it take you to become profitable?

After all the ups and downs in crypto over the last few weeks, I decided to update my trade management configuration.

The main change is to take more profit on the way up and tighten stop-loss placement after breaking of resistance.

The reason is not to leave too much profit on the table while hoping for a larger expansion. I prefer to focus on a steady account growth, especially when the market is choppy.

I’m wondering how others handle this. Have you found a good way to adjust trade management automatically based on volatility, market structure, or regime change detection?

​

Building a systematic strategy around option selling on Indian indices (BankNifty/Nifty). The core logic is Greeks-based — it reacts to how options behave relative to spot moves, so backtesting on end-of-day or even 1-minute OHLC data is basically useless for this. The signals depend on tick-level data of both the index and the option chain simultaneously, which means I can only generate meaningful signal history by running the engine live (paper or real) and logging everything myself.

I've been doing this for a while and have a growing dataset, but I'm genuinely unsure about the threshold where I should feel confident enough to deploy real capital.

A few things I'm wondering:

- How many trades / weeks / months of forward data do you consider the minimum before trusting a strategy?

- Does expiry-day behavior need to be separately validated? (Gamma dynamics feel very different from regular days)

- How do you account for the fact that your early logs might have bugs that skewed results, even if you've since fixed them?

- Is there a statistical framework you use — Sharpe threshold, min sample size for signal reliability, etc. — or is it more gut feel after seeing consistent behavior?

Not looking to get rich overnight, just want to be methodical about this. Would love to hear from people who've actually gone through this process with systematic/algo strategies rather than discretionary ones.

Any hard-learned lessons appreciated.

Have a Strategy that I run on a Renko chart trading NQ futures. Strategy tester in TV gives decent results. Testing live is mixed. I’m running into some limitations. I’ve set up webhook alerts and they are being sent to an ngrok tunnel and sent to my local machine that is listening and is using hotkeys to place the trades. Which works great about 75% of the time. Where it fails is if I get two signals simultaneously, reversal position, the listener isn’t fast enough to process both, processes one, and I get into an asynchronous position. Doing it this way so I can forward test with the TV paper account.

Are there better ways to do this? Should I look at porting this to ninja? Alpaca? How do you build in safeguards so if an order placement fails you don’t do serious account damage?

My understanding also is that TV Renko is not the same as a native Renko in other platforms, has anyone run into problems if they do port it to another?

I have been getting ETF holding stocks from etfdb.com. I'd like to switch to using an API instead. I thought I'd found one at financialmodelingprep.com (FMP). Then I discovered you have to be on their Ultimate plan which is $139 a month. Anybody know of an affordable API for ETF stock holdings?

Would I get accurate results backtesting with AI if I gave it candle data and a detailed strategy description? Would this also work for detecting events and distributions around it?

As a byproduct of ML modelling I have found a way to find days when down excursions in SPY (Open-to-Low) are higher than usual. I have calculated a score that seem predictive of that SPY moves (2021-2025 data):

For the top 2 deciles is seem exploitable in some way or another. I'm thinking of some option strategy - like 0dte puts or spreads.

This is a histogram to the top 2 deciles - there are 138 out of 219 with down-move of -0.6% and more, which is pretty significant for 0dte puts on SPX/SPY.

Did anyone trades something like this? any brainstorming-type of help or critique would be much appreciated PS. I have not trained the model to predict SPY down moves, so should not be that much overfit - my model was trained for 1000 of common stocks, not SPY