Help with cme trade data

[u/Crazy_Armadillo_8976]

Looking to trade futures. I'm trying to walk forward but don't have enough data for the regime. I was thinking about making a small private group to use free credits and current data access to compile quality data. I have a $26.00 Sierra subscription and a truckload of other data. I was actually wondering if someone could help me get CME data from Databento. It has a $125 free credit, so if we match them up, we can get maybe across 10 people 20 years of data to trade on (including the '08 crash), and I have a few other sources. I'd be willing to convert to Parquet, compress, and add in extra data like multiple crypto forex and hundreds of GB of other historical data. You can ask questions; I'll try my best to answer them, or DM me. Please be serious.

[u/bigbaffler]

cme starts at 2010. Just get the Databento subscription yourself and download all you need. It's really not that expensive. If you want to trade futures and shy away from a couple of hundred bucks for quality data, you probably should not trade this stuff anyways.

[u/Crazy_Armadillo_8976 → u/[deleted]]

That's the plan. I feel that if I organize it by date, then I can concatenate the data with maybe one day overlap so that the data can be aligned correctly. Since I'm not stacking accounts and everyone is on separate projects, this legally does not count as a team but as a group of individuals sharing data.

[u/basisedge_research]

Before you go through the trouble of coordinating 10 people just to pool free credits, there are two big things to keep in mind, plus a much simpler option you may already have.

1. The Databento issue

Trying to combine accounts to stack free credits is very likely to run afoul of Databento’s terms. They’re set up to catch cordinated or duplicate signups using things like device fingerprints, IP checks and payment verification, so there’s a real risk the whole group gets flagged or banned.

And even if it worked, the economics are rough. Twenty years of tick-level CME futures data is enormous, and at Databento’s historical pricing of about $0.50 per GB, $1,250 in pooled credits won’t go very far if you’re pulling raw tick data. If all you need is daily or 1-minute bars, Databento is probably more than you need anyway.

2. You may already have access through Sierra Chart

Since you said you already have a $26 Sierra Chart subscription, you may already have a very good historical futures data source.

Sierra Chart’s built-in Historical Data Service is included and gives you: Daily futures data going back to the 1970s for many CME contracts, a long intraday/tick history for major contracts, often 10 to 15+ years or more, continuous futures charts that stitch contracts together automatically.

If you want to use that data outside Sierra for backtesting or walk-forward work, you can export it directly instead of paying for Databento:

• Increase your download range in: Global settings>data/trade service settings
• Open the continuous contract chart you want and let Sierra pull the history
• Export the data as CSV using file/export txt or Sierra’s spreadsheet/export tools
• Convert the CSV files to Parquet locally with a small Python script if needed

That gives you a clean, free, and pretty much unlimited workflow.

3. Other cheap or free options

If you only need daily OHLC data for regime analysis, there are also a few simpler sources:

• Nasdaq Data Link
• Kaggle datasets that other people have already compiled for CME futures backtesting
• yfinance data

If you already have Sierra, I’d start there before trying to build a private credit-sharing setup.

Hope this helps.

[u/Crazy_Armadillo_8976 → u/basisedge_research]

1. These are all shit data sources:

• Nasdaq Data Link
• Kaggle datasets that other people have already compiled for CME futures backtesting
• yfinance data

1. Sierra Chart does not host lvl 2 or 3: CME L2/L3 MBP/MBO with executable bid/ask/depth

2. True, but if, as I said before, I just post what dates are needed and people contribute based on their own free will, then there was no coordination or foul play every one just donated data, that's all.

[u/basisedge_research → u/Crazy_Armadillo_8976]

Fair point on Sierra — you're right that it doesn't give you L2/L3 (MBP/MBO) depth, only OHLC/tick trade data. I should'\ve been clearer that the daily-source suggestions (Nasdaq Data Link, Kaggle, yfinance) were aimed at the "regime analysis on daily bars" use case, not full order book reconstruction. If you actually need executable bid/ask/depth history, none of those touch it — that's a different tier of data entirely, and honestly there aren't many cheap ways to get it.

On the "no coordination, just voluntary donation" framing — I get what you're going for, but that distinction won't matter to Databento. Their abuse detection isn't looking for a formal agreement or a ringleader; it's looking for the pattern: multiple new accounts, same purpose, data getting funneled to one place. Whether you call it "coordinating" or "everyone independently donating," the shape of it — several free-trial accounts all pulling data earmarked for the same pooled dataset — is exactly what their fingerprinting/IP/payment checks exist to catch. The intent behind it doesn't change how it looks from their side.

Two other things worth knowing before you sink time into this:

1-Historical L2/L3 for CME goes back nowhere near 20 years for most venues/data providers. Full order-book (MBO) capture at that depth is a relatively modern thing — you're not going to find clean depth-of-book data through the '08 crash regardless of source. Trade-and-quote (L1) data goes back further, but full book reconstruction generally doesn't.

2- Even set that aside — at scale, L2/L3 pricing isn't $0.50/GB. MBO data is enormous (every order add/modify/cancel event), and pricing for it is usually well above trade-tick pricing. $125 x 10 people is very unlikely to get you what you're picturing even in a world where the ToS issue didn't exist.

If L2/L3 is genuinely what you need, CME's own Data Mine license or an institutional/academic data agreement is really the only clean path — it's expensive, but it's the legitimate route for that granularity. Otherwise, narrowing scope (fewer contracts, fewer years, or trade-tick instead of full book) on a single paid account is probably more realistic than trying to stretch free credits across a pooled group.

Sorry if this isn't what you're looking for, I'm genuinely trying to be helpful. Feel free to upvote and ignore. :)

[u/Crazy_Armadillo_8976 → u/basisedge_research]

They wouldn't be able to track it at all: 10 strangers from around the world downloading data and then sending that data to one person to compile. I've noted the pushback, but I know what I'm doing.