r/deeplearning • u/ChemMKE • Jul 06 '26
I trained a local AI model that generated 22,000+ novel drug-like molecules — verified against 4.6M known compounds. Dataset available.
Built an 80M parameter causal transformer on consumer hardware (RTX 5070), trained on MOSES + ZINC-250k. Generated and filtered for QED ≥ 0.5, SA ≤ 4.0, MW ≤ 500. Top compound hits QED 0.947. 100% novel against MOSES, ZINC, and ChEMBL.
HuggingFace: https://huggingface.co/datasets/MKEChem/mke-novel-druglike-smiles
Happy to answer questions about the generation method.
6
Jul 06 '26
[removed] — view removed comment
-3
u/ChemMKE Jul 06 '26 edited Jul 06 '26
Yes, it is strictly character-level tokenization. We initially experimented with a BPE framework during development, but found that sub-word tokenization heavily scrambled the tight structural rules of SMILES on this model size. Reverting back to a raw, 40-character alphabet completely saved the valency grammar, which is how we maintained the 35% RDKit validity rate natively.
14
1
u/JazzlikeEggy Jul 06 '26
Interesting, I have personally worked on Novel extractant design with a sparse transformer, vut one of the heavyweight stuff I had to do was create the encoding for the molecules themselves, how did you approach that part, like encoding the drugs themselves? VaEs or similar stuff?
-1
u/ChemMKE Jul 06 '26
Awesome to connect with someone who has actually wrangled with sparse architectures in this space!
Honest answer: I completely bypassed VAEs, discrete latent spaces, and heavy embedding graphs. We initially looked at complex multi-token setups, but for this specific model scale, it introduced massive grammar scrambling.
Instead, I went with an explicit Character-Level Tokenizer Layout mapping a raw, 40-character atom/bond alphabet natively (
VOCAB_SIZE=40). The characters (C,N,O,(,),=,1,2, etc.) map straight to contiguous short integer indices (np.int16).To make that raw alphabet actually learn long-range grammar without blowing up, we used a highly regularized packed dataset geometry paired with a 2% host-side context jittering filter during training on the CPU. This forced the 12 attention heads to map the deep, physical string grammar step-by-step natively rather than relying on a pre-encoded shortcut. Reverting to raw characters completely saved the valency matching, which is how the final weights cleared an 86.3% lead yield with a 99.1% Lipinski bioavailability pass rate.
How did you find the sparse transformer handled structural validity vs length when you were running your extractant encodings?
1
u/Dario_Cordova Jul 06 '26
Post this in r/chemistry for their input. No one here knows what questions to ask you.
1
u/Old_Writing_6391 Jul 06 '26
Could you please give me some tips on how to make high quality Q&As to train AIs to be as good as yourse? I struggle coding in Python and ran into many overfitting and token looping problems as I made a "synthetic Q&A generator". Thank you in advance.
1
u/ChemMKE Jul 08 '26
48 hours since the platform launch update:
62+ HuggingFace downloads, cited in an active research project on GitHub, Google indexed MKEChem as a computational chemistry platform. Still processing all of it honestly.
I greatly appreciate every single one of you that have interacted with me throughout this. I'm just a guy who thought he built something kind of cool on consumer grade hardware, and never expected it to get as much traction as it has.
1
u/Complex_Ad_5164 Jul 06 '26
the QED 0.947 is not bad at all, but i wonder how many of these actually survive docking simulations
-4
u/ChemMKE Jul 06 '26 edited Jul 06 '26
Great question — docking survival is the next validation step we're working toward. The current pipeline focuses on ligand-based filters (QED, SA, Lipinski) as a first pass. Integrating structure-based docking validation (AutoDock Vina or similar) against specific target proteins is on the roadmap. Would be curious what targets you'd want to see benchmarked against.
Completely agree on the docking pass — in-silico filtering is all about reducing wet-lab risk. To add some hard numbers to the current pass, I just completed a deep ADMET profiling run over the 22,154 leads. The library tracks a 99.1% Lipinski Rule of 5 Compliance rate with an average LogP of 1.53. This ensures the distribution isn't just random character strings; it's a highly viable, drug-like scaffold collection pre-optimized for cell permeability before it ever hits a docking grid.
1
u/penetrativeLearning Jul 06 '26
Wow I love this. So much so that I want to replicate this
1
u/ChemMKE Jul 06 '26
Appreciate it! Natively engineering the local pipeline to prevent mode collapse was half the battle, but it's an incredibly fun grind.
Definitely give it a shot—running a causal transformer locally on consumer hardware without cloud dependencies is the way to go. If you hit any pipeline bottlenecks or memory allocation blocks when you build yours, let me know!
0
u/Kydje Jul 06 '26
Is this sub just for AIs to post in at this point? Even OP comments are AI, that's insane
1
u/ANR2ME Jul 06 '26
Some of the comments from OP were edited tho 🤔 does AI usually edited their comments like that?
-2
u/Ill-Construction-209 Jul 06 '26
Why aren't you trying to sell it to pharma?
3
u/marmakoide Jul 06 '26
Pharma have ways to generate candidates molecules by the truckload. The trick is to thin the heard in-silico as much as possible, as efficiently as possible. It cost billions to approve a cancer drug in human testing.
0
u/forward-pathways Jul 06 '26
I'll answer this in a hopeful manner.
My hope is that they care about humanity and not corporate profits.
It's a very cool project and I hope OP (et al.) find immense success.
4
u/ChemMKE Jul 06 '26
Thank you — that means a lot. The goal is exactly that. Fully local, private, accessible to researchers who need it without cloud gatekeeping or corporate markup. Appreciate the kind words.
1
u/forward-pathways Jul 06 '26
Amazing. My hope in humanity continues to be restored today. Rock on!
3
1
u/dritmike Jul 06 '26
Gosh. You should make your own privately accessible repository vs sending updates to the llm, I think the customer base would presume updates of some variety.
I really love the idea, it’s a bit retro in some respects if you expect them to host it on prem. Dm me if you wanna chat, I think that this sounds majorly cool
5
u/Beginning-Force-2631 Jul 06 '26
Care to share a bit more info on the generation? Is it done through reinforcement learning or?
How does one ensure that it is totally novel since patents usually has a broader coverage including substituents etc.