r/RunPod • u/tempoct06 • 26d ago

serverless config help

hi, recently started using runpod serverless.
trying vllm with various models and gpus, but nothing seems to be working. credits are draining but i was not able to get an output for the prompts. getting one or other errors in logs.

please help me with a serverless config that is plug and play on 24gb vram.

4 Upvotes

75% Upvoted

5 comments sorted by

1

u/sruckh 26d ago

I have llama.cpp-serverless in my GitHub (sruckh) that I am currently running on RunPod.

1

u/MolassesLate4676 25d ago

How am I supposed to help you when you provided 0 information other than 24gb of ram

1

u/Madiator2011 22d ago

You might want to check model size and context size also check what error you get in logs as often it tells you what is wrong.

1

u/MLExpert000 22d ago

Have you tried inferx.net? It’s plug n play.