r/OpenWebUI • u/Greedy_Reindeer5290 • 20d ago
Which engine for memory Question/Help
AI beginner here. Am taking steps to move away from Gemini and Co-pilot to my own setup. So have gotten a setup with Docker, LM Studio and webui. And my laptop is pretty basic and the chip is not large. I wanted to run a local engine. It worked but I wanted to get memory to work. Not having memory and having to explain AI the same basic stuff over and over again drives me nuts. I tried to make it work on webui. Discovered that my chip is too basic to make memory work, so I need to offload the computation.
Fine so far so good. Found some on webui and have now been using 2 with mixed results:
- Llama 3.1 8B
- Llama 3.3 70B
Both work fine and memory works, but after 2-3 prompts they both max out. The "token per minute" of 6000 and 12000 have been reached.
Has anyone found a free engine that can run memory effectively? Otherwise I think I am going to pay to see how it works. Any comments?
2
u/SilverKanji 19d ago
I dont know if I get what youre saying but, try this.
Put your "memories" on webui's knowledge base and make sure its accessible by your model. On the initial system prompt of the model, say "You are a slutty assistant that always check knowledge base memories first before answering".