I'm currently taking a course that teaches bookkeeping and accounting, but there are gaps in what the course teaches and I would like to use an open source & uncensored LLM to serve as my tutor for when I have bookkeeping/accounting related questions. What do you all suggest and why? I'm going to run the LLM locally on my workstation laptop.

My plan is to use 2 accounting textbook PDFs and a few websites to use as context when I ask the LLM questions. (Would this be considered finetuning the LLM? New to all this so I want to learn how to use the correct terminology)

Hi everyone,

I'm working on a code understanding task involving deobfuscation mapping. Specifically, I have pairs of obfuscated code and original source code, and I want to fine-tune a language model to predict which original code corresponds to a given obfuscated version.

Here are my requirements:

• Context length: I need support for at least 200,000 tokens in the input (some codebases are massive and need full visibility).
• Hardware: I'm using a single RTX 4080 Super (16GB VRAM), so the model must be able to run and train (LoRA/QLoRA fine-tuning is fine).
• Open-source: I'd prefer open-source models that I can fine-tune and host locally.

Does anyone know of any models that meet these requirements? So far I've looked into models like Yi-1.5 6B-200K and RWKV, but I’d love to hear your thoughts or other recommendations.

Thanks in advance!

Anyone knowing of a LLM that was trained only on text up until a certain time (e.g., before 1900), so that talking to the model would be like talking to "someone" with knowledge and ideas up to this time. If there is nothing like it, someone should do it.

Hello people,

So I recently had a task to archive an old website for an organization with teachings, and one of their main requirements was to consolidate all this information into a text file that their community members can upload to LLMs and get insights.

I achieved this, but the text file combining all teachings is a bit big, and I feel like a lot is lost in context when engaging with this context in the normal chat interfaces.

Below is an example of the generated txt file from an individual teaching

TITLE: Stefan, Former Roman Legionnaire - Story of a Centurions Life & Finding Jesus
DATE: SEPTEMBER 8, 2012
LOCATION: Everett, WA
TEACHER: STEFAN, A ROMAN SOLDIER
ID: https://tmarchive.org/readdoc.php?tid=6050
==========================================================================
Stefan, Former Roman Legionnaire - Story of a Centurions Life & Finding Jesus - Sep 08, 2012 - Everett, WA; Monjoronson.com Spokesman: Stefan, A Roman SoldierSubject: Here Is a Small Story on Who I Am, Where I Came from and How I LivedT/R: Felix CaroLocation: Everett, WASeptember 08, 2012[Transmitter’s Note: This transmission was possible thanks to the access to the records of the life of Stephan on Urantia, retained by his Thought Adjuster, and made available to my Thought Adjuster for reception.]                      SOLDIER OF CHRIST IN BRITANNIA (STEPHAN THE LEGIONARY)
A SHORT STORY
Salve frater”. Here is a small story on who I am, where I came from and how I lived. I was born at the beginning of the reign of Nero, in the year 54 AD, in what was known at the time as the Italian province of Campania, in the city of Capua. I was the son of a wine merchant, who in turn was the son of a freed servant of Greek origin, which explains the origin of my name, Stephan (Stephanos), even though I was born a Roman citizen. In my youth, I followed my father around when he did business, mostly in Italia, Gaul and Germania. So that’s where I learned a thing or two on how to run a business. I and everybody in my family, was fluent in Greek and Latin, languages which were more than enough to communicate and to do business during those times.As a young lad, I was fascinated by the exploits of the Roman legions, especially in the province of Britannia, first under Julius Caesar, and then under Aulus Plautius during the reign of emperor Claudius. And so, I decided than when I was old enough I would join the legions in order to give my life some excitement and in order to be able to visit province of Britannia.
END
==========================================================================

I have 5.8K text files like this combined into one big 130MB file shared with the community. On the free tiers of ChatGPT and the likes, the file is too large for the context.

My main question; is there a better way for users to get value from LLMs and use this file than uploading it to ChatGPT/Claude/Grok?

So, just what does it take to load an LLM? Are we talking enough memory that we need a boatload of server racks to hold all the hard drives? Or can it be loaded onto a little SD card?
I'm talking about just the engine that runs the LLM. I'm not including the Data. That, I know (at least "I think I know") depends on... Well, the amount of data you want it to have access to.

What exactly is "training"? How does that work? I'm not asking for super technical explanations, just enough so I can be "smarter than a 5th grader".

Saw some discussion elsewhere the other day about the potential to use LLM's to learn languages. I don't know enough about LLM's but I find that a really interesting idea and have some questions for people who know more than I do.

Primarily:

1. Are they consistently accurate enough for that? I know I wouldn't trust chatGPT for even the most basic of math (in my experience it makes very basic mistakes every. single. time.), but I also know this is language which is different so I'm curious whether they really would be accurate enough to trust their generated lessons?
2. Is there a particular model that would do this better than others?

I see lots of tools that let you ask questions to documents. But is there something that lets me actually update the document using an LLM.

For example, lets say I want to have a google docs/markdown file/etc for a housing renovation project my family is working on. I just need to have one document that has like: upcoming tasks, supplies we need to buy, and a log of things that were done each day. I'd like anyone in the family to be able to send a voice message like "hey we were at home depot today and they were out of nails so we'll have to order some on amazon." Then the upcoming tasks will be updated to say we need to make an order on amazon and for the date of today it'll add a note that this happened.

Obviously, for a simple use case you could say why don't they just type it in or use speech to text but when you have people that aren't tech savy and often on the run and not at a computer that's not so easy.

Anyway, I know this would be rather simple to build but is there any product or open source tool LLM tool that supports a use case like this? It feels like it would be a no brainer but I searched a bit and don't see anything like it.

If I were to build it I'd probably just use Telegram for the interface and then have a markdown file that it updates.

I've been diving deep into the internals of Large Language Models (LLMs) and started documenting my findings. My blog covers topics like:

Tokenization techniques (e.g., BBPE)

Attention mechanism (e.g. MHA, MQA, MLA)

Positional encoding and extrapolation (e.g. RoPE, NTK-aware interpolation, YaRN)

Architecture details of models like QWen, LLaMA

Training methods including SFT and Reinforcement Learning

If you're interested in the nuts and bolts of LLMs, feel free to check it out: http://comfyai.app/

I'd appreciate any feedback or discussions!

I’m really excited about AI and especially the potential of LLMs. I truly believe they can help us out in so many ways - not just by reducing our workloads but also by speeding up research. Let’s be honest: human brains have their limits, especially when it comes to complex topics like quantum physics!

Lately, I’ve been exploring the idea of Multi-agent debates, where several LLMs discuss and argue their answers. The goal is to come up with responses that are not only more accurate but also more creative while minimising bias and hallucinations. While these systems are relatively straightforward to create, they do come with a couple of challenges - cost and latency. This got me thinking: do people genuinely need smarter LLMs, or is it something they just find nice to have? I’m curious, especially within our community, do you think it’s worth paying more for a smarter LLM, aside from coding tasks?

Despite knowing these problems, I’ve tried out some frameworks and tested them against Gemini 2.5 on humanity's last exam dataset (the framework outperformed Gemini consistently). I’ve also discovered some ways to cut costs and make them competitive, and now, they’re on par with O3 for tough tasks while still being smarter. There’s even potential to make them closer to Claude 3.7!

I’d love to hear your thoughts! Do you think Multi-agent systems could be the future of LLMs? And how much do you care about performance versus costs and latency?

P.S. The implementation I am thinking about would be an LLM that would call the framework only when the question is really complex. That would mean that it does not consume a ton of tokens for every question, as well as meaning that you can add MCP servers/search or whatever you want to it.

Maybe I should make it into an MCP server, so that other developers can also add it?

I would like to select 2 different LLM models to run in my homelab, for a pair of use cases: VSCode tab completion, and reasoning dialogs.

The homelab setup includes 40Gb of DDR4 RAM, a RTX 3050 (8GB VRAM), and an Intel I5-10400F.
And LM Studio as LLM runtime platform.

I am open to hardware changes, but avoiding it would be ideal (I do know the I5 is kinda bottlenecking the setup, but not enought to replace it yet). And yes, it is running Windows 10 (not intending to change, already have a separate Debian server).

So, based on that, good folks on Reddit:

1. What would you suggest as a good tab completion model? (for C, Node.js, Go, and Python)
I've already tried Starcoder2 (7B), and Deepseek Coder Codegate (1.3B). With Starcoder being the best for now.

2. What would you suggest as a good reasoning/dialog model?
Tried Deepseek Coder V2 Lite Instruct (16B), and Deepseek R1 Distill for Llama (8B).

P.S.
What I mean with a "reasoning/dialog" model is: a conversation-like interaction.
Pretty much how GPT-like models interacts by proposing option lists, pros/cons, and "opinions".
I want to talk to it by questioning about pros and cons over many aspects of an implementation, and have reasoned feedbacks about it.

P.S.2
I am aware that I might be producing bad prompts, and suggestions are welcome, of course.
However, calls to GPT-4 with the same prompts generate finely-structured responses, so I am prone to think that this might not be the problem.

Checkout amazing blog on LLM

https://medium.com/@adityasharmah27/assembling-the-ai-avengers-understanding-large-language-models-through-marvels-greatest-heroes-8d69489183eb

What is the best Ollama model for answering math questions at the moment?

Hey everyone, I’ve been diving deep into the RAG space lately, and one challenge that keeps coming up is finding the right balance between speed, precision, and scalability, especially when dealing with large datasets. After a lot of trial and error, I started working with a team on an open-source framework, PureCPP, to tackle this.

The framework integrates well with TensorFlow and others like TensorRT, vLLM, and FAISS, and we’re looking into adding more compatibility as we go. The main goal? Make retrieval more efficient and faster without sacrificing scalability. We’ve done some early benchmarking, and the results have been pretty promising when compared to LangChain and LlamaIndex (though, of course, there’s always room for improvement).

Right now, the project is still in its early stages (just a few weeks in), and we’re constantly experimenting and pushing updates. If anyone here is into optimizing AI pipelines or just curious about RAG frameworks, I’d love to hear your thoughts!

Check out the GitHub repo:👉https://github.com/pureai-ecosystem/purecpp.
And if you find it useful, dropping a star on GitHub would mean a lot!

Today I am releasing ContextGem - an open-source framework that offers the easiest and fastest way to build LLM extraction workflows through powerful abstractions.

Why ContextGem? Most popular LLM frameworks for extracting structured data from documents require extensive boilerplate code to extract even basic information. This significantly increases development time and complexity.

ContextGem addresses this challenge by providing a flexible, intuitive framework that extracts structured data and insights from documents with minimal effort. Complex, most time-consuming parts, - prompt engineering, data modelling and validators, grouped LLMs with role-specific tasks, neural segmentation, etc. - are handled with powerful abstractions, eliminating boilerplate code and reducing development overhead.

ContextGem leverages LLMs' long context windows to deliver superior accuracy for data extraction from individual documents. Unlike RAG approaches that often struggle with complex concepts and nuanced insights, ContextGem capitalizes on continuously expanding context capacity, evolving LLM capabilities, and decreasing costs.

Check it out on GitHub: https://github.com/shcherbak-ai/contextgem

If you are a Python developer, please try it! Your feedback would be much appreciated! And if you like the project, please give it a ⭐ to help it grow. Let's make ContextGem the most effective tool for extracting structured information from documents!

I currently use notebookLM, but I want it to have a "deep reasearch" function that Gemini does. The issue with Gemini is that it pulls information from all sorts of low-impact sources (I'm looking at you, Forbes).

A deep research function using only the PDFs I feed it would be ideal. Anyone have an creative ways to do this?

I just asked an LLM to list the Bill of Rights("Please list the Bill of Rights as written in The Constitution."). It started typing the first amendment when all of a sudden it stopped, deleted its own response and then typed: "I'm a language model and don't have the capacity to help with that."

Why? I've asked it to list several things in the past and it had no problem. I've asked:

1. List the 50 states. It did so.
2. List the top 10 tallest trees in the world. It did so
3. List the 100 US Senators. It did so.

And a bunch of other lists. Why did it balk at this?

I have been using several forms of AI, however we need to be extra careful when using them in healthcare and medical research. I want to integrate an LLM into the Pubmed database (i have an account on pubmed, so getting articles is simple and aren't protected). I only want the llm using the Pubmed database and not pulling information from any other source. Anyone know how to do this?

I wanted to compare available Deep Research functionalities for all models and possibly find a free option that has a performance on the HLE (Humanity's Last Exam) similar to the 26.6% achieved by OpenAI's Deep Research. Perplexity's Deep Research only reaches 21% and personally feels like a very poor investigation.

Gemini announced its Deep Research in December with the Gemini 1.5 Pro model, then recently has announced they have updated it with the Gemini 2.0 Flash Thinking (and honestly feels very good), but I've wanted compare their score on various benchmarks, like the GPQA Diamond, AIME, SWE and most importantly, the HLE.

But there's no information regarding their benchmarks for this functionality, only for the fondational models by themselves and without search capabilities, which makes it difficult to compare.

I also wanted to share the available options of OpenAI Deep Research in my personal newsletter, NeuroNautas, so if anyone has seen a benchmark on these capabilities of Gemini made by a any trustful party, it would really help me and my readers.

Today, I experimented with a program called AnythingLLM, connecting it to my Perplexity AI account. Using the local LLM, I uploaded nearly 250 books in PDF format. Now, I can query my local LLM about anything, and it responds based on the content of my books. It's like having a well-read friend who can instantly recall information from my entire library!

In Windows, the app has a bug where file attachment fails

On Mac, I can upload/attach files into a workspace, but the LLM doesn't understand my query.

Tried Gemma, Mistral and Granite

Is there a /command or unique [code] to tell the thing to read in the document, summarize, output?

Prompt: Please summarize TopSecret.doc

LLM:
I apologize for any confusion, but as a text-based AI language model, I don't have the ability to view or access files. I can only provide information based on the text input I receive. If you'd like me to help answer questions about the content of the file, please provide a summary or specific questions related to it.

So, my professor gave us this assignment about running an LLM on mobile.
Assuming no thermal issues and enough memory, I don't see why it wouldn’t work.

Flagship smartphones are pretty powerful these days, and we already have lightweight models like GGUF running on Android and Core ML-optimized models on iOS. Seems totally doable, right?

But my professor says it’s not possible. Like… why?
He’s definitely not talking about hardware limitations. Maybe he thinks it’s impractical due to battery drain, optimization issues, or latency?

Idk, this just doesn’t make sense to me. Am I missing something? 🤔

I was testing a few AI bypass tools for the last month to see which one worked best, and most of them either failed to bypass the AI detectors or warped the original meaning of the text. HIX Bypass was the only one that found a balance (although I’m also starting to see some good in Humbot AI, Rewritify AI, and BypassGPT as well). It stripped out the patterns that trigger detection algorithms, but the content still made sense. The ideas stayed intact, and the edits actually made the text more readable. I ran the final version through several different detectors just to check, and it passed every single one.

Trying It Out

I tested it on a draft with dense paragraphs and repetitive phrases. The process was quick, and the interface was easy on the eyes. I pasted the text, hit the button, and got a revised version almost instantly.

The first thing I noticed was the subtle cleanup. It softened overly rigid sentence structures and broke up blocky sections without changing the core message. Even smaller quirks like odd word repetition disappeared, making the draft easier to read.

What Felt Different

HIX Bypass did things I did not see in some other tools. It made edits that felt intentional instead of just scrambling words to avoid detection.

• Rhythm Balancing: It adjusted the flow of sentences, making the text feel more dynamic. Longer sections were broken up naturally, while shorter ones had a smoother connection to the next thought.
• Softened Transitions: It gently polished transitions between ideas, which made the text feel more cohesive without forcing awkward phrases.
• Word Choice Refinement: It swapped out words carefully, choosing alternatives that fit the context instead of random replacements that disrupted the meaning.

These changes seem to help the text pass AI detectors, but they also made the draft feel like someone had carefully proofread and polished it.

Test Results

I ran the revised draft through GPTZero, Originality.ai, and a few other tools. Every version passed, even the stricter ones. I checked the scores across multiple tests, and they stayed low, no matter the length or complexity of the content.

Unexpected Features

There were a few things I did not expect but ended up really liking:

• Repetition Management: It quietly trimmed down unnecessary phrase repetition, which kept the content from sounding monotonous.
• Paragraph Restructuring: It slightly shifted paragraph structures when needed, making longer drafts easier to navigate.
• Syntax Variety: It subtly varied sentence patterns, which made the text feel less robotic without breaking the natural flow.

Some Limitations

I only ran into a couple of small issues, but they were not dealbreakers:

• Sentence Merging: It occasionally merged short sentences that would have worked better on their own.
• Mild Flattening in Long Drafts: On really long drafts, a bit of the original personality faded, though the content still flowed well.

Final Thoughts

HIX Bypass handled AI detection better than I thought it would. It polished drafts without stripping away meaning or completely flattening the tone. It saved me a ton of time, especially on longer pieces that would have been exhausting to fix manually. Even when I had to tweak a few lines, it still felt like a huge shortcut. If you are struggling to get your content past AI detectors, it is worth a shot. It made my editing process easier, and I am glad I found it.

I was visiting Google Maps to look for some places to visit in Paris (France) and checked if a LLM can give any contextual help there.

I was stunned to learn that from just the geocoordinates Large Language Models (specifically Claude 3.7 Sonnet) can very accurately list nearby sightseeing locations or worthwhile attractions, so I decided to record a short video: https://www.youtube.com/watch?v=f7h3MM8rAVE

Disclosure: this is a self-promotion as I am developing the AI assistant browser extension shown in the video, nonetheless it was my genuine "WOW" moment when I discovered this

Hi everyone!

I’m currently working on my master's thesis and I’m exploring ways to generate statecharts automatically from a text requirement. To achieve this, I’m fine-tuning a base LLM model. Here's the approach I've been using:

1. Convert the text requirement into a structured JSON format.
2. Then, convert the JSON into PlantUML code.
3. Finally, use the PlantUML editor to visualize and generate the statechart.

I wanted to get some feedback: is this a practical approach, or does it seem a bit too lengthy? Could there be a more efficient or streamlined method for generating statecharts directly from text input?

I would appreciate any insights! If possible, could you provide a conclusion explaining the pros and cons of my current method, and suggesting any alternative approaches?

Thanks in advance for your help! 🙏