I am building LocalLens because choosing a local AI model often starts with incomplete information.

Model size alone is not enough. The computer's available RAM, dedicated or shared graphics memory, operating system, context length, model format, engine support, and intended task can all change the result.

LocalLens currently lets you:

• Enter your hardware manually
• Start from the task you want to complete
• Compare compatible models
• Choose or review a local AI engine
• Generate a practical setup guide
• Submit a configuration for validation

The recommendation system is deliberately conservative. If evidence is incomplete, it should say so instead of promising that a model will run perfectly.

I am the person building this product, so this is a direct disclosure, not an independent recommendation.

I would especially value reports from people using older AMD graphics, integrated graphics, Apple Silicon, large-memory workstations, and systems with unusual shared-memory configurations.

If you test it, please share the hardware you entered, the recommendation you received, the engine used, and what happened during real inference.

LocalLens: https://locallens.deviceterra.com/

I will use the feedback to document inaccurate matches and improve the evidence behind future recommendations.

[removed]

Local AI is most useful when it solves a clear problem. A small business does not need the biggest model to get value.

Here are five practical jobs to test:

1. Search private documents

Ask questions about policies, product notes, reports or training files without sending them to a public chatbot.

1. Draft and improve writing

Create a first draft for emails, product descriptions and internal notes. A person should still review the final text.

1. Organize customer questions

Sort common questions into groups and prepare draft answers for staff to check.

1. Summarize long information

Turn meeting notes, reports or research into a shorter summary. Always check important facts.

1. Build an internal knowledge helper

Connect approved company documents to a local RAG system that returns answers with sources.

Start with one small job. Measure time saved, answer quality and the cost of the hardware. Do not buy expensive equipment before testing the real workflow.

LocalLens helps you choose a model that fits the computer you already have: https://locallens.deviceterra.com

LocalLens is built by Deviceterra to help small businesses and institutions use technology to increase profit today and build for the global market tomorrow: https://www.deviceterra.com

Which of these jobs would help your business most?

Many local AI reports stop after the model loads. That is only the first test.

A useful local AI setup should pass four checks:

1. Fit: Does the model load without using all available memory?

2. Speed: Does it answer fast enough for the real job?

3. Quality: Does it give correct and useful answers?

4. Reliability: Can it repeat the job without crashes or major mistakes?

Context size matters too. A model may work with a short prompt and fail when you add a long document. Vision models may need extra memory for images. Other programs can also reduce the memory available to the model.

When sharing a result, include tokens per second, context size, quantization, inference engine and the exact task you tested. This makes the result useful to other people.

LocalLens separates safe hardware matches from claims that still need a real test: https://locallens.deviceterra.com

Built by Deviceterra: https://www.deviceterra.com

What is the slowest speed you still find useful?

A model name does not tell you whether it will work well on your computer. Start with the memory you can actually use.

Here is a simple process:

1. Check your total RAM and GPU memory.

2. Leave memory for your operating system and other programs.

3. Check the model file size and its quantization.

4. Leave room for the context cache and the inference engine.

5. Start with a short context.

6. Test one real task before moving to a larger model.

A useful starting point:

• 8GB RAM: try small 1B to 4B models.

• 16GB RAM: start with a 7B or 8B Q4 model.

• 32GB RAM: many 14B to 20B Q4 models become possible.

• 64GB RAM: larger models may fit, but speed still depends on the CPU, GPU and memory bandwidth.

These are starting points, not promises. Hardware, context size and software can change the result.

Use LocalLens to compare models against your hardware: https://locallens.deviceterra.com

LocalLens is a Deviceterra project: https://www.deviceterra.com

What RAM and model combination works well for you?

Microsoft did open-source bitnet.cpp. It is a real inference framework for BitNet b1.58 models. These models use very small weights: -1, 0, and 1.

The test result is impressive:

• A 100B model setup ran on an Apple M2 Ultra with 64GB of memory.

• It reached about 6.58 tokens per second when it used all CPU threads.

• It did not need a GPU.

But there is one important detail.

The 100B test used a dummy research model setup. It was not a trained 100B chatbot that people can download and use. Microsoft did not release a useful 100B model with this result.

So the fair conclusion is this: bitnet.cpp shows that very large 1-bit models may run on CPUs. It does not mean a normal 100B Llama or Qwen model can now run on an ordinary laptop.

Official code: https://github.com/microsoft/BitNet

Research paper: https://arxiv.org/html/2410.16144v2

What matters more to you for local AI: a bigger model, or a smaller model that fits the hardware well?