From Decrypt · By Jose Antonio Lanz
In brief
Open-weight models have trailed the best closed models by an average of four months since January 2026, according to Epoch AI's May 29 analysis.
Ollama and LM Studio are free, small models need about 8GB of VRAM or Mac unified memory, and LM Studio can operate entirely offline.
The largest open models, such as Alibaba's 2.4-trillion-parameter Qwen 3.8-Max, sit far beyond consumer hardware, and closed models won 24 of 31 tests in Alibaba's own comparison.
Open-weight AI models now trail the best closed models by only about four months on average, and a growing number of them run on hardware you probably already own.
"Open-weight" means the company publishes the weights, the files holding everything the model learned, so anyone can download and run them. Two free programs, Ollama and LM Studio, handle the rest, and Gemma 4 downloads in Ollama's library range from about 4.6GB to 20GB.
With open models, you don't need to pay $20 a month for an AI subscription or send your conversations to someone else's servers to get a capable chatbot. This guide explains how to download and run AI models directly on your computer, what kind of hardware you'll need, and how to get started with free tools like Ollama and LM Studio.
Whether you want to save money, keep your conversations private, or experiment with AI models you can customize yourself, setting up a local AI assistant may be easier than you think.
Why run an AI model on your own machine
A cloud chatbot sends every prompt to a company's servers, and a court ordered OpenAI to preserve ChatGPT logs, including deleted ones, as Decrypt reported in a review of privacy-focused AI tools. A local model keeps the conversation on your computer and nobody can use it without your permission.
Myriad: Which company will IPO next? Click to make your prediction.
LM Studio's documentation says the app "can operate entirely offline, just make sure to get some model files first," and that documents attached to a chat can be used entirely offline too.
Subscriptions to paid chatbots run about $20 per month, while the software here is free. The cost shifts to hardware you may already have.
Licenses decide what you can do with downloaded weights. DeepSeek's R1 series is under the MIT License, and its model card says the models "support commercial use, allow for any modifications and derivative works," while Google's Gemma 4 and OpenAI's gpt-oss ship under Apache 2.0.
Running models on an open-source license like Apache or MIT means you are free to use, modify, and do whatever you want with a model. This means users are able to access improved and modified versions of a specific model that better fit their needs. Maybe you need a model that is good at medicine, so you don’t care about how good it codes. This level of personalization is only available in customized models.