Running AI models on your own machine

Hosted assistants are convenient, but there are good reasons to run models locally: privacy, offline use, no per-token bill, and the freedom to experiment. Here is what you need and which tools to look at first.

Hardware

• A recent GPU with at least 8 GB of VRAM runs 7B to 8B parameter language models comfortably and small image models at reduced speed.
• Apple Silicon Macs with 16 GB or more of unified memory are a strong option for language models.
• CPU-only works for transcription with Whisper and for small quantized models, but expect slow generation.

Language models

Open WebUI gives you a chat interface over any local runtime, with document upload and multiple users. For coding, Aider, Continue, and Cline all accept a local model as the backend, so you can keep your source code on your own machine.

Images and video

ComfyUI is the workhorse for local image and video generation and supports Stable Diffusion, FLUX, and LTX-Video. Fooocus is the easier option if you want good results without learning a node graph.

Speech

Whisper transcribes and translates audio locally. Fish Audio publishes open weights for text-to-speech and voice cloning that run on consumer hardware.

Automation

n8n can be self-hosted and wires local models into workflows with its LangChain nodes.

What to expect

Local models trail the largest hosted ones on hard reasoning tasks, but for drafting, summarizing, transcription, and image generation the gap is small and closing. Start with one task, measure whether the result is good enough, and expand from there.