AIArsenal/Stacks/100% Private, 100% Local AI Stack
STACK RECIPE

100% Private, 100% Local AI Stack

Frontier-ish AI that never leaves your laptop. No cloud, no tracking, no per-token billing.

FOR
Privacy-sensitive work, offline development, cost-zero experimentation
BUDGET
$0/mo (hardware: 16GB+ RAM, ideally 8GB+ VRAM)
DIFFICULTY
Intermediate
TIME TO SHIP
2 hours to download models and wire up UI
TOOLS
5 tools
THE STACK

5 tools that work together

01 · LOCAL LLM RUNTIME
OllamaFree; 100+ models locally

Ollama is the simplest way to run 100+ open models with a single CLI — auto-detects your hardware.

TRY →
  • ▸Running LLMs 100% offline on your own machine
  • ▸Developers who want zero API costs and full privacy
02 · CODING AGENT
Continue.devFully free; unlimited with local models via Ollama

Continue.dev is the best OSS code assistant and runs against any local Ollama model — zero data leaves your box.

TRY →
  • ▸Developers prioritizing privacy via local AI models (e.g., Ollama).
  • ▸Users who demand full control over their coding assistant's AI model.
OR SWAP IN
03 · TOP OPEN LLM
Llama 3.xFully open; 128K context

Llama 3.1 8B runs on a laptop; 70B on a workstation — both good enough for 90% of what GPT-3.5 handled.

TRY →
  • ▸Default choice for anyone self-hosting an LLM
  • ▸Sizes from 8B (laptop) to 405B (GPT-4-rival)
OR SWAP IN
04 · LOCAL IMAGE GEN
Stable DiffusionUnlimited self-hosted (8GB+ VRAM)

Stable Diffusion via ComfyUI gives unlimited local generation with community fine-tunes — needs 8GB+ VRAM.

TRY →
  • ▸Users with powerful GPUs (8GB+ VRAM) seeking unlimited, private image generation.
  • ▸Artists & power users desiring deep customization via LoRAs, fine-tunes, and UIs.
05 · LOCAL NOTES RAG
Smart ConnectionsFree; bge-micro-v2 locally

Smart Connections runs bge-micro-v2 locally against your Obsidian vault — AI-powered discovery with zero cloud.

TRY →
  • ▸Obsidian users who demand 100% local AI processing for their private notes.
  • ▸Knowledge managers seeking semantic note discovery within a large Obsidian vault.
OR SWAP IN
QUICK START

How to ship it

  1. 1Install Ollama from ollama.com, run `ollama pull llama3.1:8b`
  2. 2Install Continue.dev in VS Code, point it at your local Ollama endpoint
  3. 3For images: install ComfyUI + download an SDXL base model from CivitAI
  4. 4For RAG: open Obsidian, install Smart Connections from Community Plugins, let it index
  5. 5Verify: unplug your wifi — everything should still work
⚠ WATCH OUT

You need real hardware. A $2,000 MacBook Pro M3 runs 8B-70B models comfortably; a $500 laptop will only run 3B-7B. Electricity isn't free either — GPU-heavy generation can add $10-30/mo to your power bill.

Want this stack customized for you?

Ask AIArsenal to tailor it to your use case — budget, scale, privacy needs. It knows every tool in this stack.

ASK AIARSENAL →
RELATED STACKS

More recipes

Ship an AI SaaS in a Weekend
Zero to paying users in 48 hours. Every tool on the free tier.