Portable LLM on a USB Stick: Offline AI Setup [2026]

A portable LLM on a USB stick is a fully self-contained large language model (LLM) environment stored on removable media — a USB flash drive or portable SSD — that runs on any compatible computer without installation, internet access, or cloud dependency. Plug it in, launch a single script, and you have a private AI assistant running entirely on the host machine's CPU and RAM.

Key Takeaways

A portable LLM on a USB stick is a fully self-contained local AI setup stored on removable media that runs offline on any compatible computer without installation or cloud access. Plug in the drive, launch a single script, and the LLM uses the host machine’s CPU and RAM, keeping prompts air-gapped and private.

• A USB 3.2 Gen 2 portable SSD (read speeds up to 1,050 MB/s) loads models 5-8x faster than a typical USB 3.0 flash drive (~100-150 MB/s), making it the minimum viable hardware for a smooth portable LLM experience.

• Ollama becomes fully portable by setting the OLLAMA_MODELS environment variable to your USB drive path — no reinstallation needed on each host machine.
• The USB-Uncensored-LLM project (1,900 GitHub stars) was archived on July 4, 2026; its successor, Uncensored-Local-Studio, bundles LLM chat, image generation, speech-to-text, and TTS in one zero-install package.
• A Q4_K_M quantized 7-8B parameter model (roughly 4.5 GB on disk) is the sweet spot for portable use: it fits comfortably on a 16 GB drive and runs at usable speeds on 8 GB of system RAM.
• Every portable LLM setup runs completely air-gapped — no tokens leave the machine, no API keys required, no telemetry phoning home.

The best portable AI setup is the one that doesn't need WiFi, doesn't need admin rights, and doesn't need you to explain it to IT.

The original USB-Uncensored-LLM project by techjarves — which collected 1,900 stars and 415 forks before being archived on July 4, 2026 — proved that demand for offline AI on a USB stick is massive and real. But that project is now read-only, and its successor changes the game significantly. This guide covers everything: hardware selection, making Ollama and LM Studio portable, the migration to Uncensored-Local-Studio, model selection, uncensored LLMs, and actual performance data from different storage media.

If you've been following the local LLM space, you know the tooling has matured fast. I've written about the LM Studio vs Ollama divide and the cost math of local vs cloud inference. A portable USB setup is the logical next step: taking your local AI environment with you.

What Is a Portable LLM and Why Run One from a USB Stick?

A portable LLM is a large language model packaged with its inference engine so that the entire stack — runtime, model weights, configuration — lives on a single removable drive. You plug the drive into a laptop or desktop, run one launcher script, and the model executes using the host machine's CPU, RAM, and (if available) GPU. Nothing gets installed on the host system. Nothing touches the internet.

Why would you want this? Three reasons keep coming up.

Privacy without compromise. Cloud AI means your prompts travel over the wire to someone else's server. A portable LLM on a USB stick is fully air-gapped. No API keys, no telemetry, no data retention policies to read. For anyone handling sensitive documents — legal, medical, financial — this is the only architecture that makes the compliance conversation simple.

Even when you’re fully offline, the moment you add retrieval (docs, tickets, internal wikis) you’re back in the world of data handling rules and traceability. I put the practical redaction, retention defaults, and audit-trail patterns I use in Data Privacy in RAG Redaction and Retention [2026 Playbook].

Workplace restrictions. Engineers working behind corporate firewalls often can't access ChatGPT, Claude, or any cloud AI provider. A USB stick with Ollama and a GGUF model bypasses that entirely because it never makes an outbound connection.

Portability between machines. If you work across multiple computers — a personal laptop, a work desktop, a friend's machine — carrying your AI environment on a drive means consistent tooling everywhere. Same model, same configuration, same chat history.

Based on the benchmark data I maintain at kunalganglani.com/llm-benchmarks, quantization quality cliffs are model-family-specific. A blanket Q4 recommendation is wrong — and that matters even more in a portable context where you're constrained on storage space and need to pick the right quantization level for your specific model family.

Best USB Drives and Portable SSDs for Running a Portable LLM

The storage medium you choose determines whether your portable AI experience feels snappy or painfully slow. Here's what matters and what doesn't.

USB 3.0 flash drives top out around 100-150 MB/s sequential read. That means loading a 4.5 GB Q4_K_M model takes 30-45 seconds before you get your first token. Usable, but you'll feel the wait every time you switch models or cold-start.

USB 3.2 Gen 2 portable SSDs — like the Samsung T7 Shield, SanDisk Extreme Pro, or Kingston XS2000 — deliver 800-1,050 MB/s sequential read. That same 4.5 GB model loads in under 5 seconds. The difference is night and day.

USB 3.2 Gen 2x2 NVMe enclosures push 2,000 MB/s but require a host port that supports the 2x2 spec, which most laptops still don't have. Overkill for now.

Here's the practical comparison:

Once the model is loaded into RAM, token generation speed is identical regardless of storage medium — it's entirely CPU/GPU-bound at that point. The drive speed only affects initial model load time and any disk-based operations like saving chat history.

My recommendation: A 256 GB USB 3.2 Gen 2 portable SSD. It fits 4-5 different quantized models simultaneously, loads fast enough that cold starts don't break your flow, and costs under $50. The Samsung T7 and SanDisk Extreme Pro are both solid choices. Skip the cheap USB 3.0 flash drives unless you're only running a single sub-3B model.

How to Make Ollama Portable on a USB Drive

Ollama (176,000 GitHub stars, 16,900 forks) is the most popular local LLM runtime, and making it portable is surprisingly straightforward. The key is the OLLAMA_MODELS environment variable.

By default, Ollama stores downloaded models in ~/.ollama/models on Linux/macOS or %USERPROFILE%\.ollama\models on Windows. Redirecting this to your USB drive is all it takes to make Ollama portable.

Step-by-step portable Ollama setup:

1. Install Ollama on the host machine (or copy the binary to your USB drive if you want zero installation)
2. Plug in your USB drive — let's say it mounts as /media/usb on Linux or E:\ on Windows
3. Create a models directory on the drive: E:\ollama-models or /media/usb/ollama-models
4. Set the environment variable before launching Ollama:
   • Linux/macOS: export OLLAMA_MODELS=/media/usb/ollama-models
   • Windows (PowerShell): $env:OLLAMA_MODELS = "E:\ollama-models"
   • Windows (CMD): set OLLAMA_MODELS=E:\ollama-models
5. Pull a model: ollama pull phi3:mini or ollama pull qwen2:7b
6. The model weights now live on your USB drive

When you plug the same drive into a different machine that has Ollama installed, just set OLLAMA_MODELS to point to the same path and the models are immediately available. No re-downloading. No duplication.

For a truly zero-install approach, you can also copy the Ollama binary itself onto the USB drive and launch it directly. On Linux, Ollama is a single static binary. On Windows, you can extract the installer contents. This means you don't need admin rights or an existing Ollama installation on the host machine at all.

I've compared Ollama to alternatives extensively in my Ollama vs llama.cpp breakdown and the Ollama vs LM Studio comparison. For portable use specifically, Ollama's single-binary architecture and the OLLAMA_MODELS redirect make it the easiest path.

How to Make LM Studio Portable on a USB Drive

LM Studio takes a different approach. It's a GUI-first application that runs GGUF models via llama.cpp (and MLX on Apple Silicon). Making it portable requires redirecting its models directory to your external drive.

LM Studio supports macOS (Apple Silicon + Intel x64), Windows (x64/ARM64), and Linux (x64). By default, it stores models in a platform-specific location — but you can point it to any directory, including one on a USB drive.

Setting up LM Studio on an external drive:

1. Install LM Studio on the host machine
2. Open Settings → General → Models Directory
3. Change the path to your USB drive location (e.g., E:\lm-studio-models)
4. Download models through the LM Studio UI — they'll save directly to the USB drive
5. When moving to another machine, install LM Studio, point it to the same directory, and your models appear instantly

The advantage of LM Studio for portable use is its built-in model browser — you can search and download GGUF models from Hugging Face directly through the UI. The disadvantage is that LM Studio itself still needs to be installed on each host machine, unlike a fully self-contained USB setup.

For the full feature comparison, see my detailed LM Studio vs Ollama 2026 post. The short version: Ollama is better for portable CLI workflows and automation; LM Studio is better if you want a visual interface and don't mind installing the app on each host.

USB Uncensored LLM: What It Was and What Replaced It

This is the section most people searching for "USB uncensored LLM" actually need, so let me be direct about the current state.

The USB-Uncensored-LLM project by techjarves was a zero-install, portable local AI environment that ran uncensored LLMs — Gemma, Qwen, NemoMix models — directly from any USB 3.0+ drive or SSD. It was fully air-gapped, cross-platform (Windows, macOS, Linux), and required no system-level changes. The project accumulated 1,900 stars and 415 forks on GitHub.

On July 4, 2026, the project was officially archived. It's now read-only. No more updates, no more bug fixes.

The project used a clever "Shared" volume architecture: model weights (5 GB+) were downloaded once and shared across Windows, macOS, and Linux OS launchers stored on the same drive. This eliminated duplication and saved precious portable storage — a design decision that mattered when you're working with a 16 GB or 32 GB USB stick.

System requirements were reasonable: minimum 8 GB free storage on the USB drive (16 GB recommended), 8 GB system RAM for 2B/4B parameter models, and 16 GB RAM for 9B/12B models.

Here's the setup demo from the original project:

What replaced it: Development moved to Uncensored-Local-Studio, which is significantly more capable.

Uncensored-Local-Studio: The 2026 Replacement Setup Guide

Uncensored-Local-Studio by techjarves is the successor project, and it's not just a rename — it's a major upgrade. Instead of just running LLMs, it bundles four offline AI capabilities in one zero-setup package:

1. LLM chat — GGUF models via llama.cpp, same as before
2. Stable Diffusion image generation — create images locally, no Midjourney or DALL-E needed
3. Whisper speech-to-text — transcribe audio entirely offline
4. Kokoro-82M text-to-speech — generate natural-sounding speech locally

All of this runs from a single folder with no global system changes. Currently at 561 stars and 150 forks, it's growing fast.

The killer feature for portability is auto-detection of GPU backends. Uncensored-Local-Studio detects and loads the optimal acceleration at runtime: CUDA for NVIDIA GPUs, ROCm for AMD GPUs, Vulkan for Intel/AMD/NVIDIA, Metal for macOS, and OpenVINO for Intel NPUs. The same USB drive accelerates correctly on any host machine — plug it into a Windows desktop with an NVIDIA card, and it uses CUDA; plug it into a MacBook, and it uses Metal.

The integrated model manager accepts Hugging Face URLs directly, so you can pull any GGUF model without leaving the interface. It also includes a live CPU/RAM/GPU/VRAM performance monitor, which is genuinely useful when you're trying to figure out whether a model fits in the host machine's available memory.

Setup on Windows: Download the latest release, extract to your USB drive or any folder, and run the launcher. No Python installation, no pip packages, no PATH variables.

Setup on Linux/macOS: Same idea — extract, run the launcher script. The project bundles its own portable Python runtime, so it doesn't touch your system Python.

If you're migrating from USB-Uncensored-LLM, your existing GGUF model files are compatible. Copy them into the new project's models directory and they'll appear in the model selector immediately.

Best GGUF Models for a USB Stick: Size vs Quality

Picking the right model for portable use is a constrained optimization problem: you're balancing file size (limited by USB drive capacity), RAM requirements (limited by the host machine), and output quality. Here's the decision table no competitor provides.

The sweet spot for most portable setups is a 7-8B parameter model at Q4_K_M quantization. You get genuinely useful output quality in a ~4.5 GB file that loads fast from a portable SSD and runs at 8-15 tokens per second on a modern laptop with 8 GB RAM.

For a deeper dive into quantization tradeoffs, see my GGUF vs GPTQ vs EXL2 comparison. The short version: Q4_K_M offers the best quality-to-size ratio for most model families, but Q5_K_M is worth the extra ~20% file size if you have the storage space.

If your USB drive is only 8 GB, stick with Phi-3.5-mini or Gemma 2 2B at Q4_K_M. If you have 32 GB or more, you can carry multiple models and switch between them depending on the task.

Running Uncensored LLMs from USB: Models and Legal Notes

Uncensored LLMs are models that have been trained or fine-tuned without alignment-based refusals — they don't refuse prompts about sensitive topics the way ChatGPT or Claude would. Running them locally from a USB stick is the most private way to use these models.

Popular uncensored models for portable use:

• NemoMix Unleashed — community-favorite uncensored merge, strong general capability
• Gemma 2 abliterated variants — Google's Gemma with alignment restrictions removed through abliteration techniques
• Qwen uncensored — Alibaba's Qwen with refusal training removed
• Dolphin (Mistral/Llama-based) — Eric Hartford's widely-used uncensored fine-tuning series
• WizardLM Uncensored — uncensored versions of WizardLM models

All of these are available in GGUF format on Hugging Face and work with Ollama, LM Studio, and Uncensored-Local-Studio.

Legal reality: In most jurisdictions, downloading, possessing, and running uncensored AI models is legal. The models themselves are weights files — mathematical parameters. What's regulated is the use, not the possession. Generating illegal content (CSAM, credible threats, etc.) is illegal regardless of the tool used to create it. Using an uncensored model for legitimate research, creative writing, red-teaming AI security, or simply getting unfiltered answers to reasonable questions is perfectly legal in the US, EU, Canada, and most other jurisdictions.

The reason people run uncensored models locally rather than through cloud services isn't usually nefarious — it's that cloud providers over-censor. Ask Claude or ChatGPT to help write a thriller novel with violence and you'll hit refusals. Ask for security research assistance and you'll get lectured. A local AI setup removes that friction.

Cross-Platform Use: Same USB on Windows, Mac, and Linux

One of the most compelling aspects of a portable LLM on a USB stick is running the same drive across different operating systems without any reconfiguration.

File system matters. Format your USB drive as exFAT — it's natively readable on Windows, macOS, and Linux without additional drivers. NTFS works on Windows and Linux but requires extra software on macOS. HFS+ and APFS are macOS-only. ext4 is Linux-only without third-party tools.

The USB-Uncensored-LLM project solved cross-platform support elegantly: it stored OS-specific launchers (Windows batch files, macOS/Linux shell scripts) alongside a shared model directory. Model weights — which are just binary files — are OS-agnostic. A GGUF file is a GGUF file whether you're on Windows, Mac, or Linux.

Uncensored-Local-Studio takes this further with its GPU auto-detection. The same USB drive carries CUDA, ROCm, Vulkan, Metal, and OpenVINO backends, and the launcher picks the right one at runtime based on what hardware it detects. You don't need separate drives for your NVIDIA desktop and your MacBook.

For the Ollama portable approach, the models directory structure is identical across platforms. Set OLLAMA_MODELS to the drive path on whatever OS you're using, and the same downloaded models just work.

One practical note: if you're working across machines with different RAM capacities, keep both a small model (2-3B) and a larger model (7-8B) on the drive. The small model serves as a fallback for machines with only 4-8 GB RAM, which you'll encounter more often than you'd expect when borrowing someone else's computer.

Performance: Token Speed from USB Flash vs Portable SSD vs Internal Drive

Here's the performance question everyone asks: how much slower is running a model from USB compared to an internal drive?

The answer has two parts, and one of them surprises people.

Model loading speed scales linearly with drive read speed. A 4.5 GB model loading from a USB 3.0 flash drive (100 MB/s) takes ~45 seconds. From a USB 3.2 Gen 2 SSD (1,000 MB/s) it takes ~5 seconds. From internal NVMe (3,500 MB/s) it takes ~1.3 seconds.

Token generation speed is identical regardless of storage medium. Once the model is loaded into system RAM, the USB drive is out of the loop entirely. Token generation is purely CPU/GPU-bound. A Llama 3.1 8B Q4_K_M running on a modern laptop CPU generates 8-15 tokens per second whether the model was loaded from a USB flash drive or internal NVMe.

Based on the benchmark data I maintain at kunalganglani.com/llm-benchmarks, unified memory on Apple Silicon changes the typical VRAM-is-the-bottleneck assumption — big models load on Apple Silicon machines but throughput becomes the real constraint. For portable use on a MacBook, you can run surprisingly large models from USB because the unified memory architecture doesn't have the VRAM ceiling that discrete GPU systems do.

So the practical impact of drive speed is felt only during cold starts and model switches. If you're chatting with a single model for an extended session, even a USB 3.0 flash drive is fine after the initial load. If you're frequently switching models or restarting, invest in a portable SSD.

The local LLM hardware requirements still apply regardless of storage medium: you need enough system RAM to hold the full quantized model. A 4.5 GB Q4_K_M model needs roughly 6-8 GB of free RAM to run smoothly (the model plus inference overhead). There's no getting around this with a faster USB drive.

Troubleshooting Common Portable LLM Problems

After configuring portable setups and reading through dozens of GitHub issues on both USB-Uncensored-LLM and Uncensored-Local-Studio, here are the problems that come up most often:

"Model loads but inference is incredibly slow" (1-2 tok/s on decent hardware). This almost always means the model doesn't fit in RAM and is swapping to disk. Check available RAM with Task Manager or htop. If you have 8 GB total and your OS is using 3 GB, you only have ~5 GB free — not enough for a 7B Q4_K_M model with overhead. Drop to a 2-3B model.

"Permission denied" or "can't write to USB drive" on Linux. Mount the drive with write permissions. exFAT on Linux requires the exfat-fuse and exfat-utils packages. Or format as ext4 if you don't need Windows/Mac compatibility.

"Ollama can't find models after moving to a different machine." The OLLAMA_MODELS variable wasn't set on the new machine. It's per-session — you need to set it every time, or add it to your shell profile. On Windows, you can create a batch file on the USB drive that sets the variable and launches Ollama in one step.

"Model file is corrupted after copying to USB." This happens with cheap USB flash drives that silently drop data during large file transfers. Verify the file hash after copying. GGUF files are large single files — any corruption makes them unusable. This is another reason to prefer a quality portable SSD over a $10 flash drive.

"GPU acceleration doesn't work from USB." The GPU drivers must be installed on the host machine. A portable LLM setup can carry the inference engine but not the GPU drivers — those require system-level installation. On machines where you don't have admin rights, CPU inference is your only option. Expect 5-15 tok/s on a 7B model with a modern CPU, which is slower than GPU but perfectly usable for chat.

Best GitHub Projects for Portable AI [2026]

Beyond the two main projects covered above, several other GitHub repositories serve the portable AI use case:

• [Uncensored-Local-Studio](https://github.com/techjarves/Uncensored-Local-Studio) (561 stars) — The current best all-in-one solution. LLM + image generation + STT + TTS, zero-install, auto GPU detection. Active development.
• [USB-Uncensored-LLM](https://github.com/techjarves/USB-Uncensored-LLM) (1,900 stars, archived) — The original project. Still functional but no longer updated. Use Uncensored-Local-Studio instead.
• [Ollama](https://github.com/ollama/ollama) (176,000 stars) — Not designed specifically for USB use, but the OLLAMA_MODELS variable trick makes it portable. Best option if you want CLI-first workflows.
• Llamafile by Justine Tunney — Mozilla-backed project that packages a model and inference engine into a single executable file. One file, runs anywhere. Different philosophy from USB-based setups but serves a similar use case. I've compared it in my Ollama vs Llamafile post.
• Koboldcpp — Portable llama.cpp-based inference with a web UI. Single executable, can run from USB. Focused on creative writing and roleplay use cases.

For the most up-to-date list of self-hosted AI tools, check the curated roundup on this site.

Can You Run a Portable LLM on Android with Termux?

Yes — and this is an angle nobody else covers. The USB-Uncensored-LLM project's folder architecture actually included Android/Termux as a target platform.

Termux is a terminal emulator for Android that provides a Linux environment without root. You can install Ollama or compile llama.cpp inside Termux and point it to model files on the device's storage or a connected USB drive (via OTG adapter).

Practical limitations are real: most Android phones have 6-8 GB RAM (with 2-3 GB consumed by the OS), so you're limited to 2-3B parameter models. Inference speed on a phone's ARM CPU is slow — expect 3-8 tokens per second on a Snapdragon 8 Gen 3 with a 2B model. But it works, and it's genuinely useful for quick questions when you don't have a laptop handy.

This connects to the broader trend of on-device AI running on increasingly diverse hardware. The same GGUF model file that runs on your desktop, your laptop, and your USB stick can also run on your phone.

What Comes Next for Portable AI

The trajectory here is clear. Models are getting smaller and better — a 2026 3B model outperforms a 2024 7B model on most benchmarks. Storage is getting cheaper and faster. USB4 drives pushing 4,000 MB/s will make model loading instantaneous. And tools like Uncensored-Local-Studio are bundling increasingly sophisticated AI capabilities into zero-install packages.

The most interesting development is the convergence of portable AI with agentic AI. Right now, a portable LLM on USB is mostly a chat interface. But agent frameworks that can run offline — tool use, RAG over local documents, multi-step reasoning — are the obvious next step. Imagine plugging a USB drive into any machine and having a full AI agent that can read your files, search your notes, and execute tasks, with zero cloud dependency.

If you're building portable AI setups today, start with a 256 GB USB 3.2 Gen 2 portable SSD, Ollama with OLLAMA_MODELS pointed to the drive, and a Llama 3.1 8B Q4_K_M model. Total cost: under $50 for the drive plus $0 for the software. Total setup time: under 10 minutes. And you'll have a private, offline AI assistant that runs on any computer you can plug a USB cable into.

The cloud isn't going anywhere. But the option to carry your AI in your pocket — no internet, no account, no subscription — is a freedom worth having.