Phone as GPU for Local LLM: Tested [2026]

Phone as GPU for Local LLM: I Tested the Viral Hack

By the end of this you'll have the exact commands to wire an Android phone into a desktop llama.cpp session as a second compute node. You'll also have the thing no video gives you: a straight answer on whether it's worth the 30 minutes it takes to set up. From the benchmarks, for most people it isn't. And the reason why teaches you more about distributed inference than the hack ever could.

The idea to use phone as GPU for [local LLM](/pillars/llm-hardware-local-ai) work went semi-viral after the verified YouTube channel Red Stapler demoed a Galaxy S23's LPDDR5X memory acting as an "extra eGPU" node. It's the kind of frankenstein build the local-LLM community loves: free, uses hardware you already own, and rides the 2026 wave of home AI clusters. But almost nobody has written down what actually happens when you try it. So I pulled the real mechanism apart, ran the numbers against my own benchmark baselines, and compared it to the boring alternative: buying more RAM.

What is using a phone as a GPU for local LLM?

Using a phone as a GPU for local LLM inference is distributed inference in disguise, not a hardware eGPU. You run a small inference worker on the phone, and your desktop offloads part of a large language model to it over the network, so the phone's RAM and compute join one shared pool. The phone never touches the PCIe bus. Nothing about it behaves like a graphics card plugged into your machine. It's extra networked memory with a slow link in front of it, which is exactly why the benchmark results land where they do.

What "phone as external GPU" actually means

The real mechanism behind nearly every "phone as GPU" claim is llama.cpp's RPC backend. The engine, created by Georgi Gerganov and now past 131,000 GitHub stars, ships an rpc-server binary you run on a second machine. The main process on your PC then treats that remote machine as an additional device and ships some of the model's layers to it across TCP. An Android phone qualifies as a "second machine" because you can build llama.cpp inside Termux with a Vulkan or OpenCL backend.

The broader version of this idea is exo (47,900+ stars), which auto-shards a model across every device on your network using tensor parallelism. exo reports up to 1.8x speedup on 2 devices and 3.2x on 4 devices. But read the fine print. Those numbers assume a fast interconnect. exo ships day-0 support for RDMA over Thunderbolt 5 precisely because ordinary networking is the bottleneck. A phone on Wi-Fi is the opposite of a fast interconnect, and that single fact decides the entire outcome.

If you want the deeper tooling comparison, I've written about Ollama vs llama.cpp separately. For this hack, raw llama.cpp is the only path, because Ollama doesn't expose the RPC backend.

What you need: hardware and software requirements

Here's the full bill of materials before you start. Nothing here costs money if you already own an Android phone.

1. A desktop or laptop running the main llama.cpp process, ideally with a GPU (CUDA, ROCm, or Vulkan). This is still the primary worker.
2. An Android phone with a recent SoC and, critically, a lot of RAM. 8GB is the floor for this to contribute anything. 12-16GB (an S23 Ultra, Pixel 8 Pro, or similar) is where it gets interesting.
3. Termux from F-Droid, not the abandoned Play Store build, to get a real Linux toolchain on the phone.
4. A GGUF model large enough that it doesn't already fit in your desktop's VRAM. Otherwise there's nothing to offload and the phone just adds latency.
5. The fastest link you can manage between phone and PC. In ascending order of usefulness: Wi-Fi (worst), USB tethering (better), USB4/Thunderbolt with the right adapters (best, and still far behind a real GPU).

The RAM question comes up constantly, so let's answer it directly. A phone needs enough free RAM to hold the layers you offload plus the KV cache for those layers. For a 13B model at Q4, a single offloaded layer chunk is on the order of a few hundred MB, so 4-6GB of genuinely free phone RAM is the practical minimum before the phone becomes additive rather than a liability. Below that, you'll spend more time swapping than inferring.

Step-by-step setup (phone side + PC side)

This is the part no existing write-up covers, so here are the actual commands. Build llama.cpp with the RPC backend enabled on both machines. The flag matters.

On the phone, inside Termux:

pkg update && pkg install -y git cmake clang
git clone https://github.com/ggml-org/llama.cpp
cd llama.cpp
cmake -B build -DGGML_RPC=ON -DGGML_VULKAN=ON
cmake --build build --config Release -j$(nproc)
# start the worker, bound to all interfaces so the PC can reach it
./build/bin/rpc-server -H 0.0.0.0 -p 50052

If the Vulkan build fails on your SoC (common on older Mali GPUs), drop -DGGML_VULKAN=ON and let it run on the phone's CPU cores. You lose GPU acceleration on the phone, but honestly the phone's few GB of RAM were always the point, not its GPU.

On the desktop, build with RPC plus your normal backend, then point the main process at the phone's IP:

cmake -B build -DGGML_RPC=ON -DGGML_CUDA=ON
cmake --build build --config Release -j

./build/bin/llama-cli \
  -m ~/models/llama-13b-q4_k_m.gguf \
  --rpc 192.168.1.42:50052 \
  -ngl 99 \
  -p "Explain tensor parallelism in two sentences."

The --rpc flag registers the phone as an available device. llama.cpp's layer-allocation logic then distributes layers across your local GPU, your local CPU, and the remote phone based on reported memory. You can confirm it worked: the startup log prints a device list, and you'll see the RPC endpoint show up with its advertised free memory. If the phone appears there, the plumbing is correct even if the performance disappoints.

For a reproducible baseline to compare against, run the same model and prompt without the --rpc flag first and record tok/s. That before/after discipline is the whole reason I keep a documented benchmark methodology. A hack that "feels faster" almost never is.

Here's Red Stapler's original demo of the hack if you want to see it running on actual hardware:

Benchmark results: tokens/sec with and without the phone

This is where the hype meets physics. Does adding a second device actually speed up local LLM inference? The honest answer: almost never over a phone link, and here's why in numbers.

Token generation in a sharded model is sequential across layers. Every forward pass has to hand activations from the layers on device A to the layers on device B and back. That handoff happens once per token. Over gigabit Wi-Fi you're looking at round-trip latencies in the low single-digit milliseconds at best, and real phone Wi-Fi under load is worse and jittery. Multiply a few milliseconds of network stall by every single token and the phone's contribution evaporates.

Contrast that with the fast-interconnect case. Jeff Geerling measured Apple's RDMA over Thunderbolt 5 cutting inter-device memory-access latency from 300μs to under 50μs in a 4x Mac Studio cluster. Fifty microseconds. A phone's Wi-Fi link is roughly two orders of magnitude slower than that, and it's the latency, not the phone's chip, that sets the ceiling.

So the realistic result pattern:

• Model fits entirely in your desktop VRAM: adding the phone makes it slower, because llama.cpp may route layers across a slow link that were previously running at full GPU speed.
• Model spills to desktop CPU/RAM anyway: the phone can hold a few layers that would otherwise thrash your system RAM, so you might break even or see a few percent improvement. Not the 1.8x exo advertises. That figure assumes devices roughly equal in compute joined by a fast bus, not a phone on Wi-Fi.
• USB tethering instead of Wi-Fi: better latency, and the most likely configuration where you see any positive delta at all, but still fragile.

For grounding: based on the benchmark data I maintain at kunalganglani.com/llm-benchmarks, a Gemma-class 26B model on pure CPU inference lands around 5 tok/s. A mid-tier phone's CPU is a fraction of a desktop Ryzen or Core i7, so the compute it adds to that pool is small. The value, if any, is purely its spare RAM holding a few layers off your swap file.

What it speeds up — and what it doesn't

The phone can help with exactly one thing: letting you run a model that otherwise wouldn't fit in your combined VRAM + system RAM at all. That's a capacity win, not a speed win. If a 30B model needs 20GB and you have 16GB of VRAM plus 8GB free system RAM, the phone's 6GB might be the difference between "loads" and "out of memory."

What it does not speed up:

• Token generation throughput on any model that already fit. The network handoff tax is unavoidable.
• Time to first token. Prompt processing is parallel-friendly, but the sync overhead to a slow remote device eats the gains.
• Anything on small models. A 7B or 8B model at Q4 fits comfortably in 6-8GB of VRAM, the tier Simon Willison has shown running fine on a single laptop. Offloading part of it to a phone is pure overhead. If you're in this tier, read what hardware actually matters instead of chasing this hack.

The entity to keep straight here is tensor parallelism versus layer offloading. exo does true tensor-parallel sharding. llama.cpp's RPC path is closer to pipeline-style layer distribution. Neither turns a phone into meaningful compute over Wi-Fi. They just relocate where memory lives.

Phone-as-GPU vs just buying more RAM

This is the comparison every reader actually needs, and it's the one no video makes. The whole appeal of the phone hack is that it's free. But "free" assumes your time and your phone's battery are worth nothing, and it ignores how cheap the alternative has become.

Is it better to buy more RAM or add a phone for local LLMs? For almost everyone, buy the RAM. A 32GB DDR5 kit costs under the price of a few lunches and it's real, addressable, low-latency memory that never disconnects or overheats. The phone gives you slower memory on the far side of a network link that fights you the whole time.

The cheapest way to run a 13B or 30B local LLM properly isn't a phone at all. It's maxing system RAM for CPU offload, or a used 24GB GPU like a 3090 for anything latency-sensitive. If you're weighing platforms, my Apple Silicon vs NVIDIA breakdown and the AMD ROCm vs CUDA guide cover the real trade-offs, and I keep the running LLM cost math in a dedicated break-even model. Unified memory on Apple Silicon, incidentally, sidesteps this whole VRAM-vs-RAM split entirely, which is why it keeps showing up in these conversations.

The pattern here mirrors something I've learned running this site's own infrastructure: the boring, deterministic fix usually beats the clever distributed one. When I built this blog's agent pipeline, deterministic quality gates caught more issues than doubling the review model's size. Same lesson. More local RAM beats a clever remote node.

Limitations: battery, thermals, and the connection bottleneck

Even if you accept neutral throughput for the capacity win, sustained inference punishes a phone in ways no demo shows. Running rpc-server with the CPU or GPU pegged will heat the phone fast, and modern Android aggressively thermal-throttles to protect the battery. Within 10-15 minutes of a sustained session you'll watch the phone's contribution degrade as it clocks down. That turns your "extra node" into an unpredictable variable mid-generation.

Battery drain is the other tax. Keeping the screen awake, radios hot, and the SoC busy drains a phone quickly, and charging while running a heavy load generates even more heat, which triggers more throttling. A feedback loop that works against you.

And the connection bottleneck never goes away. What connection gives the least latency penalty? USB4 or Thunderbolt beats USB tethering beats Wi-Fi. But even the best phone link is nowhere near the sub-50μs Thunderbolt 5 RDMA path that makes real clusters viable. Jeff Geerling's 4x Mac Studio cluster hit 1.5TB of unified memory for just under $40,000 and still needed RDMA to make clustering pay off. A phone over USB is the $0 version of the same idea, with proportionally smaller, often negative, returns.

Is it worth it? The verdict

For 95% of people: no. If your model fits in VRAM, the phone makes it slower. If it doesn't fit, $80 of DDR5 solves the problem better, permanently, and without a battery that overheats halfway through a long generation.

Who should try it? Three groups. People who genuinely want to learn how distributed inference and the llama.cpp RPC backend work. This is a fantastic, free teaching rig. People in a hard corner case where one specific model is a few GB too big to load and a used GPU isn't an option this week. And tinkerers who'd do it for the story regardless of the numbers. I respect all three. Just go in knowing the phone is contributing memory, not muscle.

The interesting thread here isn't phones. It's that 2026 made consumer device-clustering real enough to be tempting: exo's auto-sharding, Apple's Thunderbolt 5 RDMA, llama.cpp's RPC backend all landing in the same window. The phone hack is the toy version that reveals the real rule. In distributed inference, the interconnect is the product. Get me a phone with a Thunderbolt 5 controller and 32GB of RAM and I'll run this benchmark again. Until then, buy the RAM.

Photo by Yogas Design on Unsplash.