Inference
4 posts tagged #inference
4 posts on Inference, all in the Local LLM Hardware pillar — most read: “How to Run Qwen 3.8 Flash Next 125B in Strata on an RTX 4090” and “Phone as GPU for Local LLM: Tested vs Buying RAM”.
AI and Machine Learning Phone as GPU for Local LLM: Tested vs Buying RAM
Can a Galaxy S23 act as an extra GPU node for local LLM inference? I break down the real llama.cpp RPC mechanism, the setup, and whether it beats just buying RAM.
AI and Machine Learning How to Run Qwen 3.8 Flash Next 125B in Strata on an RTX 4090
A reproducible Strata setup for Qwen 3.8 Flash Next (125B) on RTX 4090, plus the KV cache knobs that decide whether you hit ~100 tok/s or hit OOM.
Developer Tools ds4 DwarfStar Review: Redis-Style Local LLM Loops
ds4 (DwarfStar 4) turns local LLM inference into a Redis-like workflow: one long-lived server, deterministic prompt-keyed KV caching, and fast edit-run-edit loops without a GUI circus.
![nvidia gpu graphics card close-up — illustration for article on Local LLM Break-Even Math [2026]: Power,](https://img.kunalganglani.com/images/vzekdneq/production/76beafb78b6f19b7ed6235232857078a023801c2-1200x675.webp?w=600&q=75&auto=format)
AI and Machine Learning Local LLM Break-Even Math: Power, Idle, Depreciation
Most “GPU price ÷ tokens” break-even math is wrong. Here’s a spreadsheetable local LLM total-cost model that includes idle power, utilization, depreciation, failures, and opportunity cost.
Keep exploring
Close to this topic, not on this page.
