GGUF
6 posts tagged #gguf
6 posts on GGUF, all in the Local LLM Hardware pillar — most read: “Portable LLM on a USB Stick: Run Ollama or LM Studio Offline” and “LLM Quantization Levels Compared: Q4_K_M vs Q8_0 vs FP16”.
Developer Tools KoboldCpp GGUF Setup Guide: When It Beats Ollama
Run your existing .gguf models in KoboldCpp in under 15 minutes. I’ll show the exact settings for context, GPU layers, streaming, and when Ollama still wins.
Cybersecurity Verify GGUF Model Hashes: 10-Step Supply Chain Workflow
A team-ready workflow to verify GGUF integrity: compute SHA256, require signed manifests, handle mirrors safely, scan sidecars, and ship updates via an internal registry.
AI and Machine Learning LLM Quantization Levels Compared: Q4_K_M vs Q8_0 vs FP16
The practitioner's guide to choosing between Q4_K_M, Q5_K_S, Q8_0, and FP16 quantization for local LLMs — with real perplexity numbers, throughput benchmarks, and per-use-case recommendations.
![GGUF vs GPTQ vs EXL2: LLM Quantization Compared [2026]](https://img.kunalganglani.com/images/vzekdneq/production/34e429bea58e21d5119b5baad6b9efe44974be77-1200x675.webp?w=600&q=75&auto=format)
AI and Machine Learning GGUF vs GPTQ vs EXL2: LLM Quantization Formats Compared
A head-to-head comparison of GGUF, GPTQ, and EXL2 quantization formats with real quality, speed, and VRAM trade-offs — updated for the 2026 Hugging Face acquisition of ggml.ai.
Developer Tools Ollama vs llama.cpp for Local LLMs: Speed, Setup, Control
Ollama wins for developers who want a fast, polished setup with REST APIs and model management. llama.cpp wins for power users squeezing every last token of performance from their hardware.
Technology Portable LLM on a USB Stick: Run Ollama or LM Studio Offline
Run a full LLM from a USB drive with zero internet. Covers Ollama portable setup, LM Studio on external drives, USB Uncensored LLM migration, and GGUF model selection with real performance numbers.
Keep exploring
Close to this topic, not on this page.
