Articles tagged "local-ai"

Running Gemma 4 12B Locally with Speculative Decoding on Ubuntu

10-Jun-2026

Running Gemma 4 12B Locally with Speculative Decoding on Ubuntu

A practical guide to running Gemma 4 12B locally on Ubuntu using llama.cpp, Hugging Face automatic model loading, and speculative decoding with MTP.

Running Gemma 4 31B Faster at Home with Speculative Decoding

15-May-2026

Running Gemma 4 31B Faster at Home with Speculative Decoding

A practical guide to running Gemma 4 31B on a dual-GPU Ubuntu box using llama.cpp TurboQuant, Gemma 4’s MTP assistant head, and speculative decoding to nearly double token throughput.

How We Made RAG Indexing Faster With an Adaptive Embedding Endpoint Pool

11-Apr-2026

How We Made RAG Indexing Faster With an Adaptive Embedding Endpoint Pool

A simple explanation of how to speed up embeddings by routing work across fast and slow local AI endpoints without letting one slow batch block the whole indexing pipeline.