All updates · Unconfirmed

Official news, first rumor reports and substantive progress. Each entry keeps the information published at that time.

• Georgi Gerganov says llama.cpp appeared on stage at Microsoft's Windows event, per a Reddit link post

  A link post on r/LocalLLaMA shares an X post from llama.cpp creator Georgi Gerganov saying it was 'very cool' to see llama.cpp on the big stage at the day's Windows event. He wrote that the software and hardware stacks are finally coming together after years of community work and that he expects more users to embrace local AI.

  UnconfirmedPublished newsOct 7Source: r/LocalLLaMAPart of llama.cpp
• llama.cpp pull request adds a GPU cache for MoE experts kept in host memory, with up to 2.2x faster decode

  A llama.cpp pull request by am17an ports qvac-fabric's MoE expert cache, keeping an LRU cache of experts on the GPU and uploading only misses when experts live in host memory, enabled with --moe-cache-mib. Benchmarks on Qwen3.8-Flash-Next Q4_0 show 1.57-1.62x faster token generation on an RTX 4090 and up to 2.2x on an RTX 5090, per a post on r/LocalLLaMA. It currently supports single-GPU inference.

  UnconfirmedPublished newsOct 7Source: r/LocalLLaMAPart of llama.cpp
• llama.cpp pull request adds GLM5Next multi-token prediction for running GLM 5 Flash locally

  A llama.cpp pull request by pwilkin adds multi-token prediction (MTP) support and optimizations for GLM5Next, so users can run GLM 5 Flash with MTP locally, according to a post on r/LocalLLaMA.

  UnconfirmedPublished newsOct 7Source: r/LocalLLaMAPart of llama.cpp
• User finds mainline llama.cpp beats ik_llama.cpp on a hybrid multi-GPU rig, per a Reddit post

  A post on r/LocalLLaMA says mainline llama.cpp consistently outperforms the ik_llama.cpp fork on the author's hybrid multi-GPU setup, despite recommendations to the contrary.

  UnconfirmedPublished newsOct 6Source: r/LocalLLaMAPart of llama.cpp

RSS for this view →