Daily AI news, sourcedUpdates / Chips & infra
Updates Chips & infra · Unconfirmed · 56 items · newest issue first · page 1 of 3
AreaAllModelsAgents & codingApps & productsChips & infraResearchPricing & limitsBusiness & fundingPolicy & safetyOther
- llama-doctor, a new open CLI and live dashboard, reads a llama.cpp server's metrics and grades what is wrongUnconfirmedr/LocalLLaMAOct 19Chips & infra
- chatllm.cpp adds support for InternLM's Intern-Decision models, text only for nowUnconfirmedr/LocalLLaMAOct 19Chips & infra
- Basalt inference engine adds native NVFP4 support for Qwen3.8 Flash-Next on Blackwell GPUs, alongside faster prefill workUnconfirmedr/LocalLLaMAOct 18Chips & infra
- audio.cpp adds Drama3 text-to-speech, running up to about 5x faster than real time on an RTX 5090Unconfirmedr/LocalLLaMAOct 16Chips & infra
- User reports ~79 tok/s running DeepSeek-V4-Flash-0731 on four Radeon AI PRO R9700s via Affinity’s refused four-card layoutUnconfirmedr/LocalLLaMAOct 16Chips & infra
- Developer releases a reverse proxy that wakes or shuts down a local LLM server on incoming requestsUnconfirmedr/LocalLLaMAOct 16Chips & infra
- LocalLLaMA user offloads speculative-decoding drafters to an AMD Strix machine over llama.cpp RPC to stretch two RTX 4060s from 150K to 195K contextUnconfirmedr/LocalLLaMAOct 16Chips & infra
- LocalLLaMA user measures KLD and top-1 agreement of Unsloth Qwen3.8 Flash Next quants on English and Finnish textUnconfirmedr/LocalLLaMAOct 15Chips & infra
- Developer's PulsarLM inference engine drops old, low-attention tokens to keep long chat and agent contexts boundedUnconfirmedr/LocalLLaMAOct 15Chips & infra
- LocalLLaMA user builds three four-GPU Radeon AI Pro R9700 machines on $127 used EPYC CPUs for about $10,700 each and 384GB of VRAMUnconfirmedr/LocalLLaMAOct 15Chips & infra
- LocalLLaMA user benchmarks AMD's MI350P against Nvidia's RTX Pro 6000 Blackwell after tuning AMD kernels to double its out-of-box speedUnconfirmedr/LocalLLaMAOct 15Chips & infra
- LocalLLaMA user reports 176 tokens/s decode and 6,450 tokens/s prefill with Q4 Qwen3.8-Flash-Next on two Radeon AI Pro R9700s using the Radiance engineUnconfirmedr/LocalLLaMAOct 15Chips & infra
- LocalLLaMA user shares Strata settings that run a Q_2 QFN build at 36 tokens per second with 176K context on 8 GB of VRAMUnconfirmedr/LocalLLaMAOct 14Chips & infra
- Developer releases BitRun, a standalone runner for 1-bit LLMs that reports up to 127 tokens per second on an Apple M4 Pro CPUUnconfirmedr/LocalLLaMAOct 14Chips & infra
- Developer demos jitllm moving all of Llama-3.2-3B's layers between GPUs mid-reply in 8 ms without re-running prefillUnconfirmedr/LocalLLaMAOct 14Chips & infra
- Xyntetik Runner, a single-binary C engine for GGUF models, trains LoRA adapters directly on quantized GGUF filesUnconfirmedr/LocalLLaMAOct 14Chips & infra
- LlamaForge v0.16, an open-source browser control panel for upstream llama.cpp, adds local agents whose quotes are checked by codeUnconfirmedr/LocalLLaMAOct 14Chips & infra
- LocalLLaMA post says Qwen 27B runs at about 20 tokens per second on a jailbroken PS5's GPUUnconfirmedr/LocalLLaMAOct 14Chips & infra
- ninfer-ext fork of NInfer serves Gemma 4 and Qwen3.6 on a single RTX 5090 with published prefill speedsUnconfirmedr/LocalLLaMAOct 14Chips & infra
- LocalLLaMA user pairs an M5 Ultra with an M3 Ultra over Thunderbolt 5 RDMA to split GLM-5.3-Flash prefill and decodeUnconfirmedr/LocalLLaMAOct 14Chips & infra