Daily AI news, sourcedUpdates / Chips & infra
Updates Chips & infra · 115 items · newest issue first · page 3 of 6
AreaAllModelsAgents & codingApps & productsChips & infraResearchPricing & limitsBusiness & fundingPolicy & safetyOther
- llama.cpp adds --spec-draft-n-ubatch to set the draft context's batch size and save VRAM in speculative decoding
- LocalLLaMA user shares Strata settings that run a Q_2 QFN build at 36 tokens per second with 176K context on 8 GB of VRAMUnconfirmedr/LocalLLaMAOct 14Chips & infra
- Developer releases BitRun, a standalone runner for 1-bit LLMs that reports up to 127 tokens per second on an Apple M4 Pro CPUUnconfirmedr/LocalLLaMAOct 14Chips & infra
- MarkTechPost ranks nine AI gateways for 2026, with Bifrost first ahead of Portkey and LiteLLM, per MarkTechPost
- Developer demos jitllm moving all of Llama-3.2-3B's layers between GPUs mid-reply in 8 ms without re-running prefillUnconfirmedr/LocalLLaMAOct 14Chips & infra
- KoboldCpp 1.123 adds Qwen Image 2.1 support, approval modes for its built-in agent and full batched-mode featuresOfficialGitHub @LostRuinsOct 14Chips & infra
- Xyntetik Runner, a single-binary C engine for GGUF models, trains LoRA adapters directly on quantized GGUF filesUnconfirmedr/LocalLLaMAOct 14Chips & infra
- NVIDIA publishes its Run:ai GPU orchestration platform for Kubernetes on GitHub under Apache 2.0OfficialNVIDIA Technical BlogOct 14Chips & infra
- LlamaForge v0.16, an open-source browser control panel for upstream llama.cpp, adds local agents whose quotes are checked by codeUnconfirmedr/LocalLLaMAOct 14Chips & infra
- Macau Customs seizes two GeForce RTX 5090 cards alongside cigarettes in a smuggling crackdown, per VideoCardzReportedVideoCardzOct 14Chips & infra
- LocalLLaMA post says Qwen 27B runs at about 20 tokens per second on a jailbroken PS5's GPUUnconfirmedr/LocalLLaMAOct 14Chips & infra
- Mesa merges support for running YOLOX object detection with PyTorch on NPUs, per Phoronix
- Mynet's CTO says PolarDB Serverless scales its AI-driven analytics workloads without provisioning for peaks
- ninfer-ext fork of NInfer serves Gemma 4 and Qwen3.6 on a single RTX 5090 with published prefill speedsUnconfirmedr/LocalLLaMAOct 14Chips & infra
- LocalLLaMA user pairs an M5 Ultra with an M3 Ultra over Thunderbolt 5 RDMA to split GLM-5.3-Flash prefill and decodeUnconfirmedr/LocalLLaMAOct 14Chips & infra
- AnythingLLM Desktop replaces its bundled Ollama engine with llama.cpp managed by AMD's LemonadeOfficialAnythingLLMOct 14Chips & infra
- Diskformer.js runs an 80B MoE model in a Chrome tab by paging experts from disk under llama.cpp's WebGPU backendUnconfirmedr/LocalLLaMAOct 14Chips & infra
- A 40-person company serves local AI from one four-GPU RTX Pro 6000 server costing about 55,000 eurosUnconfirmedr/LocalLLaMAOct 14Chips & infra
- User test finds RTX 3090 Founders Edition VRAM hits 112C during LLM token generationUnconfirmedr/LocalLLaMAOct 14Chips & infra
- Sipeed teases MaixStation, saying it will beat NVIDIA's DGX SparkUnconfirmedr/LocalLLaMAOct 13Chips & infra