Daily AI news, sourcedUpdates / Chips & infra
Updates Chips & infra · Unconfirmed · 56 items · newest issue first · page 2 of 3
AreaAllModelsAgents & codingApps & productsChips & infraResearchPricing & limitsBusiness & fundingPolicy & safetyOther
- Diskformer.js runs an 80B MoE model in a Chrome tab by paging experts from disk under llama.cpp's WebGPU backendUnconfirmedr/LocalLLaMAOct 14Chips & infra
- A 40-person company serves local AI from one four-GPU RTX Pro 6000 server costing about 55,000 eurosUnconfirmedr/LocalLLaMAOct 14Chips & infra
- User test finds RTX 3090 Founders Edition VRAM hits 112C during LLM token generationUnconfirmedr/LocalLLaMAOct 14Chips & infra
- Sipeed teases MaixStation, saying it will beat NVIDIA's DGX SparkUnconfirmedr/LocalLLaMAOct 13Chips & infra
- SonicBoom, a Python-free C++/CUDA runtime, claims 52 tok/s decode on a laptop RTX 3050 versus 45 for llama.cppUnconfirmedr/LocalLLaMAOct 13Chips & infra
- Developer adds n+2 expert lookahead to SSD-streamed Qwen Flash Next on a 64GB Mac Mini M5Unconfirmedr/LocalLLaMAOct 13Chips & infra
- Developer releases vramble, an open-source arbiter that gives local AI services one GPU lease at a timeUnconfirmedr/LocalLLaMAOct 13Chips & infra
- Developer publishes a lecture and article on building a GPU compiler that lowers PyTorch graphs to CUDAUnconfirmedr/LocalLLaMAOct 13Chips & infra
- LocalLLaMA user ports Gemma 4 E4B EXL3 via exllamav3 as a Jetson Orin alternative to llama.cppUnconfirmedr/LocalLLaMAOct 12Chips & infra
- LocalLLaMA user posts a final zzpanic vllm-radiance build tuned for one R9700 running Qwen3.8-27BUnconfirmedr/LocalLLaMAOct 11Chips & infra
- LocalLLaMA user reports Next Flash IQ3_XXS ~70 tok/s or dual Qwen 3.6 35B-A3B ~145 tok/s on $500 BC-250 boardsUnconfirmedr/LocalLLaMAOct 11Chips & infra
- Basalt inference engine runs Qwen3.8 Flash-Next at 665 tok/s on an RTX 5090 plus 5060 TiUnconfirmedr/LocalLLaMAOct 10Chips & infra
- LlamAmpere update runs Qwen3.8 27B with 200K context and MTP on 12GB Ampere GPUsUnconfirmedr/LocalLLaMAOct 10Chips & infra
- LoRA-over-GGUF recipe trains Qwen3.8-Flash-Next in 40 GiB VRAM without CPU offloadUnconfirmedr/LocalLLaMAOct 9Chips & infra
- LocalLLaMA shares Vulkan llama.cpp GPU benchmarks sorted by price/performanceUnconfirmedr/LocalLLaMAOct 9Chips & infra
- User clusters six mismatched devices to run a 120B model none could load aloneUnconfirmedr/LocalLLaMAOct 9Chips & infra
- Builder runs Qwen3.8-Flash-Next at 60–100 t/s decode on eight Radeon Pro V620s with a custom vLLM forkUnconfirmedr/LocalLLaMAOct 9Chips & infra
- RTX 5070 Ti user reports a multi-hour CUDA job ending in an NVRM GPU lock watchdog errorUnconfirmedr/LocalLLaMAOct 9Chips & infra
- TornadoVM and jitLLM claim about 90% of llama.cpp local NVIDIA performance by compiling Java to CUDAUnconfirmedr/LocalLLaMAOct 9Chips & infra
- audio.cpp update cuts Higgs Audio TTS peak VRAM by about 48% and speeds HTDemucs and PocketTTSUnconfirmedr/LocalLLaMAOct 9Chips & infra