Is it true that tODO(en): Code and tool calls can peak up to 275 tok/s - Reasoning traces up to 190 tok/s Hardware: 5080 16GB PCIe gen5x16, 64 DDR5 6400, Core Ultra 9 285K https://github.com/jesdga95/basalt Some PRs for the optimizations under my name are still pending (xyehya) but upstream already have solid gains & memory optimizations?
UnconfirmedLast checked
First seen
Current progress
No conclusive verdict. TODO(en): Code and tool calls can peak up to 275 tok/s - Reasoning traces up to 190 tok/s Hardware: 5080 16GB PCIe gen5x16, 64 DDR5 6400, Core Ultra 9 285K https://github.com/jesdga95/basalt Some PRs for the optimizations under my name are still pending (xyehya) but upstream already have solid gains
TODO(en): Code and tool calls can peak up to 275 tok/s - Reasoning traces up to 190 tok/s Hardware: 5080 16GB PCIe gen5x16, 64 DDR5 6400, Core Ultra 9 285K https://github.com/jesdga95/basalt Some PRs for the optimizations under my name are still pending (xyehya) but upstream already have solid gains & memory optimizations
TODO(en): Code and tool calls can peak up to 275 tok/s - Reasoning traces up to 190 tok/s Hardware: 5080 16GB PCIe gen5x16, 64 DDR5 6400, Core Ultra 9 285K https://github.com/jesdga95/basalt Some PRs for the optimizations under my name are still pending (xyehya) but upstream already have solid gains