On Oct 27, Ai2 released Old Ideas, Novel Problems study and benchmark on LLM novelty judges.
Official source: Ai2 · DailyCited record
- Official
Ai2 releases Old Ideas, Novel Problems study and benchmark on LLM novelty judges
Ai2 published this on Oct 27, 2026 (source: huggingface.co).
Ai2’s Hugging Face blog says large language model “novelty judges” are brittle: across GPT and Claude judges, small prompt changes shifted accuracy by more than 50 points on some AI-generated idea pairs, and purpose-built novelty checkers underperformed a simple prompted Claude Opus 4.6 baseline. Ai2 releases the study, benchmark, data, code, and a pipeline built from ICLR 2026 peer-review novelty signals.
Source: Ai2 - Official
Ai2 replaces its priority-based GPU scheduler with time budgets and fair-share allocation
Hugging Face @allenai published this on Oct 9, 2026 (source: huggingface.co).
Ai2's AI infrastructure team says it replaced a priority-based scheduler for its clusters of thousands of NVIDIA H100, B200 and B300 GPUs with a system of GPU time budgets, hierarchical fair-share allocation and a time-slicing contract. With requests for two to three times more GPUs than it has, Ai2 says the change turned decisions about how much compute each research project gets into a transparent budgeting process while keeping clusters fully occupied.
Source: Hugging Face @allenai - Official
Ai2 open-sources AstaBrief 8B, generating cited research reports in a single pass
Ai2 published this on Oct 2, 2026 (source: allenai.org).
Ai2 open-sourced AstaBrief 8B on October 2: built on Qwen3-8B, it turns a question plus literature excerpts into a cited research report in one pass, without retrieval. It now powers the Fast mode of Asta reports, averaging 51.1 seconds — 3.5x faster than the Thinking mode, per Ai2 — and weights and training data are released under Apache 2.0.
Source: Ai2