Daily AI news, sourcedWhere it stands
As of Oct 19, AuditKIT, with code on Lexsi Labs' GitHub, stores benchmark accuracy, judge-based scores, runtime performance, tool-use behavior and retrieval quality in a single result representation, with direction-aware metric contracts and configuration fingerprints.
- Official
AuditKIT, a source-available evaluation library from Lexsi Labs, puts benchmark, judge, runtime, tool-use and retrieval results in one format
Posted on arXiv on Oct 19, 2026 (source: arxiv.org).
AuditKIT, with code on Lexsi Labs' GitHub, stores benchmark accuracy, judge-based scores, runtime performance, tool-use behavior and retrieval quality in a single result representation, with direction-aware metric contracts and configuration fingerprints. For agents it matches tool calls turn by turn, separates parallel from sequential execution and keeps verified task outcomes apart from trajectory diagnostics, marking missing evidence as unscored rather than failed. The authors present seven case studies that they say show how answer extraction, runtime behavior and metric choice change evaluation outcomes. The paper has not yet been peer-reviewed.
Source: arXiv cs.CL
- Official
- An institution announces something that has already happened, through its own channel or a founder or executive account.
- Statement by …
- A named person's views, predictions, plans or comments, including remarks reported by media when the original is unavailable.
- Reported by …
- Only media or another third party is saying it. We name the outlet.
- Unconfirmed
- Circulating online (posts, comments, leaks). No company, named person or outlet has confirmed it.
Items older than 72 hours never go in Top stories and show their original date.