A benchmark for evaluating AI agents on research workflows across scientific domains
Does Code Cleanliness Affect Coding Agents? A Controlled Minimal-Pair Study
As autonomous coding agents see rapid adoption, their evaluation has primarily focused on task completion rates holding the target codebase fixed. This leaves a critical question unanswered: does the structural and stylistic quality, or ``cleanliness'' of the underlying code affect an agent's ability to navigate and modify it? To isolate the effect of code cleanliness from agent capability, we introduce an evaluation protocol built around minimal pairs: repositories that match on architecture, dependencies, and external behaviour, but differ on static-analysis rule violations and cognitive complexity. The pairs are constructed in both directions, by agent pipelines that either degrade a clean repository or clean a messy one. We author 33 tasks across six such pairs, evaluated through hidden tests at the application's public surface. Across 660 trials with Claude Code, code cleanliness does not change the agent's pass rate. However, it substantially alters the agent's operational footprint: agents working on cleaner code use 7 to 8% fewer tokens and reduce file revisitations by 34%. Our findings suggest that traditional maintainability principles remain highly relevant in the era of AI-driven development, shaping the computational cost and navigational efficiency of coding agents. Code cleanliness joins model choice, harness, and prompting as a factor that materially affects agent behaviours.
The Eternal Sloptember
I’m calling it now, the adoption of AI agents into software development will be one of the most costly mistakes in the field’s history. Agents cannot program, and it’s taking longer and longer to realize that they can’t. They are a highly sophisticated statistical model designed to mimic the distribution of programming. The output is broken, but in a way that’s getting harder and harder to detect. Which is exactly what you’d expect from an increasingly accurate statistical model.
PrimeIntellect-ai/prime-agent
A self-improving RLM agent for coding workflows and long-running autonomous tasks.
AI Index | Stanford HAI
The mission of the AI Index is to provide unbiased, rigorously vetted, and globally sourced data for policymakers, researchers, journalists, executives, and the general public to develop a deeper understanding of the complex field of AI. To achieve this, we track, collate, distill, and visualize dat
The Scientific Contribution Graph: Automated Literature-based Technological Roadmapping at Scale
Sir Isaac Newton famously wrote, “If I have seen further, it is by standing on the shoulders of giants”. Scientific contributions are rarely developed in isolation, but build upon prior contributions, such as problem framings, experimental methods, and empirical findings. Understanding these prerequisite relationships is important for studying scientific progress, and for automated scientific discovery systems that must reason about which existing capabilities can be used to develop new ones (e.g. Lu et al., 2024; Jansen et al., 2025b; Baek et al., 2025).
The AI "Evaluation Crisis" Is an Opportunity to Get Data Flow Right
Why the AI evaluation crisis could force a reckoning on dataset provenance, attribution, and consent.
More Than a Tool
What We Learned from a Year of Conversations About AI
Cognition | Agent Trace: Capturing the Context Graph of Code
We’re excited to join in Cursor, Cloudflare, Vercel, git-ai, OpenCode and others in support of [Agent Trace](https://agent-trace.dev/). As described in the spec, Agent Trace is an open, vendor-neutral spec for recording AI contributions alongside human authorship in version-controlled codebases.
The 8 Levels of Agentic Engineering — Bassim Eledath
AI's coding ability is outpacing our ability to wield it effectively. That gap closes in levels — 8 of them. Here's the progression from tab complete to autonomous agent teams.
AI should help us produce better code - Agentic Engineering Patterns
AI should help us produce better code - Agentic Engineering Patterns
AI Tools Accelerates Coding, but Not Overall Software Delivery, GitLab Research Finds
GitLab's 2026 AI Accountability Report highlights an AI Paradox: although 78% of developers say they code faster, overall software delivery has not accelerated due to downstream testing and review bottlenecks and new challenges for enterprise governance and traceability.
KARMA: Leveraging Multi-Agent LLMs for Automated Knowledge Graph Enrichment
Maintaining comprehensive and up-to-date knowledge graphs (KGs) is critical for modern AI systems, but manual curation struggles to scale with the rapid growth of scientific literature. This paper presents KARMA, a novel framework employing multi-agent large language models (LLMs) to automate KG enrichment through structured analysis of unstructured text. Our approach employs nine collaborative agents, spanning entity discovery, relation extraction, schema alignment, and conflict resolution that iteratively parse documents, verify extracted knowledge, and integrate it into existing graph structures while adhering to domain-specific schema. Experiments on 1,200 PubMed articles from three different domains demonstrate the effectiveness of KARMA in knowledge graph enrichment, with the identification of up to 38,230 new entities while achieving 83.1% LLM-verified correctness and reducing conflict edges by 18.6% through multi-layer assessments.
Coasts — Containerized Hosts for AI Agents
Free, open source parallel runtimes for AI agents. Run multiple isolated environments on your machine — no cloud, no conflicts.
Track, control, and optimize for AI agents and bots | Known Agents (formerly Dark Visitors)
Use Known Agents to turn the rising wave of AI agents, LLM assistants, and other bots crawling your website into a new growth channel for your business
