ScholarEvolve lifts Qwen3.5-27B task completion to 81.4% without retraining
Researchers at the University of California, Santa Barbara and Microsoft developed ScholarEvolve, which turns techniques from academic papers into code upgrades for AI-agent harnesses without changing model weights. VentureBeat reports that it raised Qwen3.5-27B’s task completion on AppWorld Normal from 69.0% to 81.4%; on tau-Bench Telecom, the researchers’ framework lifted GPT-5.4-mini from 72.7% to 81.9%.
Bottom line — ScholarEvolve improved Qwen3.5-27B by 12.4 percentage points, but VentureBeat notes its generated code needs security checks before production use.
Go deeper 5
-
VentureBeat says ScholarEvolve searches arXiv for methods to address failures, then implements and tests upgrades across five harness modules.
-
The Qwen3.5-27B harness rose from 49.6% to 63.6% on AppWorld tasks involving unseen APIs, according to VentureBeat.
-
Upgrades included syntax checks for tool calls, relevance-based context budgeting and reusable pseudocode templates for API sequences.
-
VentureBeat reports the researchers tested three rounds of literature updates without changing model weights or previously collected task data; production adaptation was not tested.
-
The repository linked by the researchers was empty at VentureBeat’s publication, and the article says the framework did not describe security scanning for retrieved code.