New CLE benchmark to assess continual learning in LLM agents
In brief
- CLE measures LLM agents’ improvement through experience across 600 levels spanning 120 tasks.
- Diagnostic feedback and revision yield a median gain of 5.3 percentage points.
- The best system clears only 47.7% of levels, revealing current limits of continual learning.
Researchers introduce Continual Learning Exam (CLE), a 600-level benchmark designed to evaluate how LLM agents improve through accumulated experience in realistic professional workflows. According to the announcement, CLE covers 120 tasks split into five levels each, spread across 89 professional roles and 12 occupational domains.
The evaluation compares two setups: a single-submission approach and one with diagnostic feedback and revision after failure. With feedback, models achieve a median gain of 5.3 percentage points, but the best system clears only 47.7% of levels and completes all five steps in just 23.3% of tasks, highlighting current limitations in continual learning for frontier LLMs.
What it changes for you
- EducationEducational institutions could use CLE to design adaptive learning paths that measure long-term learner progression.
- Public SectorPublic administrations could assess agents automating complex administrative processes and improving through accumulated experience.
This summary gives you the essentials. The full article, its figures and quotes are on Hugging Face Daily Papers.
Read the original articleAI-generated summary of the source, checked by rules. How we write · Report an error