UserArena: Interactive benchmark for long-horizon shopping agents
In brief
- UserArena offers 64,729 real session trajectories with 13 action types.
- The benchmark allows offline validation against production A/B tests.
- OPTD improves simulator fidelity for long e-commerce interactions.
Researchers unveil UserArena, an interactive e-commerce benchmark built on real-world data. According to the announcement, it includes 64,729 session trajectories from 11,010 users, covering 566,102 items and 13 action types. The tool provides an interactive environment and enables offline validation against production A/B test results.
The benchmark evaluates five performance levels: output validity, local decisions, sequential behavior, personalization, and outcomes. The authors also test On-Policy Trajectory Distillation (OPTD), a method to enhance simulator fidelity over long interactions.
What it changes for you
- Retail & E-commerceTechnical teams can now evaluate and improve shopping agents using real-world data and metrics aligned with production outcomes.
This summary gives you the essentials. The full article, its figures and quotes are on Hugging Face Daily Papers.
Read the original articleAI-generated summary of the source, checked by rules. How we write · Report an error