Over the past few months, tech execs like Elon Musk have touted the performance of their company’s AI models on a particular benchmark: Chatbot Arena. Maintained by a nonprofit known as LMSYS, Chatbot ...

Microsoft has unveiled a groundbreaking benchmark called Windows Agent Arena (WAA) to test artificial intelligence agents in realistic Windows operating system environments. This new platform aims to ...

A new paper from AI lab Cohere, Stanford, MIT, and Ai2 accuses LM Arena, the organization behind the popular crowdsourced AI benchmark Chatbot Arena, of helping a select group of AI companies achieve ...

Join our daily and weekly newsletters for the latest updates and exclusive content on industry-leading AI coverage. Learn More LMSYS organization launched its “Multimodal Arena” today, a new ...

The rapid proliferation of AI chatbots has made it difficult to know which models are actually improving and which are falling behind. Traditional academic benchmarks only tell you so much, which has ...

Meta, Google, and OpenAI allegedly exploited undisclosed private testing on Chatbot Arena to secure top rankings, raising concerns about fairness and transparency in AI model benchmarking. A handful ...