#ai-benchmarking

[ follow ]
#artificial-intelligence #open-source #legal-ai #specialized-language-models #legal-reasoning
fromLawSites
1 week ago

AI Legal Research Startup Descrybe Launches 'Legal Reasoning' Tool; Says It Outperforms ChatGPT, Claude, and Gemini on Bar Exam Benchmark

DescrybeLM answered all 200 correctly. The general-purpose models each missed between 13 and 23 questions, achieving accuracy rates ranging from 88.5% to 93.5%. Rubric-scored reasoning quality - a separate measure evaluating whether systems correctly identified governing legal rules and applied them to the facts - followed a similar pattern. DescrybeLM scored 99.70% on that dimension.
Artificial intelligence
Artificial intelligence
fromFuturism
3 weeks ago

AIs Controlling Vending Machines Start Cartel After Being Told to Maximize Profits At All Costs

Claude Opus 4.6 substantially outperforms competitors at managing a simulated vending business, achieving higher profits and using aggressive market strategies including price coordination.
Artificial intelligence
fromZDNET
1 month ago

Worried AI will take your remote job? You're safe for now, this study shows

AI models performed significantly worse than human remote freelancers on diverse, complex work tasks, though AI capabilities continue to improve.
#artificial-intelligence
fromwww.scientificamerican.com
3 months ago
Artificial intelligence

Each Time AI Gets Smarter, We Change the Definition of Intelligence

Definitions and benchmarks of humanlike intelligence shift as AI improves, complicating contractual rights and independent verification of any AGI claim.
fromFuturism
9 months ago
Artificial intelligence

Apple Researchers Just Released a Damning Paper That Pours Water on the Entire AI Industry

Apple researchers question the reasoning capabilities of leading AI models, calling current industry claims an 'illusion of thinking'.
Artificial intelligence
fromFuturism
9 months ago

Apple Researchers Just Released a Damning Paper That Pours Water on the Entire AI Industry

Apple researchers question the reasoning capabilities of leading AI models, calling current industry claims an 'illusion of thinking'.
more#artificial-intelligence
Artificial intelligence
fromTechCrunch
5 months ago

OpenAI says GPT-5 stacks up to humans in a wide range of jobs | TechCrunch

GDPval benchmark evaluates AI models across 44 occupations in nine GDP-contributing industries; GPT-5-high matched or exceeded experts 40.6% of the time.
Artificial intelligence
fromInfoQ
5 months ago

Kaggle Introduces Game Arena to Benchmark AI Models in Strategic Games

Kaggle and Google DeepMind launched Kaggle Game Arena to benchmark AI decision-making by running all-play-all strategy game competitions with open-source environments.
fromTechCrunch
9 months ago

LM Arena, the organization behind popular AI leaderboards, lands $100M | TechCrunch

LM Arena has become an essential crowdsourced benchmarking project for AI labs, raising $100 million in seed funding to further its mission of evaluating AI models.
Artificial intelligence
Artificial intelligence
fromTechRepublic
10 months ago

OpenAI's o3: AI Benchmark Discrepancy Reveals Gaps in Performance Claims

The performance of OpenAI's o3 model on benchmarks significantly differed from earlier claims, revealing the complexity and variability in AI evaluations.
Artificial intelligence
fromTechCrunch
10 months ago

AI benchmarking platform Chatbot Arena forms a new company | TechCrunch

Chatbot Arena is establishing a company to elevate its AI benchmarking capabilities while ensuring impartiality in its evaluations.
fromtechcrunch.com
10 months ago

Debates over AI benchmarking have reached Pokemon

Last week, a post on X claimed Google's Gemini model surpassed Anthropic's Claude model in Pokemon, stirring controversy over AI benchmarks and implementation.
Artificial intelligence
[ Load more ]