OpenAI launches new benchmark to test AI on everyday tasks
What's the story
OpenAI has launched LifeVal, a novel benchmark to evaluate how well artificial intelligence (AI) models can assist in completing everyday tasks. The tool was announced on Monday and is aimed at assessing AI systems based on their practical utility rather than just academic knowledge or performance on traditional benchmarks. LifeVal covers a wide range of activities, from finding childcare and managing finances to handling government paperwork.
Benchmark utility
LifeVal focuses on real-world tasks and decisions
OpenAI has been using LifeVal for a year to monitor and improve its models.
The benchmark is designed to evaluate how well these models can assist in tasks people face every day.
These tasks include budgeting, recipe modification, grocery ordering, and workout planning.
The goal is to see if AI can help with decisions and responsibilities that matter in people's lives.
Research process
Creating LifeVal: Researching time use across demographics
To create the benchmark, OpenAI studied two datasets: the American Time Use Survey and the Multinational Time Use Study.
These studies helped identify activities people spend their time on, how they differ across demographic groups, and where AI assistance could be beneficial.
The American Time Use Survey tracks how Americans spend their time on paid work, childcare, volunteering, and socializing.
Global insight
Supplementing data with new survey responses
The Multinational Time Use Study compiles over a million diary days from more than 100 randomly sampled national-scale surveys into a common format.
This dataset enables researchers to study how people allocate their time between work, leisure, and other activities across 30 countries over a 55-year span.
OpenAI supplemented these datasets with a new demographically representative survey of 3,995 respondents.
Task categories
Defining focus areas and tasks for benchmark evaluation
The survey responses helped define the benchmark's focus areas, which include caregiving, health, education, household administration, personal finances, shopping, and accessing government services.
Some of the tasks covered by LifeVal are finding childcare, preparing budgets, adapting recipes to meet specific requirements (like dietary restrictions), ordering groceries, and organizing exercise routines.
The benchmark is meant to assess whether AI models can help people navigate decisions and responsibilities that matter in their daily lives.