NewsBytes
    Hindi Tamil Telugu
    More
    In the news
    Narendra Modi
    Amit Shah
    Box Office Collection
    Bharatiya Janata Party (BJP)
    OTT releases
    Hindi Tamil Telugu
    NewsBytes
    User Placeholder

    Hi,

    Logout

    India
    Business
    World
    Politics
    Sports
    Technology
    Entertainment
    Auto
    Lifestyle
    Inspirational
    Career
    Bengaluru
    Delhi
    Mumbai

    Download Android App

    Follow us on
    • Facebook
    • Twitter
    • Linkedin
    Home / News / Technology News / Real-world data for AI training exhausted? Elon Musk thinks so
    Summarize
    Next Article
    Real-world data for AI training exhausted? Elon Musk thinks so
    Musk made the comments during a recent livestream

    Real-world data for AI training exhausted? Elon Musk thinks so

    By Akash Pandey
    Jan 09, 2025
    12:30 pm

    What's the story

    Elon Musk, owner of AI firm xAI, has agreed with other AI experts that the pool of real-world data to train AI models is almost empty.

    Speaking during a live-streamed discussion with Stagwell Chairman Mark Penn, Musk said, "We've now exhausted basically the cumulative sum of human knowledge... in AI training. That happened basically last year."

    This comes in line with former OpenAI Chief Scientist Ilya Sutskever's claim at NeurIPS, a machine learning (ML) conference last December.

    New direction

    What is the future of AI training?

    Sutskever had hinted that the AI industry has hit a point of "peak data," and this scarcity will require a change in the way models are being developed today.

    Musk supported this notion, offering synthetic data—information generated by AI models themselves—as the answer.

    He said, "The only way to supplement [real-world data] is with synthetic data, where the AI creates [training data]. With synthetic data ... [AI] will sort of grade itself and go through this process of self-learning."

    Industry shift

    Tech giants turn to synthetic data for AI training

    Several tech giants, including Meta, Microsoft, OpenAI, and Anthropic, are already using synthetic data to train their flagship AI models.

    According to Gartner's research, 60% of the data used for AI and analytics projects in 2024 were synthetically generated.

    This trend is already visible in Microsoft's Phi-4 and Google's Gemma models, both trained on a combination of real-world and synthetic data.

    Cost and risk

    Synthetic data: A cost-effective alternative with potential risks

    The use of synthetic data also comes with financial benefits. AI start-up Writer claimed that its Palmyra X 004 model, trained mostly on synthetic sources, only cost $700,000 to develop. That's way lower than the estimated $4.6 million for a similarly-sized OpenAI model.

    However, researches suggest potential risks of synthetic data like model collapse—where a model's outputs become less "creative" and more biased over time due to the inherent biases and limitations in the training data used by these models.

    Facebook
    Whatsapp
    Twitter
    Linkedin
    Related News
    Latest
    Elon Musk
    Meta
    Microsoft
    OpenAI

    Latest

    IPL 2025, RCB: Squad analysis, schedule, Probable XI, and more  Bhuvneshwar Kumar
    Is NEP's 3-language policy Hindi imposition? Tamil Nadu thinks so MK Stalin
    IPL 2025: PBKS to finish as table-toppers, predicts Shashank Singh Punjab Kings (PBKS)
    Khushdil Shah fined by ICC, receives three demerit points: Details Khushdil Shah
    Dale Steyn hails Bumrah, Rabada as complete bowlers in T20s  Dale Steyn
    Samay Raina issued second summons in 'India's Got Latent' row Canada
    Conan O'Brien to return as Academy Awards host in 2026 Academy Awards
    Tamannaah joins Ajay Devgn, Sanjay Dutt in 'Ranger'  Ajay Devgn
    Peruvian fisherman survives 95 days adrift in Pacific Ocean Peru
    iPhone 17 Air, Ultra to replace Plus, Pro Max variants Apple

    Elon Musk

    France warns Trump against threatening EU's 'sovereign borders'  Brexit
    SpaceX Starship's 7th test flight delayed to 'sometime next week' SpaceX
    Is Elon Musk going mad? His biographer certainly thinks so United States of America
    'Your posts could get someone killed': TED chief to Musk X

    Meta

    Why Meta will show eBay listings on Facebook Marketplace Facebook
    Mark Zuckerberg's $900K 'Hand Made 1' watch grabs attention Mark Zuckerberg
    Meta replaces fact-checking program with X-like 'Community Notes' system Mark Zuckerberg
    UFC's Dana White joins Meta's board ahead of Trump's inauguration Ultimate Fighting Championship (UFC)

    Microsoft

    Microsoft rolls back Bing Image Creator update following quality complaints OpenAI
    Microsoft to lay off underperforming employees in workforce restructuring move Satya Nadella
    Microsoft aims to train 10M Indians in AI by 2030 Satya Nadella
    Microsoft announces $3B investment in India's AI and cloud ecosystem Narendra Modi

    OpenAI

    OpenAI's Sam Altman faces sexual abuse lawsuit filed by sister Sam Altman
    Google DeepMind to develop AI that will simulate real world Google
    OpenAI is losing money on ChatGPT's $200/month Pro plan: Altman ChatGPT
    AI agents to join workforce in 2025: OpenAI CEO Sam Altman
    Indian Premier League (IPL) Celebrity Hollywood Bollywood UEFA Champions League Tennis Football Smartphones Cryptocurrency Upcoming Movies Premier League Cricket News Latest automobiles Latest Cars Upcoming Cars Latest Bikes Upcoming Tablets
    About Us Privacy Policy Terms & Conditions Contact Us Ethical Conduct Grievance Redressal News News Archive Topics Archive Download DevBytes Find Cricket Statistics
    Follow us on
    Facebook Twitter Linkedin
    All rights reserved © NewsBytes 2025