Mira Murati's AI start-up unveils model built for real-time interactions
What's the story
Thinking Machines Lab, an artificial intelligence start-up founded by Mira Murati, has unveiled a new category of AI systems called "interaction models." The innovative technology is aimed at solving a major problem with current AI systems: the long pause between asking something and getting a response. The new multimodal AI systems are designed for real-time communication, processing audio and visual input simultaneously to enable continuous reactions and reduce response latency.
Technological leap
The tech behind the interaction models
The tech behind these interaction models is a new architecture based on "full-duplex" interaction. This system breaks communication into micro-turns of about 200 milliseconds, allowing the AI to continuously react to visual and auditory input even while speaking. The main model, TML-Interaction-Small, is a 276-billion parameter mixture-of-experts model focused on fast conversational handling and immediate responses. It works with a secondary asynchronous "background" model that handles more complex tasks like reasoning or web searches.
Innovative technique
Encoder-free early fusion for faster processing
Thinking Machines Lab's interaction models also use a novel technique called "encoder-free early fusion." This method processes raw audio and visual signals directly through lightweight embedding layers within the transformer, avoiding heavy external encoders usually used for audio and video understanding. The result, according to the company, is significantly lower latency. On FD-bench, a benchmark focused on interaction quality and conversational timing, TML-Interaction-Small achieved response latency below 0.4 seconds.
Practical applications
Potential applications in various industries
The real-time interaction model could be a game changer for enterprises and industrial use cases. It could monitor video feeds in real time and react instantly, detecting abnormalities or safety violations as they happen. In customer service, lower latency could make AI conversations feel more natural and less transactional. The model also has a built-in sense of time awareness, enabling contextual instructions without manually specifying timestamps or measurements.