In a groundbreaking achievement, researchers from Stanford and the University of Washington have developed a competitive AI reasoning model, named s1, in just 26 minutes for under $50! A recent paper details the innovative training process, which utilized a small dataset of only 1,000 questions, showcasing the efficiency of their approach.
The team employed a technique called distillation, allowing the s1 model to learn from Google’s Gemini 2.0 Flash Thinking Experimental model, despite Google’s restrictions against creating competing models. With only 16 Nvidia H100 GPUs, the researchers initially experimented with a larger dataset of 59,000 questions but found that trimming it down yielded better results. ✨
The s1 model also features a unique test-time scaling method, enabling it to “think” longer before answering, which enhances its reasoning accuracy. The researchers claim that s1 outperforms OpenAI’s o1-preview on math questions by up to 27%!
This development signals a potential shift in the AI landscape, suggesting that smaller, cost-effective models could challenge the dominance of tech giants like OpenAI and Google, who typically invest billions in AI training.
